Senior Data Engineer
ASCAP
New York, NYThis was removed by the employer on 8/9/2019 12:13:00 PM PST
Not to worry we have many other jobs on the site;
Browse all jobs
Browse the IS/IT Category
Search for Senior Data Engineer jobs in New York-NY
Search all Senior Data Engineer postings
Full Time Job
Senior Data Engineer (contract)
Job Description
ASCAP is home to more than 700,000 music creator members across all genres - the greatest names in music, and thousands more in the early stages of their careers. We are the world leader in performance royalties, advocacy and service for music creators, and are the only PRO in the US run by its members including songwriters, composers and music publishers.
ASCAP technologists live our mission and we are passionate about what we do for our customers and we practice what we preach. Our technologists serve with humility and a deep respect for their responsibility in helping our business partners and members achieve their goals and realize their dreams. We have an infectious and lively culture and we recognize our successes monthly at our Thursday on-site social hour celebrations. We stand behind our mission and are committed to delivering the impossible.
Bottom line? We outthink ordinary. Discover what you can do with technology at ASCAP!
We're looking for a expert big data and cloud engineer to build data solutions for our music analytics team in NYC. Do you have experience transforming and moving data in both on-prem and cloud big data platforms? Are you experienced in relational and dimensional modeling? Do you want challenging work that gives you the opportunity to build solutions on a diverse technology stack and own what you build? If so, this might be the role for you.
The Senior Data Engineer is responsible for processing structured and unstructured data, validating data quality, and developing and supporting data products. We are looking for someone with strong hands on experience in all layers of the full stack involving data. We especially need your experience with Cloudera Hadoop, Data warehousing and AWS Cloud technologies. The Data Engineer plays a significant role in Agile planning, providing advice and guidance, and monitoring emerging technologies. This is not a junior programmer position and requires extensive hands on coding and design experience.
Technology We Use
• Cloudera Hadoop, Impala, Hive, Kudu, Yarn, Hue, Spark
• Python, Java, Scala, Unix shell scripting
• AWS, EMR, Snowflake
• Jenkins, Terraform, and CloudFormation
Your Role
• Design, code, test, correct and document programs and scripts using agreed standards and tools to achieve a well-engineered result
• Derive an overall strategy of data management, within an established information architecture (including both structured and unstructured data), that supports the development and secure operation of existing and new information and digital services
• Plan effective data storage, security, sharing and publishing within the organization
• Gather and process raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks
• Ensure data quality and implement tools and frameworks for automating the identification of data quality issues
• Collaborate with internal and external data providers on data validation providing feedback and making customized changes to data feeds and data mappings
• Mentor and lead data engineers providing technical guidance and oversight
• Provides ongoing support, monitoring, and maintenance of deployed products
Qualifications
• 5 – 8 years of development experience at an Enterprise level with at least two of these languages: Python, Java, or Scala
• Experience in a Dev Ops environment including: Jenkins, Terraform, and CloudFormation or similar technologies
• Experience working with high volume heterogeneous data in Cloudera Hadoop
• Business objects, Tableau and experience with batch and real-time data processing frameworks
• Strong background with data modelling, data access, and data storage techniques
• Strong Experience with relational databases like Oracle, MS SQL server and DynamoDB
• Working knowledge of Salesforce, Red Shift, Athena, Snowflake and MySQL especially desired
• Experience in relational and dimensional modeling (star schema, snowflake schema, data marts)
• Extensive hands on experience writing complex SQL queries involving multi table joins.
• Bachelor's degree in Computer Science or related field or equivalent combination of industry related professional experience and education
What We Look For
• Strong hands on experience in all layers of the full stack involving data, front end experience
• Expert level knowledge on Cloudera Hadoop and allied technologies.
• AWS Cloud Technologies, Unix, Master level SQL writing skills
• Cloud components including: EMR, Lambda, Glue, VPC Security groups, EC2, SNS, and S3
• Experience streaming datasets, Visualizations with Tableau, Sqoop, Data stage
• Excellent problem-solving skills, willingness to take ownership and risks, and enthusiasm in the face of technical challenges
• Active participation in the engineering community, up to date knowledge on new data technologies and best practices
What We Love About You
• You love our users. You deeply understand our users and put them at the center of everything you do. You aim to serve and delight them every day.
• You do the right thing. You are respectful and act with the highest integrity. If you see something that isn't right, you say something.
• You debate it. You ask questions to understand a perspective and are comfortable respectfully challenging assumptions. You are not turned off by constructive conflict to get to the right answer.
• You own your outcomes. You set clear ambitious goals. You anticipate obstacles, persevere, and are accountable for your commitments.
• You make fast decisions. You are an effective and timely communicator. You understand how to collaborate, compromise, and escalate when needed.
• You get better every day. You welcome the gift of feedback. You never settle in your quest to grow and develop. By being here, you make our company stronger.