Data Engineer
A+E Networks
New York, NYThis was removed by the employer on 3/11/2020 6:38:00 AM PST
Not to worry we have many other jobs on the site;
Browse all jobs
Browse the IS/IT Category
Search for Data Engineer jobs in New York-NY
Search all Data Engineer postings
Full Time Job
A Technical contributor who brings hands-on knowledge of media data and all phases in building large-scale cloud based distributed data processing systems and applications to Build and Maintain Data/Analysis Infrastructure, and Data Science models for AMRLD audience data, audience engagement data (social, web, email, mobile, etc.), and other consumer data.
This role requires partnering closely with a team of data scientists, business analysts & data engineers leading the cloud-based Big Data & Analytics strategy . The successful candidate will implement complex AWS-based big data projects with a focus on collecting, parsing, managing, analyzing, querying, and visualizing large sets of data to turn information into insights using multiple technology platforms.
This role also requires an understanding of how a secure big data cloud environment is architected to gain real insights faster, with less friction and complexity.
Key Functions:
The successful candidate will perform the following tasks:
• Lead the design, implementation, and continuous delivery of pipelines using distributed AWS-based big data technologies supporting data processing initiatives across batch and streaming datasets
• Development and design using Python language and Big Data Frameworks such as Spark, AWS Athena, Presto, APIs, etc.
• Create Use Case analytics in our cloud data environment writing SQL, Python, and D3 queries to extract and visualize necessary data points on Nielsen's AMRLD
• Assist in building and maintaining Data Science models in AWS
• Providing administrative support on deployed AWS platform components
• Identifying, evaluating and implementing cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs
• Reviewing, analyzing and evaluating market requirements, business requirements and project briefs to design the most appropriate end-to-end technology solutions
• Processing and managing high volume real time customer interaction streams
• Providing architectural support by building Proof of Concepts & Prototypes
WHAT YOU'LL LEARN:
As the media industry grapples with fragmentation and disruption, a key facet of Research is to be the driving force in helping their company capture, report on, monetize, and understand audiences. An opportunity lies in Research to work across divisions and departments to create a unified language that ensures that internal clients understand-and take advantage of-complex metrics. The key is giving the necessary context so that decisions can be made. While this sounds disarmingly simple, it is a challenge to get the majority of the company to speak the same language and understand new vernacular. Research is a central force within the broader organization in gaining an objective perspective for a wide array of complex questions. It may sound academic – and certainly there is a portion of the job that requires a love of learning – but the primary attribute that will bring you success is curiosity. For example, you'll gain a deeper understanding of what motivates someone to tune into a show, buy a product based on placement of an ad, become a fan on social media, or why certain types of programs are successful across regions of this country and abroad. Ultimately, each research candidate gets a better understanding of the rigor that goes into making business decisions by uncovering not boulders, but pebbles along the path to connecting to A E Networks core asset, its content.
YOUR STORY: [ BEHAVIORS] (what you need to have)
The successful candidate will possess the following Skills and Experience:
• 8 years professional Software Industry experience in the Media Industry (Linear Television, required, Omniture Digital experience also highly desired)
• Extensive experience with Nielsen AMRLD
• Proficiency working with structured, semi-structured, and unstructured data sets including linear data, social, web logs and real time streaming data feeds
• Extensive experience with Social, Web, Email, and other audience engagement data
• 5 years development experience with AWS services - specifically collecting, processing and analyzing large volumes of data
• Must have hands-on experience with AWS: S3 / EMR / Data Pipeline / Lambda, EC2, RedShift, Snowflake, Cloud Formation and CLI experience
• Expert in Software development and design in Python
• 5 years' experience in SQL (must), and Python (must)
• Hands on experience with Apache Spark / Hadoop
• Extensive Knowledge and Use of Visualization D3/JavaScript (must), and Tableau (must)
• Able to tune Big Data solutions to improve performance and end-user experience
• Experience in Data Science (Machine Learning/AI, Tensor Flow, Cluster Analysis, Decision Trees, Combining multiple trees into an Ensemble model, GBM/xgboost models, calculating WAPE and R-squared values, etc.)
• Expert level usage with GitHub is preferred
• Spark Developer / AWS Developer certification is a plus
• Bachelor' degree or higher in Computer Sciences or similar
THAT SOMETHING EXTRA:
If you've read this far, you're likely a great fit for us…and maybe you're fluent in sign language, have studied the circus arts or are currently writing a screenplay. If you have a special skill or backstory that might directly or indirectly help you succeed in this role, we'd love to hear about it in your cover letter.