Data Engineer
ViacomCBS
New York, NYThis was removed by the employer on 11/18/2019 5:55:00 AM PST
Not to worry we have many other jobs on the site;
Browse all jobs
Browse the IS/IT Category
Search for Data Engineer jobs in New York-NY
Search all Data Engineer postings
Full Time Job
Overview and Responsibilities
The Data Solutions team sits within the larger Advanced Advertising organization at Viacom, supporting the entire Ad Sales organization across the Global Entertainment Group and Kids and Family clusters. At the core, we are a data-driven team that demonstrates data science to create strategic efficiencies and empower our ad sales teams with innovative offerings. We are passionate about developing data products that not only get utilized across the company, but also get talked about in the press for innovation
The Data Solutions team at Viacom is looking for a qualified Data Engineer. The selected candidate will be responsible for building, expanding, and optimizing our data pipeline architecture. They will also be responsible for optimizing data flows, maintaining, improving, cleaning, and manipulating data in operational and analytics databases. The Data Engineer will support our data scientists and analysts to deliver best-in-class analytics tools and data products to Viacom.
Responsibilities
• Create and maintain large scale data structures and pipelines to organize data for new and existing projects and data products
• Optimize data delivery by creating ETL processes and developing tools for real-time and offline analytic processing
• Build scalable infrastructure required for optimal ETL of data from a wide variety of data sources using SQL, NoSQL, AWS technologies, Spark, Python, Java and any of the major languages
• Participate in the assessment, selection, and integration process of current big data tools and frameworks required to satisfy business needs ensuring that all systems meet business objectives
• Integrates multi-formatted data from different sources, assuring that they adhere to high data quality standards and meet functional business requirements
• Design, construct and maintain disaster recovery procedures
• Monitoring process performance, advising on necessary infrastructure changes
• Recommend ways to constantly improve data quality and reliability
• Collaborate with data scientists, project managers, and other partners to integrate algorithms and models into automated processes
• Maintain required level of data security for each data set
Basic Qualifications
• 1-3 years of experience in a Data Engineer role
• Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
Additional Qualifications
• Experience with the following:
• Big data tools such as Spark and Kafka
• Big Data ML toolkits such as SparkML
• SQL and NoSQL databases
• Data pipeline and workflow management tools such as Airflow and Luigi
• AWS cloud services including EC2, EMR, RDS, Redshift
• Stream-processing systems such as Storm and Spark-Streaming
• Object-oriented/object function scripting languages: Python, Java, C , Scala, etc.
• Shell scripting and unix commands
• Proficient understanding of distributed computing principles
• Integration of data from multiple data sources
• Performing root cause analysis
• Knowledge of various ETL techniques
• Good understanding of Lambda Architecture
• Strong communication skills and experience working with cross-functional teams
• Strong problem solving skills and critical thinking ability