Big Data Developer
Discovery CommunicationsSterling, VA
Full Time Job
The position is for a senior Developer who has full technical knowledge of all phases of applications systems analysis and programming. This position is responsible for design and development for Bigdata and devises or modifies procedures to solve complex problems that require the consideration of operating systems capacity and limitations, operating time and form of desired results. Provides technical contribution in the development of detailed specifications from which programs will be written and perform design, coding, testing, debugging and documentation. This position will be required to frequently interact with other development personnel and would often have the responsibility for quality assurance review and evaluation of new or existing code set.
1. Design processes to extract, transform and load the source data to the target and Data lake.
2. Develop complex code to load data in to Data lake using AWS/EMR Spark and Scala
3. Perform unit and system testing on the processes and resulting data lake tables.
4. Create test cases and test the data.
5. Mentor others on the team with product functionality and provide guidelines and solutions on performance tuning techniques and industry best practices.
6. Understanding the existing environment, translating into new environment requirements and planning tasks to meet those requirements.
* Bachelor’s Degree in Engineering, Computer Science or related discipline;
* 5 years hands-on ''big data'' experience (Hadoop, HDFS, PIG, SPARK, Scala )
* Experience with cloud ''big data'' infrastructure (AWS EMR, Hortonworks HDP, Cloudera,S3)
* Experience creating and managing complex data architectures
* Experience with data warehousing databases Redshift, oracle and techniques .
* Thorough understanding of cloud computing;
* Extensive data analysis skills
* Understanding and experience with IAAS, PAAS, SAAS delivery models and the current competitive landscape.
* Manipulating, aggregating and deriving useful information from data stored across many sources and different databases
* Strong Java and Scala development skills
* Talend Big data experience is plus
* Must have the legal right to work in the United States
Sterling, Virginia, VA