Whalerock IndustriesWest Hollywood, CA
Full Time Job
Whalerock Industries is looking for a highly motivated data engineer who can build an end-to-end data pipeline spanning various data sources and data destinations.
You are details oriented, efficient, and care about the organization of data; yet keep an eye on the bigger picture that drives the business/product goals for. You also know to take a disparate set of data, in a variety of formats and create database tables and views to bring harmony to all of our business critical metrics and information.
You have moved large-scale data from databases, REST end-points, message-queues into relational databases. You have not just moved them once, but you know how to schedule these data-pipeline applications incrementally, with de-duping, with aggregations and integrations, with ability to heal seamlessly from the point of last failure.
You have experience working in various business/product verticals from internet startups to entertainment and web focused properties. You live and breathe concepts like de-duping, map-reduce, incremental loads, idempotency, J/XPATH processing, columnar databases, message queues, anything AWS, API calls, and with a passion to always learn more.
We value initiative and ability to work with minimal supervision, but we value equally the ability to follow direction and be part of the team. We thrive in a culture of respect and building technology together. We learn from each other.
The ideal candidate…
• Analyze data using Python, SQL, BASH, Docker and other necessary libraries, tools, and technologies.
• Design and implement end-to-end automated Data pipelines to store data in Amazon Redshift.
• Implement API data pulls/pushes from a wide variety third party data vendors including Facebook, Amazon, Apple, and others.
• Work with existing ETL frameworks in an effort to maintain and extend them with best practices.
• Monitor and troubleshoot existing ETL processes.
• Respond to critical Data support issues in a timely manner with some on-call support for critical job failures.
• Passionate about data.
• Detail-oriented and efficient.
• Fundamentally an software engineer that enjoys creating tools that empower people to make better decisions.
• Excellent working knowledge of SQL and schema/table architecture.
• Knowledgeable of concepts to optimize your code for performance and maintainability.
• 3 years experience building advanced end-to-end data pipelines using Python and SQL.
• 2 years working on SQL based databases.
• Experience with AWS cloud offerings for Data applications.
• Experience working with HTTP/REST API frameworks.
• Cross team communication skills.
• Experience with Google products especially Google Cloud Storage, Google-Analytics and Google BigQuery.
• Experience with Amazon Redshift or minimally PostgreSQL.
• Working knowledge of most of the following: Pandas, Boto, SQLAlchemy, awscli, gsutil, and Jupyter.
• Experience containerizing applications using Docker and writing shell scripts in BASH.
Success at Whalerock Industries requires that…
• You have a sense of humor.
• You like getting your hands dirty.
• You love solving problems and helping people to the point they smile.
• You are entrepreneurial, smart, and passionate.
• You know the acronyms but speak in plain English.
• If you don't know something, you say so. Then go figure it out quickly.
• You naturally go the extra mile.
• You thrive in a fast-paced work environment, readily adapting to continually changing priorities and aggressive project timeframes.
• You give credit to others where credit is due and take responsibility for misses when they rest with you.
Please no third parties or vendors.