Data Engineer

This role is to build very large, scalable platforms using cutting edge data technologies, from the ground up and plan to leverage the most recent Big Data technologies, working closely with product engineering teams and data scientists. 

Responsibilities:

1. Design and implement data infrastructure and processing workflows required to support data science, machine learning, BI and reporting 
2. Build robust, efficient and reliable data pipelines consisting of diverse data sources
3. Design and develop real time streaming and batch processing pipeline solutions
4. Own the data expertise and data quality for the pipelines
5. Drive the collection of new data and refinement of existing data sources

Requirements:

1. A minimum of 2 years of experience in data engineering, ETL or work in SQL
2. Bachelor's degree in Computer Engineering or a related field
3. Proficiency in Python, Java, Scala, or any two.
4. Strong command of SQL
5. Proficient understanding of distributed computing principles
6. Experience with schema design and dimensional data modeling
7. Experience working with either a Map Reduce or a MPP system on any size/scale
8. Knowledge of NoSQL databases will be a plus
9. Knowledge of Talend 
10. Good understanding of Lambda / Kappa Architecture, along with its advantages and drawbacks.
11. Familiarity or experience in Cloudera/Hortonworks
12. Experience with multiple data technologies and concepts such as Airflow, Kafka, Hadoop, Hive, Spark, MapReduce, SQL, NoSQL, and Columnar databases.