Required Skills (Mandatory): Data Engineer, Python, Spark/PySpark, ETL, AWS Services: EMR, Athena, S3, AWS Glue.
Job Summary:
- Minimum of 3-4 years project experience building optimized ETL pipelines
- Minimum of 3 years building Spark ETL pipelines. Ability to design, develop, and optimize Spark ETL pipelines
- Extensive experience with Python with familiarity with libraries and frameworks for data ingestion and transformation
- Strong command of SQL, with experience writing complex queries, optimizing performance, and working with databases
- Solid understanding and Experience with the following AWS Services: EMR, Athena, S3, AWS Glue
- Experienced in building and managing Airflow DAGs.