Key Skills: ETL, Python, Data Bricks, SQL
Roles and Responsibilities:
- Design and implement end-to-end data pipelines using Python and Airflow.
- Develop high-performance SQL queries for data extraction, transformation, and reporting.
- Automate and modularize ETL workflows with Python scripting and reusable Airflow DAGs.
- Utilize AWS services (S3, Glue, Redshift, Aurora RDS, CloudWatch) for scalable data solutions.
- Integrate and transform data from multiple sources such as Alteryx, PostgreSQL, Oracle, and PL/SQL into unified models.
- Ensure data pipelines are scalable, secure, and optimized for analytics and reporting needs.
- Collaborate with cross-functional teams to support data-driven decision-making.
Skills Required:
- Must-Have:
- Strong proficiency in ETL development and Python programming.
- Experience in data pipeline design, automation, and orchestration (e.g., Airflow).
- Hands-on expertise in data integration and transformation.
- Nice-to-Have:
- Exposure to Data Bricks for advanced analytics and big data processing.
- Strong SQL skills, including performance tuning and query optimization.
- Experience working with AWS cloud services (S3, Glue, Redshift, Aurora RDS, CloudWatch).
- Soft Skills:
- Strong analytical and problem-solving skills.
- Ability to manage multiple priorities and deliver high-quality results under deadlines.
- Excellent communication and collaboration skills for cross-functional engagement.
Education: Bachelor's degree in Computer Science, Information Technology, or a related field