Data Engineer

We are seeking a skilled and certified Data Engineer with proven hands-on experience in Snowflake and Apache Spark. The ideal candidate holds a SnowPro Core or Advanced Certification and a Databricks Certified Associate Developer for Apache Spark, demonstrating their expertise in modern data platforms and distributed data processing. This role will involve building scalable data pipelines, optimizing data storage, and enabling real-time and batch data processing.

Key Responsibilities

 

• Design, develop, and maintain robust, scalable ETL/ELT pipelines using Apache Spark and Snowflake.
• Leverage Databricks for data processing, transformation, and analytics in distributed environments.
• Develop efficient SQL and Spark applications to process and analyze large volumes of data.
• Implement and maintain data warehousing solutions using Snowflake with best practices for performance, cost, and security.
• Collaborate with data scientists, analysts, and business stakeholders to meet data needs.
• Ensure data quality and integrity through unit testing, data validation, and monitoring.
• Optimize and troubleshoot Spark jobs, SQL queries, and Snowflake data workflows.
• Integrate with various data sources (cloud storage, APIs, RDBMS) and tools (Airflow, DBT, etc.).
• Apply data governance and compliance policies in data pipeline design and execution.