Description

Key Skills: Data engineering, Apache Airflow, GCP, BigQuery, GCS, SQL, ETL/ELT, Docker, Kubernetes, data governance, Agile, CI/CD, DevOps, pipeline orchestration, technical leadership.

Roles & Responsibilities:

  • Evaluate and provide scalable technical solutions to address complex and interdependent data processes.
  • Ensure data quality and accuracy by implementing data quality checks, data contracts, and governance processes.
  • Collaborate with software development teams and business analysts to understand data requirements and deliver fit-for-purpose data solutions.
  • Lead the team in delivering end-to-end data engineering solutions.
  • Design, develop, and maintain complex applications to support data processing workflows.
  • Develop and manage data pipelines and workflows using Apache Airflow on GCP.
  • Integrate data from various sources into Google BigQuery and Google Cloud Storage (GCS).
  • Write and optimize advanced SQL queries for ETL/ELT processes.
  • Maintain data consistency and troubleshoot issues in data workflows.
  • Create and maintain detailed technical documentation for pipelines and workflows.
  • Mentor junior data engineers and provide technical leadership and support.
  • Lead project planning, execution, and successful delivery of data engineering initiatives.
  • Stay updated with emerging trends and technologies in data engineering and cloud computing.

Experience Requirement:

  • 6-8 yeras of experience in leading the design, development, and deployment of complex data pipelines.
  • Strong working knowledge of Apache Airflow on GCP for orchestration.
  • Hands-on experience integrating data into Google BigQuery and GCS from various sources.
  • Proficient in writing and optimizing complex SQL queries for large-scale data processing.
  • Practical knowledge of containerization technologies like Docker and Kubernetes.
  • Experience in implementing data governance and adhering to data security best practices.
  • Familiarity with Agile methodology and working in cross-functional teams.
  • Experience with CI/CD pipelines and DevOps practices for data engineering workflows.

Education: B.Tech M.Tech (Dual), B.Tech, M. Tech

Key Skills
Education

Any Graduate

Back To Jobs