Description

Job Description:

Data Engineering Testing:

  • 5+ years of experience in QA automation for data engineering, ETL, or similar domains.
  • Experience with data testing tools and frameworks such as Great Expectations or similar.
  • Strong expertise in Python for test automation, with experience in libraries like PyTest or unittest.
  • Proficiency in SQL for data validation and querying.
  • Hands-on experience with testing large-scale data pipelines in data platforms like Azure Databricks (preferred) or Snowflake

Azure Ecosystem Expertise:

  • Experience with Azure Data Cloud - ADF, Synapse, ADLS Gen 2, Cosmos DB
  • Familiarity with Azure Event Hub or Kafka for validating streaming data pipelines.
  • Understanding of Azure Blob Storage and Azure Data Lake for data validation tasks.
  • Exposure to monitoring tools like Azure Monitor and Azure Log Analytics for analysing test results and pipeline performance.
  • Familiarity with data security and governance practices for Azure Databricks
  • Familiarity with modern data Lakehouse data formats - delta table, parquet
  • Knowledge of data Modeling concepts such as Star Schema, Data Vault and Medallion Architecture

Key Skills
Education

Any Graduate

Back To Jobs