Description

Key Responsibilities:

  • Develop and implement test strategies, test cases, and automation scripts to validate data pipelines in Azure and Databricks environments.
  • Perform data validation, reconciliation, and comparative analysis between source and target systems.
  • Validate ETL/ELT pipelines built using ADF and Databricks.
  • Collaborate with Data Engineers and Product Owners to understand STM (Source-to-Target Mapping) and ensure transformation logic is correctly implemented.
  • Monitor and validate data quality across Delta tables, and Data Warehouses.
  • Identify data anomalies, document defects, and drive them to resolution with the engineering team.
  • Support CI/CD pipelines by integrating data testing into DevOps workflows.
  • Contribute to test data management, metadata validation, and regression testing.
  • Provide regular reporting on test execution results, defect metrics, and QA health.

 

Required Skills:

  • Proven experience in data QA/validation in cloud-based data platforms.
  • Strong knowledge of Azure Data Factory, Databricks.
  • Proficiency in SQL and scripting languages such as Python.
  • Hands-on experience with data profiling, data reconciliation, and schema validation.
  • Understanding of SCD Type 2 and data transformation logic.
  • Familiarity with DevOps tools like Azure DevOps or GitHub Actions for CI/CD integration.
  • Experience working with large datasets, performance testing, and data lineage tools

Key Skills
Education

Any Graduate

Back To Jobs