Description:
- Bachelor's degree in Computer science or equivalent, with minimum 12+Years of relevant experience.
- Strong Experience of data governance and compliance standards
- Minimum of 5 years of experience in data engineering, data quality management, and data lake implementation.
- Proven experience with Databricks and Azure.
- Strong analytical and problem-solving skills.
- Excellent communication and presentation skills.
- Ability to work effectively in a fast-paced, dynamic environment.
Preferred Skills:
- Experience with other data quality tools and technologies.
- Certification in data management or data quality.
- Familiarity with machine learning and AI techniques.
Responsibilities:
Collaboration & Mitigation:
- Work with business teams and upstream application teams to mitigate data quality issues at the source.
- Collaborate with stakeholders to define and implement new data quality (DQ) business rules.
- Build exception handling and inline data quality processes within Databricks for the entire data lifecycle.
Data Profiling & Rule Definition:
- Perform data profiling to identify data quality issues and opportunities for improvement.
- Work with business stakeholders to define and implement data quality rules and standards.
- Ensure data quality rules are integrated into data processing workflows and monitored for compliance.
Issue Management:
- Implement a robust issue management process to track and remediate data quality-related issues.
- Identify data quality issues proactively by understanding different consumption patterns and implementing checks and balances.
- Monitor data quality and publish regular reports on data health.
Leadership & Collaboration:
- Lead analysts and the daily monitoring team to identify data anomalies.
- Work closely with domain leads and business data stewards in the remediation process.
- Foster a culture of data quality awareness and continuous improvement.
Reporting & Communication:
- Create and present data quality reports to senior management and stakeholders.
- Communicate data quality progress, challenges, and solutions effectively to both technical and non-technical audiences.
- Collaborate with cross-functional teams to ensure data quality standards are met.