Key Responsibilities:
- Design, build, and optimize robust data pipelines and ETL automation using Databricks and Azure.
- Collaborate with data architects, DBAs, product owners, and stakeholders to align on project goals and solutions.
- Develop scalable data pipelines in Azure using Python and Databricks for large datasets.
- Build and optimize data lakes and backend architectures for efficient ETL processes.
- Handle healthcare-related data (claims, pharma, PHI/PII) with strict adherence to data privacy regulations.
- Implement security measures such as tokenization for sensitive data.
- Leverage Azure cloud technologies to support large-scale data processing and analytics.
- Integrate middleware solutions like MuleSoft for data processing between SAP and Databricks.
- Drive data transformation and automation initiatives within the engineering team.
Required Qualifications:
- 5+ years of experience in Databricks and ETL development.
- Strong Azure Data Engineering background.
- Proficiency in Azure Data Factory.
- Experience working with PHI/PII or HIPAA-protected data.
- Expertise in ETL automation and development.
- Strong knowledge of Python for data engineering tasks.
- Familiarity with healthcare or pharma-related data and compliance standards.
Preferred Qualifications:
- Experience integrating middleware solutions such as MuleSoft.
- Knowledge of SAP data processing workflows.
- Strong collaboration and communication skills