- Design and implement scalable data processing solutions using Azure Databricks.
- Collaborate with data Scientist and analysts to understand data needs.
- Optimize data pipelines and workflows for performance and scalability.
- Ensure data quality and integrity throughout Data Transformation and Load processes.
- Develop and maintain data architecture and best practices.
- Troubleshoot and resolve data related issues in a timely manner.
- Setup Performance Monitoring and Alerting for pipeline and data integrity
- Proficient in ETL between different RDBMS like Oracle to Data Bricks and NoSQL Databases.
- Working knowledge in Python/Pyspark/ Scala.
- exp in mongo db is must to have
Experience in MongoDB will be added advantage