Senior Data Engineering Consultant

Key Skills: Data Engineer, Pyspark

Roles and Responsibilities:

• Design, develop, and maintain data pipelines using Databricks and PySpark.
• Implement and manage data solutions on Azure, AWS, or GCP.
• Optimize SQL queries; manage Azure SQL Database, Synapse Analytics, and Azure Data Factory.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Develop CI/CD pipelines using Azure DevOps to automate deployment processes.
• Ensure data integrity and accuracy through testing and validation.
• Stay current with industry trends and best practices in data engineering.

Skills Required:

Must-Have:

• Strong experience as a Data Engineer
• Expertise in building data pipelines using Databricks
• Proficiency in SQL query optimization
• Knowledge of Azure Data Services (Azure SQL Database, Synapse Analytics, Azure Data Factory)
• Experience in implementing data solutions on cloud platforms (Azure, AWS, or GCP)
• Hands-on experience with CI/CD pipelines using Azure DevOps
• Solid understanding of data validation and quality assurance techniques

Nice-to-Have:

• Experience with PySpark
• Familiarity with industry best practices in data engineering
• Ability to collaborate with cross-functional teams for data requirement gathering and delivery

Education: Any Graduation in Computer Science