Azure Senior Data Engineer

Key Skills: Spark, Azure Synapse, SQL, Azure ADF, Pyspark, Python.

Roles & Responsibilities:

• Design, implement, and maintain reliable and scalable data infrastructure.
• Write, deploy, and maintain software to build, integrate, manage, and quality-assure data.
• Develop and deliver large-scale data ingestion, processing, and transformation projects on the Azure cloud.
• Mentor and share knowledge with the team through design reviews, discussions, and prototypes.
• Collaborate with customers to deploy, manage, and audit standard processes for cloud products.
• Adhere to and advocate for software and data engineering standard processes, including data engineering pipelines, unit testing, monitoring, alerting, source control, code review, and documentation.
• Deploy secure and well-tested software that meets privacy and compliance requirements; develop, maintain, and improve CI/CD pipelines.
• Ensure service reliability by following site-reliability engineering standard processes, including on-call rotations for maintained services and defining and maintaining SLAs.
• Design, build, deploy, and maintain infrastructure as code and containerize server deployments.
• Work as part of a cross-disciplinary team in a Scrum/Agile setup, collaborating closely with data engineers, architects, software engineers, data scientists, data managers, and business partners.

Experience Required:

• 5 - 8 years of experience in designing and delivering end-to-end Azure data solutions involving Azure Synapse, ADF, and Spark.
• Strong background in writing complex SQL queries and optimizing performance.
• Experience with Pyspark or Python in data engineering workflows is an added advantage.
• Demonstrated capability in deploying scalable data pipelines using cloud-native tools.
• Exposure to CI/CD, infrastructure as code, and Agile delivery methodologies.
• Experience working in multi-functional teams with agile practices.

Education: B.Tech M.Tech (Dual), BCA, B.E., B.Tech, MCA