Lead Data Engineer Azure

Job description:

We are looking for a highly experienced Senior Data Engineer with a strong focus on Azure Data Services and DBT, complemented by expertise in Snowflake and Snowflake Cortex. The ideal candidate will lead the design and development of scalable data pipelines, ensure data quality and governance, and support AI-driven analytics using Snowflake Cortex and GenAI capabilities. This role is ideal for someone who thrives in a cloud-native, data-first environment and enjoys working across engineering and analytics teams. 

Key Responsibilities 

Azure & DBT-Focused Data Engineering 

• Design, build, and optimize scalable data pipelines using Azure Data Factory, DBT, and Databricks. 
• Develop modular, testable DBT models for data transformation and analytics. 
• Implement data orchestration and workflow automation in Azure environments. 
• Ensure high performance, reliability, and scalability of data workflows. 

Snowflake & Cortex Integration 

• Build and maintain Snowflake-based data architectures and pipelines. 
• Leverage Snowflake Cortex functions for: 
• Anomaly Detection, Time-Series Forecasting, Classification 
• Text Completion, Embedding, Sentiment Analysis, Summarization 
• Integrate Snowflake Cortex with UI tools like Copilot, Universal Search, and Document AI. 

Data Quality & Governance 

• Implement robust data validation frameworks using: 
• Snowflake Cortex anomaly detection 
• Custom logic via Snowpark UDFs 
• Collaborate with analytics teams to deliver clean, trusted datasets. 
• Ensure compliance with data governance, privacy, and security standards. 

 Required Qualifications 

• 7+ years of experience in data engineering, with a strong focus on Azure Data Factory, DBT, and Databricks. 
• Deep expertise in Snowflake and Snowpark APIs. 
• Experience with Snowflake Cortex ML and LLM functions. 
• Strong proficiency in Python and SQL. 
• Familiarity with open-source LLMs and GenAI integration within Snowflake. 
• Excellent understanding of data modeling, transformation, and pipeline optimization. 
• Strong collaboration skills across engineering, analytics, and business teams.