Azure Data Engineer

• We are seeking a skilled and proactive Azure Data Engineer with expertise in Azure Data Factory, Databricks, Azure functions, python/pyspark/scala .
• KQL is an added advantage.
• The successful candidate will play a key role in managing and optimizing ETL pipelines for a large-scale, petabyte-level data lake environment.
• This role is critical for ensuring system reliability, performance, and observability across our Azure infrastructure

Essential Job Functions:

• Design and implement azure data pipelines
• Develop and optimize KQL queries for telemetry analysis, alerting, and dashboarding.
• Collaborate with DevOps, SRE, and application teams to ensure end-to-end observability and performance tracking.
• Build and maintain custom dashboards and alerts for real-time system insights.
• Ensure solutions are scalable, secure, and aligned with best practices.

Qualifications:

• 5 years of exp required

Required Skills:

• Strong hands-on experience with Aure Data factory, Databricks, Azure functions and python/pyspark/scala.
• Experience in Kusto Query Language (KQL) and Azure Data Explorer (ADX) .
• Hands on Experience with Azure services such as Event Hub, Event grid, Data Factory, and Blob Storage.
• Solid understanding of Azure cloud resources , pipelines , and data lake environments .
• Excellent troubleshooting and analytical skills.
• Strong communication and documentation abilities.
• Microsoft certifications in Azure (e.g., AZ-104, AZ-400, or DP-500).
• Experience in high-scale environments (terabytes to petabytes of data).
• Knowledge of automation tools (e.g., PowerShell, ARM templates, Terraform).
• Exposure to DevOps practices and CI/CD pipelines.

Preferred:

• Microsoft certifications in Azure (e.g., AZ-104, AZ-400, or DP-500).
• Experience in high-scale environments (terabytes to petabytes of data).
• Knowledge of automation tools (e.g., PowerShell, ARM templates, Terraform).
• Exposure to DevOps practices and CI/CD pipelines.

Mandatory Skills

• Azure Data Factory, Databricks, Azure functions, python/pyspark/scala.
• Experience in Kusto Query Language (KQL) and Azure Data Explorer (ADX)