Azure Data Engineer

We are looking for a seasoned Azure Data Engineer with 5 years of experience to join our innovative team. The ideal candidate will have a deep understanding of Azure Data Factory and other Azure data services. In this role, you will be responsible for designing, implementing, and optimizing data pipelines, ensuring seamless data integration, transformation, and delivery to support our data analytics and business intelligence needs.

Key Responsibilities:

· Develop and Maintain Data Pipelines:Design, develop, and manage robust ETL/ELT pipelines using Azure Data Factory.Integrate data from various sources such as SQL databases, Blob Storage, Data Lake, APIs, and on-premises systems.

· Data Integration and Transformation:Implement complex data transformation logic using Azure Data Factory Data Flow and Mapping Data Flow.Ensure data is cleansed, validated, and transformed efficiently to meet business requirements.

· Optimize and Monitor Workflows:Continuously monitor data pipelines to ensure optimal performance, reliability, and cost-effectiveness.Troubleshoot and resolve issues related to data integration and workflow failures.

· Collaboration and Documentation:Work closely with data scientists, data analysts, and business stakeholders to understand and translate data requirements into scalable solutions.Document data integration processes, transformation logic, and pipeline workflows for transparency and maintenance.

· Security and Compliance:Implement and enforce data security measures in line with organizational policies and industry best practices.Ensure compliance with data governance and regulatory requirements throughout the data lifecycle.

· Continuous Improvement:Stay current with the latest features and updates in Azure Data Factory and other Azure data services.Propose and implement enhancements to existing data integration processes and workflows.

Qualifications:

Bachelor's degree in Computer Science, Information Technology, or a related field

5 years of experience in data engineering, with a strong focus on Azure Data Factory.

Proficiency in SQL and experience with various data storage solutions (e.g., Azure SQL Database, Azure Data Lake, Blob Storage).

Hands-on experience with data transformation tools and languages (e.g., T-SQL, Python, PowerShell).

Familiarity with other Azure services such as Azure Synapse Analytics, Azure Databricks, and Azure Data Lake Storage.

Experience with CI/CD practices and version control systems like Git.

Strong understanding of data warehousing concepts, data modeling, and data integration best practices.

Excellent analytical, problem-solving, and communication skills.

Ability to work independently and collaboratively in a fast-paced environment.

Preferred Skills:Experience with other ETL/ELT tools like SSIS, Informatica, or Talend.Knowledge of big data technologies and frameworks.Azure certifications (e.g., Azure Data Engineer Associate, Azure Solutions Architect).