Azure Data Engineer

Key Skills: Azure Synapse, Azure Databricks, Azure, Azure Devops, Azure AI, Azure Api, Azure AD, PLSQL

Roles and Responsibilities:

• Design and Develop Data Pipelines: Build and maintain scalable data pipelines using Azure Data Factory, ensuring efficient and reliable data movement and transformation.
• File-Based Data Management: Handle data ingestion and management from various file sources, including CSV, JSON, and Parquet formats, ensuring data accuracy and consistency.
• ETL Implementation: Implement and optimize ETL (Extract, Transform, Load) processes using tools such as Azure Data Factory, Azure SQL Database, and Azure Synapse Analytics.
• Cloud Storage Management: Work with Azure Data Lake Storage to manage and utilize cloud storage solutions, ensuring data is securely stored and easily accessible.
• Automation with Data Factory: Leverage Azure Data Factory's automation capabilities to schedule and monitor data workflows, ensuring timely execution and error-free operations.
• Performance Monitoring: Continuously monitor and optimize data pipeline performance, troubleshoot issues, and implement best practices to enhance efficiency.
• Team Collaboration: Collaborate with Technical Architects, Business Analysts, and other engineers to build scalable and reliable end-to-end data solutions for reporting and analytics.
• DevOps Framework: Defining and implementing DevOps framework using CI/CD pipelines.
• SQL Development: Write efficient, clean, and well-documented SQL queries for data extraction, manipulation, and analysis.
• SQL Performance Optimization: Optimize performance of SQL-based queries, stored procedures, and jobs in Azure environments.
• Data Security & Compliance: Implement data security best practices and ensure compliance with data privacy regulations (HIPAA, etc.).
• Technical Leadership: Provide technical leadership and mentoring to junior engineers and team members.
• Technology Adoption: Stay current with emerging Azure technologies and trends, recommending improvements to existing systems and solutions.

Skills Required:

• Strong expertise in Data Analytics for analyzing and interpreting large datasets.
• Proficiency in Azure Boards-GitHub for managing project tasks and source code version control.
• Extensive experience with Azure Data Factory for building and managing scalable data pipelines.
• In-depth knowledge of Azure Data Lake for managing cloud storage solutions and data access.
• Hands-on experience with Azure Synapse for data integration and analytics solutions.
• Proficient in Azure DevOps for implementing CI/CD pipelines and automating deployments.

Education: Bachelor's or Master's degree in Computer Science, Information Systems, Data Engineering, or a related technical field