ETL Developer

Summary:

• An Azure Fabric ETL Developer is responsible for designing, developing, and maintaining ETL processes in Azure environments.
• This role involves extracting data from various sources, transforming it to meet business requirements, and loading it into data warehouses or data lakes.
• The developer collaborates with data engineers, analysts, and business stakeholders to ensure data integrity and availability for analysis.

Key Responsibilities:

• Design and implement ETL processes using Azure Data Factory and other Azure services.
• Develop data pipelines to automate data ingestion and transformation tasks.
• Optimize ETL workflows for performance and scalability.
• Collaborate with stakeholders to gather requirements and ensure data meets business needs.
• Monitor ETL jobs, troubleshoot issues, and ensure data quality.
• Maintain documentation for ETL processes, data sources, and transformations.
• Excellent problem-solving skills, communication skills and ability to work collaboratively with diverse teams.

Technical Skills:

• Proficiency in using Azure Data Factory, Azure Synapse Analytics, and Azure Blob Storage for data ingestion, transformation, and storage.
• Create and manage data movement pipelines that integrate multiple data sources and Power BI data flows, supporting data-driven decision-making.
• Experience with ETL tools and frameworks, particularly Azure Data Factory, for orchestrating data workflows.
• Strong skills in languages such as SQL for database querying, Python for custom transformations and data manipulation.
• Knowledge of data modeling concepts, including star and snowflake schemas, and experience with data warehousing solutions.
• Familiarity with Azure Databricks or similar technologies for processing large datasets.
• Skills in integrating various data sources, including relational databases, NoSQL databases, and APIs.
• Ability to optimize ETL processes for performance, including knowledge of indexing, partitioning, and query optimization.
• Experience with monitoring tools and practices for tracking ETL job performance and troubleshooting issues.
• Understanding of CI/CD pipelines and version control systems (e.g., Git) for managing ETL code and deployments.
• Knowledge of data security best practices, including data encryption and compliance with regulations such as GDPR.
• Skills in ensuring data accuracy, consistency, and reliability through data validation and cleansing techniques.
• Experience with tools like Azure DevOps for project management and collaboration within teams.