Azure Data Engineer

Responsibilities:

Azure Data Platform Development: Design, develop, and implement end-to-end data solutions on Azure, encompassing data lakes, data warehouses, and data pipelines.

Data Pipeline Engineering: Build and optimize high-performance data pipelines using Azure Databricks, PySpark, Spark, and Python to ingest, process, and transform large datasets from diverse sources.

Data Modeling and Warehousing: Implement data modeling techniques and design efficient data warehousing solutions to support business intelligence and analytics initiatives.

Data Integration with Azure Fabric: Utilize Azure Fabric to seamlessly integrate data from various systems, ensuring data consistency, reliability, and accessibility across the organization.

Semantic Modeling and Optimization: Develop and implement Azure Semantic Models to empower users with self-service data exploration and analysis.

Data Analysis with SQL Cubes: Design and implement SQL Cubes for multidimensional data analysis, enabling efficient querying and reporting.

Data Quality and Governance: Ensure data quality, accuracy, and integrity by implementing data validation rules, data cleansing processes, and data governance best practices.

Collaboration and Communication: Collaborate effectively with data scientists, data analysts, and other stakeholders to understand data requirements, provide technical guidance, and deliver high-quality data solutions.

Performance Optimization: Optimize data pipelines and queries for performance, scalability, and cost-effectiveness.

Documentation: Create and maintain comprehensive technical documentation, including data flow diagrams, data models, and ETL specifications.

Qualifications:

Must-Have Skills:

Extensive hands-on experience with Azure Databricks for large-scale data processing.

Strong proficiency in PySpark, Spark, and Python for data manipulation, transformation, and analysis.

Experience with Data Lake implementation on Azure.

Experience with Azure Fabric for data integration and management.

Experience with SQL Cube.

Strong SQL skills, including the ability to write complex queries and stored procedures.

Solid understanding of data warehousing concepts, data modeling techniques, and ETL processes.

Experience with data quality and data governance principles.

Excellent analytical, problem-solving, and troubleshooting skills.

Strong communication, collaboration, and interpersonal skills.

Ability to work independently and effectively in a remote environment, while adhering to PST working hours.