Azure Data Engineer

As a Jr. level Azure Data Engineer, your role involves designing, implementing, monitoring, and maintaining data and analytical solutions on Microsoft Azure.

Data Pipeline Design and Implementation:

•       Create and maintain data pipelines that facilitate data movement, transformation, and integration.

•       Ensure efficient and reliable data flow from source systems to target destinations.

Data Storage Solutions:

•       Design and manage data storage solutions, including:

•       Azure Data Lake: Storing large volumes of raw data.

•       Azure SQL Database: Providing structured data storage.

•       Azure Blob Storage: Handling unstructured data.

•       Relational Databases: SingleStore, Microsoft SQL Server, Oracle

Data Processing and Transformation:

•       Use tools like Databricks and dbt to process and transform data.

•       Leverage Azure Databricks, on-premise solutions, or Data Orchestration tools for running data transformations in production

Data Integration:

•       Collaborate with data analysts and scientists to understand data requirements.

•       Design effective data workflows that enable data-driven decision-making.

Performance Optimization:

•       Optimize SQL queries for better performance.

•       Ensure consistency between development and production environments.

Security and Compliance:

•       Implement secure and compliant data processing pipelines.

•       Use Azure services and frameworks to produce cleansed and enhanced datasets for analysis

Skills:

•       Microsoft Azure, Azure Databricks, and DBT

•       Knowledge of data catalog tools (e.g., Microsoft Purview)

•       Proficiency in data processing languages, including SQL and Python

•       Familiarity with Data Orchestration Tools

•       Excellent problem-solving skills and ability to work in a collaborative environment

Skills and Experience:

Required Skills:

•       Microsoft Azure

•       Problem-solving

•       Data pipelines

•       Python

•       SQL