Data Engineer

Job Description

This is a remote position.

1. Python & PySpark
 

2. Azure Cloud Services such Synapse & Data Bricks & Data Factory (Data Bricks is mandatory)
 

3. OOPS Concept
 

4. Data Modelling with scalability
 

Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements
 

Solution design using Microsoft Azure services and other tools
 

The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)
 

Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications
 

Build data pipelines to collectively bring together data
 

Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse