Senior Data Engineer

Description:

We are looking for a Senior Data Engineer with 8 - 12 years of experience who will be responsible for designing, developing, and maintaining data pipelines and infrastructure, enabling the organization to collect, store, and analyse data efficiently. You will play a critical role in ensuring data quality, reliability, and accessibility, while also collaborating with stakeholder and cross-functional teams to support data-driven decision-making.

Responsibilities:

Data Pipeline Development: Design, implement, and maintain robust and scalable data pipelines that extract, transform, and load (ETL) data from various sources into data warehouses or data lakes.

Data Modelling: Develop and optimise data models and schema to support data analysis and reporting needs.

Data Integration: Collaborate with data analysts, and other stakeholders to understand data requirements and integrate data sources into analytical platforms.

SQL Expertise: Write and optimise complex SQL queries for data extraction, transformation, and reporting.

Data Architecture: Contribute to the design and maintenance of the organisation's data architecture, making strategic decisions about data storage, processing.

Security and Compliance: Ensure data security and compliance with relevant data protection regulations, industry standards, and internal data governance policies.

Documentation and Collaboration: Document data pipelines, procedures, and best practices to facilitate knowledge sharing.

Skills:

PySpark, Python, GCP, Hadoop, ETL, SQL, AWS, Google Cloud