Data Engineer

Essential Duties and Responsibilities

Design, develop, support and maintain data pipelines using Python and Astronomer Airflow to support data flow into customer-facing digital applications.

Collaborate with members of the delivery team, and other stakeholders to understand data requirements for digital products and implement solutions accordingly.

Build ETL processes to extract, transform, and load data from various sources into data stores that feed into digital applications.

Ensure the reliability, scalability, and performance of data pipelines, with a focus on minimizing latency and maximizing data quality.

Monitor and maintain data pipelines, proactively identifying and resolving issues to ensure consistent data delivery to applications.

Implement data validation and quality checks to ensure data accuracy and integrity within our digital platforms.

Continuously optimize and enhance data workflows and processes to support evolving product and business needs.

Document data engineering processes, including pipeline design, data flows, and operational procedures.

Stay up-to-date with the latest trends and best practices in data engineering, particularly in relation to digital applications. Performing gap analysis to identify improvement opportunities.

Engage in Agile ceremonies, primarily within the delivery team as well as the wider Genus IT Agile Release Train.

Contribute to analysis, documentation and requirements gathering within the ABS Digital team.

Participate in code reviews, providing constructive feedback to peers and ensuring high code quality.

The above list is not exhaustive, and the post holder will undertake such other duties as required.

Work within the UK time zone.

Genus Core Behaviors / Competencies

Customer impact: builds strong, profitable, sustainable customer relationships, anticipating and exceeding customer expectations to increase demand for services and products in order to build loyalty.

Managing external environment: anticipates and responds quickly to environmental changes for the benefit of the business and customers, through strong external networks and deep understanding of the markets.  

Execution orientation: Drives to set ever higher standards and achieve results through determination, resilience and commitment. Develops solutions to enhance the service offering and drive continuous improvement.

Setting Direction: develops simple, deliverable plans based on pragmatic new thinking, ideas or concepts. Assesses accurately commercial risk and return.

Change management: Champions, leads, support or embeds change to improve things. Communicates well and helps others by overcoming barriers.

Analysis and Decision Making: Analyses opportunities and problems thoughtfully and thoroughly to make good and timely decisions.  

Team Mobilization: contributes towards the success to be a part of a high performing diverse team.

Collaboration: ‘One team’ approach: gains commitment to strategic vision and goals. Builds and maintains networks and relationships, sharing knowledge and experience, delivering on commitments.

The following are qualities that are the foundations on which Genus team members work:

• Integrity

• Honesty

• A desire to work to make a difference in the communities & countries that we work in

• Delivery on commitments – do what you say you are going to do

• Alignment with the business goals and values

Qualifications and Experience

Bachelor’s degree in Computer Science, Data Engineering, or a related field, or equivalent practical experience.

3+ years of experience as a Data Engineer, with a focus on supporting data pipelines for digital applications.

Proficiency in Python programming and experience with Astronomer Airflow or Apache Airflow, including DAG creation, workflow management, and scheduling.

Solid understanding of ETL processes and data integration techniques.

Experience working with SQL and relational databases and data warehousing solutions.

Familiarity with cloud platforms, specifically Microsoft Azure and data-related services.

Understanding of data architecture principles and best practices for data management in customer-facing applications.

Strong problem-solving skills, with the ability to troubleshoot and resolve data pipeline issues quickly.

Experience working with data pipelines in a customer-facing digital environment, such as web or mobile applications.

Knowledge of data governance, privacy, and security best practices.

Experience with containerization and orchestration tools like Docker.

Understanding of data modelling and API integration for digital applications.

Strong problem-solving skills, with attention to detail and a commitment to delivering high-quality work.

Excellent communication skills, with the ability to work effectively in a collaborative, cross-functional, team-oriented environment.

Experience in working in a fast paced and cross functional environment, utilizing strong organizational skills with the ability to handle multiple priorities and deliver to deadlines.

Broad experience across a number of IT disciplines.

Be flexible with respect to job responsibilities and consistently strive to be an effective team member.

The above position description is intended to describe the general content, identify the essential functions of, and requirements for the performance of this position. It is not to be construed as an exhaustive statement of duties, responsibilities or requirements.