Data Engineer - Specialist

What you’ll do

Design, architect, and implement scalable and secure data pipelines on GCP, utilizing services like Dataflow, Pub/Sub, and Cloud Storage.

Develop and maintain data models, ensuring data quality, consistency, and accessibility for various internal stakeholders.

Automate data processes and workflows using scripting languages like Python, leveraging technologies like Spark and Airflow.

Monitor and troubleshoot data pipelines, identifying and resolving performance issues

proactively.

Stay up-to-date with the latest trends and advancements in GCP and related technologies, actively proposing and evaluating new solutions.

Implement data governance best practices, including data security, access control, and lineage tracking.

Lead security initiatives, design and implement security architecture.

Lead data quality initiatives, design and implement monitoring dashboards.

Mentor and guide junior data engineers, sharing knowledge and best practices to foster a high-performing team.

Role requires a solid educational foundation and the ability to develop a strategic vision and roadmap for D&A’s transition to the cloud while balancing delivery of near-term results that are aligned with execution.

What experience you need 

BS degree in a STEM major or equivalent discipline; Master’s Degree strongly preferred

8+ years of experience as a data engineer or related role, with experience demonstrating leadership capabilities 

Cloud certification strongly preferred

Expert level skills using programming languages such as Python or SQL (Big Query) and advanced level experience with scripting languages. 

Demonstrated proficiency in all Google Cloud Services

Experience building and maintaining complex data pipelines, troubleshooting complex  issues, transforming and entering data into a data pipeline in order for the content to be digested and usable for future projects; Proficiency in Airflow strongly desired

Experience designing and implementing advanced to complex data models and experience enabling advanced optimization to improve performance

Experience leading a team with Git expertise strongly preferred

Hands on Experience on Agile Methodoligies

Working Knowledge of CI/CD

What could set you apart:

Self-starter that identifies/responds to priority shifts with minimal supervision.

Strong communication and presentation skills

Strong leadership qualities

A well-balanced view of resource management, thinking creatively and effectively to deploy the team whilst building skills for the future

Skilled in internal networking, negotiating and proactively developing individuals and teams to be the best they can be

Strong communicator & presenter, bringing everyone on the journey.

Knowledge of Big Data technology and tools with the ability to share ideas among a collaborative team and drive the team based on technical expertise and learning, sharing best practices

Excellent communication skills to engage with senior management, internal customers and product management

Sound understanding of regulations and security requirements governing access to data in Big Data systems

Sound understanding of Insight delivery systems for batch and online

Should be able to run Agile Scrum-based projects

Demonstrated problem solving skills and the ability to resolve conflicts

Experience creating and maintaining product and software roadmaps

Experience overseeing yearly as well as product/project budgets

Working in a highly regulated environment