Data Engineer

Job Description

• The candidate will be responsible for building out a new data pipeline to consume data from multiple sources (MSSQL, Oracle, files, APIs, etc) into cloud storage and/or Snowflake databases. They will use Streamsets to complete these activities.
• The candidate will be responsible for ensuring the data pipelines implement data privacy standards, perform well with documented data delivery SLAs, and meet all compliance standards defined by the company and/or any relevant regulatory/contractual obligations.

Roles and Responsibilities

• Create and maintain optimal data pipeline to provide operational efficiency and meet other key business performance metrics.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Define indicators of performance and quality metrics and ensure compliance with data related policies, standards, roles and responsibilities, and adoption requirements.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and regions.
• Work with data and analytics experts to strive for greater functionality in our data systems.

Qualifications

• 8+ years of experience in data engineering with at least 3 years spent building data pipelines
• Bachelor’s Degree required (Computer Science, MIS, or Information Management preferred)
• ETL experience pulling from SQL Server and Oracle with strong understanding of Change Data Capture (CDC) implementation and challenges
• Strong SQL experience
• Experience scripting with Python
• Experienced with source code control management tools such as Git or SVN
• Database configuration experience
• Cloud experience with Azure, AWS, Google Cloud, or another platform
• Big Data experience with Snowflake, Greenplum, Netezza, or Teradata

Desired Skills

• Experience with Spark and Streamsets
• Experience with Java, Scala or Groovy
• Experience with Kafka consumer
• Experience with CI/CD tools
• Experience with containerization technologies such as Docker or Kubernetes 

Leadership

• Personal and professional integrity and strong work ethics
• Ability to articulate vision of transformation efforts and a sense of mission
• Willingness to take charge and provide direction
• Results orientation, willingness to commit to a direction and drive operations to completion
• Demonstrated ability to manage adversity and challenging situations

Relationship Management

• Ability to develop cooperative and constructive working relationships
• Ability to handle complaints, settle disputes and resolve conflicts and negotiate with others
• Collaborative team player orientation towards work relationships, strong culture awareness

Project Oversight and Decisioning

• Ability to break down complex problems and projects into manageable goals
• Ability to get to the heart of the problem and make sound and timely decisions to resolve problems

People Management and Development

• Effectiveness in building trust, respect, and cooperation among teams