Design, build and maintain data processing pipelines.
Design and develop using various data languages like Python, R, Scala etc.
Define API’s and micro services consumed by other applications/teams.
Evaluate and/or implement changes in production transactional system to support ETL data pipelines as needed.
Build, test and maintain tools, infrastructure to support Data science initiatives.
Monitor and maintain live data ingestion; data pipelines and ETL.
Understand best practices for data separation; security and disaster recovery through multiple data centers and Cloud regions.
Provide insight into system data quality; recommend; and implement solutions to resolve data quality issues.
Perform basic analysis to transform data into actionable information for use by the stakeholders.
Required Skills
Strong DevOps, Data Engineering skill and ML background.
Support and work with a diverse cross-functional remote teams in a dynamic environment.
Understanding of Agile methodology.
Knowledge of containerization and orchestration (such as Docker, Kubernetes).
Ability to handle multiple tasks and priorities.
Ability to collaborate with team members to meet project deadlines and milestones.
Required Experience
5+ years of software and data engineering experience with focus on designing and developing with Customer Data technologies and platforms, Databases, and large-scale data processing systems.
3-5 years of working with customer data flows and mappings across Customer Data Platform (CDP), Salesforce Marketing Cloud (SFMC), Salesforce Service Cloud (SFCC), Mulesoft APIs, ETL tools etc.
3-5 years of technical design and development experience with data development languages like SQL, Python, R, SQL etc.
Experience with end-to-end data pipeline automation process, including data input, data quality validation, data extraction and data reporting.
Experience with big data technologies: Spark, Distributed File Systems, Hive, Parquet and column stores in general.
Education Requirements
Bachelor’s Degree in Computer Science, Information Technology or a closely related field.