Data Engineer with GCP

Responsibilities

• Create innovative solutions to engineer business data processes by leveraging Big Data and Cloud technology, systems development methodology, and industry standards.
• Design, implement, build, test, and deploy data pipelines to support complex business applications working with our internal Data Science and client teams.
• Present solutions to project teams, management, and client stakeholders.
• Take full ownership of features and see them through from their inception to deployment.
• To make a significant contribution to the overall delivery of advanced machine learning and of applications.
• Play an active role in client engagement, sharing progress updates and working with client teams to achieve the best possible results.

Required Skills

• Senior knowledge of GCP.
• Proficient in a Scala, Python programming for building and automating data pipelines.
• Pre and post validation of data, strong skills in data validation frameworks, reconciliation of large data set between different systems
• Strong understanding and hands-on programming/scripting experience skills – UNIX shell, Perl, and JavaScript.
• Hands-on development, with a willingness to troubleshoot and solve complex problems.
• Good to have knowledge on container technology such as Docker, and Kubernetes.
• Good to have knowledge on build management and CI/CD tools (Concourse, Jenkins), and monitoring tools (App Dynamics, etc.).
• Good knowledge on Airflow orchestration and preparing DAGs.
• Ability to work in team in diverse/ multiple stakeholder environment.
• Ability to communicate complex technology solutions to diverse teams namely, technical, business and management teams.
• Good expertise in impact analysis due to changes or issues.
• Excellent verbal and written communication skills.

Required Experience

• 4+ years of experience in software development life cycle.
• 4+ years of experience in Project life cycle activities on development and maintenance projects.
• At least 1 year of experience in Relational Modeling, Dimensional Modeling and Modeling of Unstructured Data.
• Hands on knowledge of Google Cloud Platform on GCS, Bigtable, Big Query.
• Good experience in end-to-end implementation of DW BI projects, especially in data warehouse and mart developments.
• Experience and desire to work in a Global delivery environment.
• Good understanding of Data integration, Data Quality and data architecture.
• Experience in building streaming pipelines using Kafka.
• Experience in preparing test scripts and test cases to validate data and maintaining data quality.

Education Requirements

• Bachelor’s Degree in Computer Science, Computer Engineering or a closely related field.