Data Engineer

Responsibilities:

• Build and test Cloud-based applications for new and existing backend systems to help facilitate development teams to migrate to the cloud with an emphasis on quality, best-- practice coding standards, and cost-effectiveness.
• Build platform reusable code and components that could be used by multiple project teams.
• Provide cloud integration development support to various project teams.
• Leverage modern design patterns and architectural principles to build a platform of reusable code and components that can be used across projects and teams.
• Write both unit and integration tests, and develop automation tools for daily tasks.
• Support product owner in defining future stories and tech lead in defining technical requirements for new initiatives.
• Build rapid technical prototypes for early customer validation of new technologies.
• Collaborate effectively with Data Science to understand, translate, and integrate methodologies into engineering build pipelines.
• Collaborate with cross-functional teams and stakeholders to align development objectives with broader business goals.

Requirements:

• 4-8 years of hands-on software development with a bachelor's degree in computer science, and engineering.
• Must have strong cloud Implementation expertise in cloud architecture.
• Must have very good knowledge of storage, network, and computing services.
• Have sound knowledge of multi-zone, region-based designs.
• Must have the ability to provide solutions utilizing best practices for resilience, scalability, cloud optimization, and security.
• A quick learner, who can pick up new technologies, program languages, and frameworks in a short period.
• Experience in software development using programming languages and tools/services: Python or Scala and strong in SQL.
• Experience in big data processing tools/languages and distributed computing using Spark Scala or similar.
• Experience with orchestration tools: Apache Airflow or similar tools.
• Strong knowledge of Unix/Linux OS, commands, shell scripting, python, JSON, and YAML.
• Agile scrum experience in application development is required.
• Strong knowledge of AWS S3 PostgreSQL or MySQL.
• Deployment and automation: Terraform and Cloud Formation would be a plus.
• Strong knowledge in Compute: EC2 EMR, AWS Lambda.
• Strong knowledge of GitLab /Bitbucket.
• Mindset and attributes.
• Strong verbal/written communication and interpersonal skills.
• Must have strong analytical and technical skills in troubleshooting and problem resolution.
• Ability to coach, mentor, and provide guidance to junior colleagues