Data Engineer

• Develop and maintain backend systems using Python, PySpark ensuring high performance, scalability, and reliability
• Participate in the design and implementation of data engineering and ETL pipelines using PySpark
• Collaborate with cross-functional teams to identify and prioritize project requirements
• Mentor and guide engineers, providing technical guidance and code reviews
• Stay up-to-date with the latest technologies and frameworks, and apply this knowledge to improve existing systems and processes
• Lead the technical direction of the team, including architecture, design, and implementation of software systems
• Collaborate with product owners to define and prioritize requirements
• Develop and maintain technical documentation, including architecture diagrams, and technical specifications
• Participate in code reviews, ensuring high-quality code and adherence to coding standards
• Collaborate with DevOps and Operations teams to ensure smooth deployment and operation of applications

• Expert in DevOps practices and tools for CI/CD pipelines.
• Willingness to learn new technologies and adapt to new challenges
• Proven experience in leading technical teams, including technical guidance, mentoring, and coaching
• Strong technical leadership skills, including the ability to make technical decisions, prioritize tasks, and manage technical resources
• Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams
• Ability to drive technical innovation, including researching new technologies, evaluating technical options, and recommending technical solutions
• Strong problem-solving skills, with the ability to debug complex issues, optimize system performance, and ensure high-quality software delivery
• Experience with Agile development methodologies, including Scrum or Kanban, and the ability to apply these principles to lead the team

• Experience with cloud platforms, including Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP)
• Experience with PySpark for Data Engineering/ETL pipelines
• Familiarity with containerization using Docker and orchestration using Kubernetes