Description

Key Skills: Java, Big Data, Python, DevOps, Rdbms, AWS, Apache Spark, PostgreSQL

Roles and Responsibilities:

  • Design and Development:
    • Architect, design, and develop robust, scalable, and efficient data pipelines.
    • Design and manage platform solutions to support data engineering needs, ensuring seamless integration and performance.
    • Write clean, efficient, and maintainable code.
  • Leadership and Collaboration:
    • Lead and mentor a team of data engineers, providing technical guidance and fostering a collaborative environment.
    • Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
    • Drive the adoption of best practices in the software development lifecycle (SDLC), including code reviews, testing, and CI/CD.
  • Data Management and Optimization:
    • Ensure data quality, integrity, and security across all data pipelines.
    • Optimize data processing workflows for performance and cost efficiency.
    • Develop and maintain comprehensive documentation for data pipelines and related processes.
  • Innovation and Continuous Improvement:
    • Stay current with emerging technologies and industry trends in big data and cloud computing.
    • Propose and implement innovative solutions to improve data processing and analytics capabilities.
    • Continuously evaluate and improve existing data infrastructure and processes.

Skills Required:

  • 8+ years of experience in software engineering with a focus on data engineering and building data platforms.
  • Strong programming experience using Python or Java.
  • Proven experience with big data technologies such as Apache Spark, Amazon EMR, Apache Iceberg, Amazon Redshift, or similar.
  • Proven experience with RDBMS (Postgres, MySQL, etc.) and NoSQL (MongoDB, DynamoDB, etc.) databases.
  • Proficiency in AWS cloud services (e.g., Lambda, S3, Athena, Glue) or comparable cloud technologies.
  • Demonstrated leadership experience with a track record of leading and mentoring engineering teams.
  • In-depth understanding of SDLC best practices, including Agile methodologies, code reviews, and CI/CD.
  • Experience working with event-driven and serverless architecture.
  • Experience with platform solutions and containerization technologies (e.g., Docker, Kubernetes).
  • Excellent problem-solving skills and ability to work in a fast-paced, dynamic environment.
  • Strong communication skills, both written and verbal.

Education: Bachelor's or Master's degree in Computer Science, Engineering, or related field

Key Skills
Education

Any Graduate

Back To Jobs