Description

Job Description

  • Experienced Kafka Engineer with expertise in Confluent Kafka, Java/Scala, and distributed systems. Skilled in designing scalable, fault-tolerant Kafka-based data pipelines, troubleshooting messaging issues, and optimizing performance. Strong background in cloud deployments, microservices, and Agile development with an automate-first approach.
  • Responsibilities:
  • Identify and rectify Kafka messaging issues within justified time.
  • Work with the business and IT team to understand business problems and design, implement, and deliver an appropriate solution using
  • Agile methodology across the larger program.
  • Work independently to implement solutions on multiple platforms (DEV, QA, UAT, PROD).
  • Provide technical direction, guidance, and reviews to other engineers working on the same project.
  • Administer distributed Kafka clusters in Dev, QA, UAT, and PROD environments and troubleshoot performance issues.
  • Implement and debug subsystems/microservices and components.
  • Follow an automate-first/automate-everything philosophy.
  • Hands-on in programming languages.

Key Skills & Expertise:

  • Deep understanding of Confluent Kafka: Thorough knowledge of Kafka concepts like producers, consumers, topics, partitions, brokers, and replication mechanisms.
  • Programming language proficiency: Primarily Java or Scala, with potential for Python depending on the project.
  • System design and architecture: Ability to design robust and scalable Kafka-based data pipelines, considering factors like data throughput, fault tolerance, and latency.
  • Data management skills: Understanding of data serialization formats like JSON, Avro, and Protobuf, and how to manage data schema evolution.
  • Kafka Streams API (optional): Knowledge of Kafka Streams for real-time data processing within the Kafka ecosystem.
  • Monitoring and troubleshooting: Familiarity with tools to monitor Kafka cluster health, identify performance bottlenecks, and troubleshoot issues.
  • Cloud integration: Experience deploying and managing Kafka on cloud platforms like AWS, Azure, or GCP.
  • Distributed systems concepts

Key Skills
Education

Any Graduate

Back To Jobs