Spark/Kafka Lead Developer

• We are seeking a Spark/Kafka Lead Developer. In this role, you will enable digital transformation for our clients in a global delivery model, research on technologies independently, recommend appropriate solutions and support technology-specific best practices and standards.
• You will be responsible for collaborating with key partners and applying your technical proficiency across different stages of the Software Development Life Cycle.
• You will be part of a learning culture, where partnership and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued.
• Candidates must be located within commuting distance of Houston, TX or must be willing to relocate to the area.
• This position may require travel to project locations

Required Qualifications:

• Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of Information Technology experience.
• Experience in Hadoop ecosystem & Apache Spark.
• Experience in Kafka streaming.
• Solid understanding and hands-on experience in SQL, Unix shell scripting

Preferred Qualifications:

• Proficiency in programming languages such as Scala, Java, or Python
• At least 4 years of experience in Apache Spark, including Spark Core, Spark SQL, Spark Streaming, and Spark MLlib
• Lead the development and optimization of batch and real-time data pipelines, ensuring scalability, reliability, and performance.
• Architect, design, and deploy data integration, streaming, and analytics solutions leveraging Spark, Kafka, and Snowflake.
• Extensive hands-on experience with Apache Kafka, including topics, partitions, producers, consumers, and stream processing
• Solid understanding of distributed computing concepts, data partitioning, fault tolerance, and parallel processing
• Experience with cluster management tools like Apache Mesos or Apache YARN is desirable
• Good understanding of data integration, data quality and data architecture
• Experience in Relational Modeling, Dimensional Modeling and Modeling of Unstructured Data
• Good understanding of Agile software development frameworks
• Strong communication and Analytical skills
• Ability to work in teams in a diverse, multi-stakeholder environment comprising of Business and Technology teams
• Experience and desire to work in a global delivery environment
• The job entails sitting as well as working at a computer for extended periods of time.
• Should be able to communicate by telephone, email or face to face.
• Travel may be required as per the job requirements