Spark Developer

Responsibilities

• Work as part of a dynamic team throughout the software development life cycle.
• Collaborate with product and engineering leadership on delivery strategy, roadmaps, and timelines.
• Implement responsive and performant interfaces using Javascript, CSS, HTML and more.
• Build scalable microservices using Python or JVM languages. to establish product requirements.
• Conduct design and code review to ensure compliance with standards and expected quality.
• Collaborate with architects, engineers, data scientists, and analysts to implement enhancements or new applications.
• Estimate level of effort to develop code to meet acceptance criteria.

Required Skills

• Strong expertise in Spark, Scala and PySpark.
• Ability to lead, guide and mentor junior developers.
• Strong SQL, Python and shell scripting.
• Working with Apache Kafka expertise is required.
• Strong expertise in Apache Spark using Scala and PySpark is mandatory.

Required Experience

• Strong working experience on Spark Core and Spark SQL.
• Solid experience of Hive, Sqoop, Hadoop and ETL processes.
• Experience in Hive, Sqoop, noSQL databases are required.
• Experience with Hadoop tools (Kafka, Hive).
• Working experience in Oracle, SQL Server and other major databases.
• Experience with Hadoop configuration and maintenance is a plus.

Education Requirements

• Bachelor’s Degree in Computer Science, Computer Engineering or a closely related field.