Big Data Hadoop Spark Developer

We are seeking a highly skilled Big Data Hadoop Developer with expertise in Apache Spark to join its technology team in New York. The candidate will be responsible for designing, building, and optimizing large-scale data processing systems to support critical business applications and analytics.

Responsibilities:

• Design and develop data ingestion, processing, and transformation pipelines using Hadoop ecosystem tools (HDFS, Hive, HBase, Oozie, Sqoop, Flume).

• Build and optimize distributed data processing applications using Apache Spark (Core, SQL, Streaming).

• Work with structured and unstructured data to develop scalable and high-performance solutions.

• Collaborate with data architects, business analysts, and application teams to understand requirements and deliver robust solutions.

• Implement best practices for performance tuning, data security, and data governance.

• Troubleshoot and resolve issues related to Hadoop clusters, Spark jobs, and data pipelines.

• Participate in Agile/Scrum development cycles, contributing to sprint planning, code reviews, and technical discussions.

Required Skills:

• 5–8 years of experience in Big Data / Hadoop ecosystem.

• Strong expertise in Apache Spark (Core, SQL, Streaming) for batch and real-time processing.

• Hands-on experience with HDFS, Hive, HBase, Sqoop, Oozie, Flume, and YARN.

• Solid programming skills in Java / Scala / Python