Big Data Engineer

We are seeking a talented Big Data Engineer with expertise in Scala, Spark, and the Hadoop Ecosystem to design, develop, and implement robust big data solutions. The ideal candidate will possess a strong understanding of Hive & SQL, along with excellent communication skills, and experience in the Banking/Capital Markets domain. The role involves working with real-time data processing platforms and delivering efficient data solutions while thriving in a fast-paced environment.

Key Responsibilities

Big Data Development:

• Design and implement scalable solutions using Spark, Scala, and Hadoop-based tools such as HDFS, Hive, and HBase.
• Develop real-time data processing pipelines leveraging Spark Streaming for advanced analytics.

Data Management:

• Utilize Hive & SQL for data storage, querying, and transformation in a distributed environment.
• Ensure accurate data integration and optimize processing workflows.

Linux-Based Ecosystem:

• Work within a Linux-based Hadoop environment, leveraging tools like Impala, Hive, and HBase.

Problem-Solving and Communication:

• Exhibit excellent problem-solving and decision-making skills to tackle technical challenges effectively.
• Demonstrate strong communication abilities, ensuring clarity in collaboration with teams and stakeholders.

Banking/Capital Markets Exposure:

• Apply knowledge of Banking/Capital Markets domain to enhance data solutions tailored to industry needs.

Documentation and Process Analysis:

• Create detailed documentation, including process flow diagrams, business models, and functional designs.

Multi-Tasking and Delivering Results:

• Manage multiple priorities effectively and deliver against agreed objectives in high-pressure environments.

Qualifications & Skills

Minimum Qualifications:

• Spark (Must Have)
• Scala (Must Have)
• Hive & SQL (Must Have)
• Hadoop (Must Have)
• Strong proficiency in Linux-Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase).
• Excellent verbal and written communication skills (Must Have).

Preferred Qualifications:

• Familiarity with Banking/Capital Markets domain (Good to Have).
• Hands-on experience with real-time data processing platforms such as Spark Streaming.
• Advanced analytical skills with expertise in process flow diagrams, business modeling, and functional design