Senior Data Engineer - Big Data Machine Learning

Key Skills: Big Data, Spark, Scala, Flink, AWS/Azure/GCP, Data Modeling, Data Lake, NoSQL, Streaming Data, Feature Store, Machine Learning Pipelines, Performance Tuning, Distributed Systems, Cloud Deployment.

Roles & Responsibilities:

• Develop machine learning-powered data products to enhance customer experiences.
• Build, optimize, and maintain robust data architecture for data capture, storage, processing, serving, and querying.
• Develop internal platforms such as Feature Store to accelerate model training and simplify featurization.
• Design and build data products including personalization and recommendation systems, customer segmentation, insights engines, and engagement platforms.
• Work on large-scale streaming datasets to power personalized user experiences in real time.
• Collaborate with business, product, and marketing teams to support data-driven decision-making.
• Provide architectural direction and make technology choices aligned with domain-specific needs.
• Ensure platform scalability, performance tuning, and efficient processing of petabyte-scale data.

Experience Requirements:

• 8-12 years of hands-on experience with Big Data technologies including Spark and Scala.
• Strong experience in data modeling, database design, and analytical processing on large datasets.
• Proficiency in stream processing engines such as Spark Structured Streaming or Flink.
• Solid experience with distributed systems, JVM-based systems, and Spark performance tuning.
• At least 2 years of cloud deployment experience on AWS, Azure, or Google Cloud Platform.
• Proven track record of deploying products like Business Data Lakes and NoSQL databases.
• Ability to evaluate and select appropriate Big Data, NoSQL, and analytics tools and frameworks.
• Prior experience architecting domain-centric Big Data solutions.
• Strong problem-solving skills with the ability to guide and mentor team members.
• Excellent communication and collaboration skills.

Education: B.Tech