Sr. Big Data Engineer

Job Description:

Job Overview

We are looking to hire a talented big data engineer to develop and manage our company’s Big Data solutions. In this role, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate with development teams, build cloud platforms, and maintain the production system.

To ensure success as a big data engineer, you should have in-depth knowledge of Hadoop technologies, excellent project management skills, and high-level problem-solving skills. A top-notch Big Data Engineer understands the needs of the company and institutes scalable data solutions for its current and future needs.

Big Data Engineer Responsibilities

• Meeting with managers to determine the company’s Big Data needs.
• Developing bigdata solutions on AWS, using Athena, Glue, EMR and open source Hadoop/Spark platforms
• Loading disparate data sets and conducting pre-processing services using Athena, Glue, Spark, Hive or Pig..
• Collaborating with the software research and development teams.
• Building cloud platforms for the development of company applications.
• Maintaining production systems.

Big Data Engineer Requirements:

• Bachelor’s degree in computer engineering or computer science.
• 4-6 years of previous experience as a big data engineer.
• In-depth knowledge of Hadoop, Spark, AWS data analytics services
• Must have experience on AWS data analytics services like S3, Athena, Glue, Redshift, EMR
• Must have worked with JSON, Parquet file format
• Familiarity with Data warehousing will be a plus
• Must have Knowledge of any of scripting languages like Java, Python, and R.
• Must have Knowledge of NoSQL and RDBMS databases.
• Good communication skills.
• Ability to solve complex data processing, transformation related problems