Hadoop Administrator

Job Description/Responsibilities

As a Hadoop Administrator, you will be responsible for the design, implementation, maintenance, and support of our Hadoop infrastructure, with a focus on AWS (Amazon Web Services) and EMR (Elastic MapReduce). You will collaborate closely with cross-functional teams to ensure the reliability, scalability, and performance of our big data ecosystem.

Key Responsibilities

Design, deploy, and maintain Hadoop clusters on AWS using EMR.
Configure and tune Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, and HBase for optimal performance.
Monitor cluster health and performance, troubleshoot issues, and implement solutions to ensure high availability and reliability.
Implement security best practices to protect data and infrastructure.

Automate routine tasks and processes to streamline operations.

Collaborate with data engineers, developers, and other stakeholders to optimize data pipelines and workflows.
Provide technical guidance and support to internal teams on Hadoop-related technologies.
Stay abreast of emerging technologies and best practices in big data and cloud computing.

Qualifications

Bachelor's degree in Computer Science, Engineering, or a related field.

years of experience as a Hadoop Administrator or similar role.

Proficiency in administering Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, and HBase.
Hands-on experience with AWS services, particularly EMR, EC2, S3, IAM, and CloudWatch.
Strong Linux system administration skills.
Experience with configuration management tools such as Ansible, Puppet, or Chef.
Knowledge of security concepts and best practices in a cloud environment.
Excellent troubleshooting and problem-solving skills.
Strong communication and collaboration skills