AWS Data engineer

Job Description/Responsibilities:

As a data engineer in the DE team, you will apply your strong technical experience building highly reliable services on managing and orchestrating multi-terabyte scale data lakes and implement a Data Mesh architecture, working closely with the Data Architecture/Modeling team. You enjoy working in an agile environment and are able to take vague requirements and transform them into solid solutions. You are motivated by solving challenging problems, where innovation, problem-solving, and creativity is as important as your ability to write code and test cases.

Minimum Qualifications and Expectations:

• At least 3 years (5 or 10 based on level) of professional experience as a software engineer or data engineer

• A BS in Computer Science or equivalent experience

• Strong programming skills (some combination of Python, Java, and Scala)

• Experience writing SQL, structuring data, and data storage practices

• Experience NoSQL databases like Mongodb and Cassandra

• Experience with data modeling

• Knowledge of data warehousing concepts

• Experienced building data pipelines and micro services

• Experience with Spark, Kafka, Flink, Hive, Airflow and other streaming and data pipeline technologies to process large volumes of streaming data

• Experience working on Amazon Web Services (in particular using EMR, Kinesis, RedShift, S3, SQS and the like)

• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Skills Required:

On prem to AWS migration, Real time data processing, Python or Java, AWS services- Lambda, Glue, Kinesis, Financial services or banking.