Data Engineer

Primary Responsibilities and Skills: (Primary Skill : Data Engineering with DPL 50% , Secondary Skill : Python-Pyspark 50%)
• Demonstrate proficiency with Python & Pyspark programming language and shell scripting languages
• Must have good hands-on expertise in Advanced Python
• Testing and debugging applications with Python test framework tools like Pytest, PyUnit, etc
• In-depth knowledge of Python frameworks and libraries, such as Django or Flask
• Strong fundamentals with experience in back-end object-oriented programming, preferably Python
• Must have experience in Flask Framework
• Familiarity with integration Restful APIs and an understanding of microservices architecture
• Solid experience with relational Database, preferably PostgreSQL
• Experience with infrastructure automation (Terraform)
• Experience with No-SQL Database, preferably MongoDB
• Strong knowledge of AWS platform including ECS, Lambda, Step Function, EC2 and experience with Boto3 SDK
• Good to have certification in AWS
Data Pipeline Management (DPL) - Involves overseeing the flow of data from various sources to destinations, ensuring it is processed, transformed, and stored efficiently. Exposure to key components such as 1. Data Ingestion , 2. Data Processing, 3. Data Orchestration, 4. Data Placement , 5. Data Quality Management, 6. Data Monitoring and Observability.
Process Skills:
Thorough understanding of the software development lifecycle
Outstanding written/verbal communication skills with experience distilling and presenting complex quantitative analysis
Interpersonal Skills:
Well-developed analytical & problem-solving skills
Strong oral and written communication skills
Excellent team player, able to work with virtual teams.
Ability to learn quickly in a dynamic start-up environment.
Must be able to work under own initiative with limited guidance from team manager due to the fast pace and high-volume of work managed by the team.
Able to talk to client directly and report to client/offshore.
Flexibility to work on different Shifts and Stretch