PySpark/Azure Developers

Responsibilities

• Work on a collaborative Data Analytics team.
• Responsible for modernizing a retail analytics platform sitting on a Hadoop (Spark + Python) ecosystem.
• Formulate and define system scope and objectives for assigned projects.
• Develop appropriate software solutions to requests for new services and functionality.
• Develop detailed specifications, application coding, unit and systems testing for the IT department.
• Examine, devise and modify procedures to solve moderately complex problems in regard to existing system capabilities, operating time or the form of desired results.
• Coordinate and support software production schedules and processing.
• Work with units throughout to ensure smooth delivery of existing services and program modifications.

Required Skills

• Exposure Scala, Databricks, Delta lake and notebooks development.
• Able to do low level design and build reusable components.
• Zeal to learn new tool/technologies and adoption.
• Exposure with DevOpS and CICD.
• Must be able to show/review design and code on daily basis.
• Must know test cases creation, unit testing, integration testing and QA testing.
• Should be able to make design recommendations, and build out new design patterns.

Required Experience

• Overall 4-8 years of experience with a strong background in EDW & ETL.
• 2+ Years of PySpark coding and Azure experience is must.
• Must have experience in building ETL/ELT through PySpark.
• Must have experience in designing job orchestration (ADF), sequence, metadata design, Audit trail, dynamic parameter passing and error/exception handling.

Education Requirements

• Bachelor’s Degree in Computer Science, Computer Engineering or a closely related field.