The Data Engineer is responsible for supporting the design and development of ETL/ELT code that extracts and manipulates data from a variety of complex sources and populates the Enterprise Data Warehouse including new and legacy sources transitioning from legacy reporting environments
- Interface with BSAs, solution designers, data architects, and other business stakeholders to understand the business requirements
- Assist with analysis and profiling of various source data and understand high level relationships between data sources
- Parse unstructured and semi-structured data such as JSON, XML…etc.,
- Design & develop efficient mapping and workflows to load data onto Data Marts
- Design and develop ETL/ELT pipeline for CI/CD
- Develop, test and document ETL/ELT solutions in a complex data warehouse environment with numerous desperate source systems
- Use MS SQL Server 2012, Snowflake and SSIS/ADF/DBT for data integration, Control-M for package scheduling and TFS or Azure GIT repository for version control and task management
- Perform DB modifications in accordance with team processes and standards