ETL Developer

Job Responsibilities

• Collaborating with client to develop ETL/ELT design specifications, ensuring alignment with business requirements and technical feasibility. 
• Implementing highly scalable Business Intelligence solutions using Snowflake Data Warehouse, encompassing data modeling, data operations, Data collections and development of data pipelines. 
• Developing multiple ETL/ELT Orchestration and Transformation Pipelines/Jobs to Extract, Transform and Load the Data from multiple sources like Oracle, PostgreSQL, Netezza to target Databases like Snowflake, Salesforce. 
• Developing an ETL framework to handle multiple source files at a time using parallel processing. 
• Developing Power BI reports and Dashboards to visualize business/customer data refresh by Snowflake views. 
• Developing data models and Datamart’s to support Power BI reports and analytics using business data. 
• Implementing data quality processes to ensure data integrity and data accuracy within Power BI reports. 
• Implementing DAX expressions for complex data analysis. 
• Developing multiple python scripts using AWS Lambda to archive the processed, error files and to increase the efficiency of the framework by utilizing MATILLION for SNOWFLAKE. 
• Developing Error Handling Notification using AWS SNS Notification Service. 
• Implementing AWS Simple Queue Service (SQS) to bring the data from external stage: Cloud AWS Simple Storage Service (S3). 
• Scheduling ELT/ETL pipelines to run in the particular time intervals. 
• Validating customer data between multiple database systems like Netezza, Snowflake and DBeaver.
• Play a key role in code migration and deployment across development, QAT, and production environments, ensuring smooth release cycles. 
• Providing documentation support by creating mapping documents, project run books, and comprehensive documentation on Matillion ETL processes through confluence. 
• Participating in Data SLA discussions to align technical and business requirements. 
• Conducting technical walk-through meetings on project workflow and assisting BI developers, ETL/ELT developers in troubleshooting, fostering knowledge sharing and professional growth. 
• Actively participating in Agile methodologies, including Scrum/Agile environments, grooming sessions, business discussions, client calls and stand-up calls.