Data Engineer

what we are looking

·        Work with business stakeholders, Business Systems Analysts and Developers to ensure quality delivery of software.
·        Interact with key business functions to confirm data quality policies and governed attributes.
·        Follow quality management best practices and processes to bring consistency and completeness to integration service testing
·        Designing and managing the testing AWS environments of data workflows during development and deployment of data products
·        Provide assistance to the team in Test Estimation & Test Planning
·        Design, development of Reports and dashboards.
·        Analyzing and evaluating data sources, data volume, and business rules.
·        Proficiency with SQL, familiarity with Python, Scala, Athena, EMR, Redshift and AWS.
·        No SQL data and unstructured data experience.
·        Extensive experience in programming tools like Map Reduce to HIVEQL
·        Experience in data science platforms like SageMaker/Machine Learning Studio/ H2O.
·        Should be well versed with the Data flow and Test Strategy for Cloud/ On Prem ETL Testing.
·         Interpret and analyses data from various source systems to support data integration and data reporting needs.
·        Experience in testing Database Application to validate source to destination data movement and transformation. 
·        Work with team leads to prioritize business and information needs.
·        Develop complex SQL scripts (Primarily Advanced SQL) for Cloud ETL and On prem.
·        Develop and summarize Data Quality analysis and dashboards.
·        Knowledge of Data modeling and Data warehousing concepts with emphasis on Cloud/ On Prem ETL.
·        Execute testing of data analytic and data integration on time and within budget.
·        Work with team leads to prioritize business and information needs
·        Troubleshoot & determine best resolution for data issues and anomalies
·        Experience in Functional Testing, Regression Testing, System Testing, Integration Testing & End to End testing.
·        Has deep understanding of data architecture & data modeling best practices and guidelines for different data and analytic platforms
Requirements:
·        Extensive Experience in Data migration is a must ( Teradata to Redshift preferred)
·        Extensive testing Experience with SQL/Unix/Linux scripting is a must
·        Extensive experience testing Cloud/On Prem ETL (e.g. Abinitio, Informatica, SSIS, Datastage, Alteryx, Glu)
·        Extensive experience DBMS like Oracle, Teradata, SQL Server, DB2, Redshift, Postgres and Sybase.
·        Extensive experience using Python scripting and AWS and Cloud Technologies.
·        Extensive experience using Athena, EMR, Redshift, AWS, and Cloud Technologies