Quality Assurance Engineer

• Develop and implement comprehensive the highest standards of quality, reliability, and accuracy for applications built around Retrieval-Augmented Generation (RAG) models and related quality assurance strategies specifically tailored to AI-driven Retrieval-Augmented Generation (RAG) systems.
• Design, execute, and automate tests to validate AI model AI technologies.
• You will develop comprehensive testing strategies, automate test cases, and validate AI-driven functionality to ensure a robust and seamless integration, response quality, accuracy, scalability, and system performance.
• Perform root-cause analysis of defects, AI model inaccuracies user experience.
• Develop and implement quality assurance processes specifically designed for AI-integrated applications leveraging RAG models.
• Collaborate, or retrieval issues, and implement corrective and preventive actions (CAPA).
• Collaborate closely with engineering, product, and data science teams to understand closely with development, data science, and product teams to define clear acceptance criteria and validate AI-driven functionalities.
• Design and automate robust test scenarios including functional, integration, regression, and performance testing for AI requirements, define test cases, and ensure quality throughout the AI product lifecycle.
• Analyze and monitor AI-specific quality metrics, such-based features.
• Conduct root cause analysis for defects or issues related to AI model predictions, retrieval accuracy, and application performance, as accuracy, precision, recall, latency, and user-feedback data
• Identify and implement automation tools and frameworks for continuous testing proposing clear corrective actions.
• Track quality metrics, analyze AI model outputs and accuracy, and generate insightful reports to identify patterns and of AI-driven functionalities and interfaces.
• Support regulatory compliance and data governance requirements specific to AI and machine learning applications.
• Generate areas for continuous improvement.
• Support regulatory, compliance, and ethical standards in AI model deployment and quality assurance processes.

Qualifications:

• Bachelor’s degree in engineering, Computer Qualifications
• Bachelor’s Science, or related technical field.
• 3+ years of experience in degree in Engineering, Computer Science, or a related field.
• Minimum of 3+ years of experience in a Quality Engineering or QE role, ideally Quality Engineering with experience testing AI-driven applications preferred.
• Hands-on experience with test automation frameworks (Selenium, Cypress, JUnit, TestNG) within software or AI-focused teams.
• Hands-on experience with test automation frameworks and tools (Selenium, Cypress, JUnit, TestNG, Postman).
• Experience or and ability to automate testing of APIs and web interfaces.
• Familiarity with AI/ML systems and relevant testing methodologies, particularly familiarity with testing AI models, NLP systems, embeddings, and vector-based retrieval systems.
• Familiarity with common AI model evaluation around NLP, Retrieval-Augmented Generation (RAG), or similar model architectures.
• Experience conducting root cause analyses, CAPA metrics and methods.
• Experience with performance testing and load testing tools to validate scalability and reliability of AI services.
• Excellent analytical processes, and statistical analysis, troubleshooting, and root-cause analysis skills.
• Strong communication and collaboration
• Experience with performance and load testing tools (e.g., JMeter, Gatling) to ensure scalability and reliability of AI-driven applications.
• Knowledge of AI model evaluation metrics (precision, recall, F1-score, accuracy, BLEU, ROUGE, etc.) and techniques for validation.
• Familiarity with vector databases and embedding-based retrieval technologies (e.g., Pinecone, Weaviate, Chroma).
• Strong analytical, troubleshooting, and root-cause analysis skills with the ability to systematically address issues and propose effective corrective actions.
• Excellent communication skills and the ability to collaborate effectively across multidisciplinary teams (engineering, product management, data science).
• Experience working in agile software development environments (Scrum, Kanban)