QA Engineer

We are looking for a QA Engineer with a passion for ensuring the quality and reliability of AI-powered applications, especially those built using Generative AI technologies. You will play a critical role in validating AI systems, including LLMs, RAG pipelines, and backend services. Your work will help shape the safety, accuracy, and robustness of AI-driven features used by thousands (or millions) of users.

Key Responsibilities

Functional Testing

· Design, execute, and maintain test cases for APIs, backend services, and UI interfaces.

· Conduct regression, integration, and exploratory testing of AI-powered features.

· Validate RESTful APIs and end-to-end workflows in web applications.

AI/LLM-Specific Testing

· Test outputs from LLMs and GenAI pipelines for factual accuracy, coherence, tone, and safety.

· Evaluate outputs for hallucination, repetition, bias, toxicity, and other model behavior anomalies.

· Create structured test prompts and scenarios to assess the consistency and reliability of AI responses.

· Perform adversarial and edge case testing, including prompt injection and jailbreak scenarios.

Automation Testing

· Develop and maintain automation test suites using tools like Selenium, Playwright, Cypress, or PyTest.

· Automate API testing workflows using Postman, REST-assured, or similar tools.

· Integrate test automation into CI/CD pipelines for continuous validation and fast feedback loops.

Non-Functional Testing

· Conduct performance, load, and stress testing on model inference APIs and backend services.

· Measure and validate latency, throughput, and resource utilization for GenAI features.

· Test system behavior under scale and ensure failover and resilience capabilities.

Data and Model Validation

· Collaborate with data scientists to verify training data integrity and model evaluation results.

· Validate semantic search results in vector databases used in RAG (Retrieval-Augmented Generation) pipelines.

· Test chaining logic and prompt orchestration using frameworks like LangChain.

Security, Compliance, and Governance

· Perform tests to identify data privacy risks, prompt injections, and leakage of sensitive content.

· Support red-teaming efforts to proactively identify vulnerabilities in GenAI systems.

· Contribute to documentation of risk mitigation plans and compliance readiness for audits.

Required Skills & Qualifications

· Bachelor’s degree in Computer Science, Engineering, or related field.

· 3+ years of professional experience in software QA; exposure to AI/ML projects is a plus.

· Strong understanding of QA methodologies, test planning, and execution.

· Hands-on experience testing APIs, backend systems, and web applications.

· Proficiency with test automation frameworks like Selenium, Playwright, Cypress, or PyTest.

· Experience with API testing tools like Postman, REST-assured, or similar.

· Familiarity with LLMs, NLP concepts, and GenAI tools (e.g., OpenAI, Hugging Face, LangChain).

· Working knowledge of Git, CI/CD tools, and test management platforms