AI Assurance Quality Engineer

About the role

Cognizant is seeking an AI Assurance Quality Engineer to lead the testing and validation of AI‑driven conversational solutions. You will work on chatbots, virtual assistants, Retrieval‑Augmented Generation (RAG) applications, and agentic AI systems, ensuring they meet high standards of reliability, performance, and regulatory compliance.

Key responsibilities

• Design and implement automated test frameworks for LLM‑based and conversational AI applications.
• Develop Python scripts to generate, execute, and report test cases across diverse AI solutions.
• Evaluate non‑deterministic AI outputs using golden datasets, rubric‑based assessments, and LLM‑as‑a‑Judge frameworks.
• Collaborate with product and domain experts to define AI failure modes, quality risks, and mitigation strategies.
• Maintain documentation of test results, statistical quality metrics, and continuous improvement recommendations.

Required profile

• Minimum 4 years of software quality engineering experience.
• Strong background in airline/travel reservation systems such as Amadeus Altéa, Sabre, or Navitaire.
• Hands‑on experience with GDS, NDC, IATA fare construction, fare rules, ancillaries, IROPS, payments and refunds.
• Proven ability to work with non‑deterministic AI models and assess their outputs.

Required skills

• Python scripting and test‑automation development.
• Testing of LLM‑based applications, chatbots, virtual assistants, RAG and agentic AI solutions.
• Prompt engineering, context management, function/tool calling, and retrieval mechanisms.
• AI failure‑mode analysis, statistical quality metrics, and human‑review workflows.
• Experience with golden datasets, rubric‑based assessments, and LLM‑as‑a‑Judge frameworks.