Key responsibilities include:
Designing and executing comprehensive test strategies and frameworks for LLM-powered AI agents and applications.
Conducting adversarial and edge-case testing to ensure robustness and mitigate risks such as RAG poisoning or prompt injection.
Validating the accuracy, concurrency, and effectiveness of RAG pipelines and knowledge base integrations.
Engineering and optimizing prompts for generative models using techniques like zero-shot, few-shot, and chain-of-thought prompting.
Collaborating with AI/ML and DevOps teams to resolve performance issues and contribute to continuous improvement efforts.
The Must Haves:
Bachelor’s or Master’s degree in Computer Science, Data Science, Artificial Intelligence, or a related field.
4+ years of experience in software testing, preferably focused on AI/ML or cloud-based systems.
Proficient in Python or similar programming languages.
Hands-on experience with AI/ML model testing methodologies (functional, performance, integration, security, metamorphic testing, etc.).
Working knowledge of APIs, SQL/NoSQL databases, and CI/CD pipelines.
Experience validating and troubleshooting large-scale datasets, data pipelines, and LLM applications.
Understanding of AI vulnerabilities and risk mitigation strategies in model validation.
Assets:
Experience with prompt engineering, including iterative refinement and prompt performance evaluation.
Familiarity with frameworks such as TensorFlow, PyTorch, or Google ADK.
Exposure to testing and evaluating RAG pipelines and knowledge-grounded AI systems.
Background in AI system security, including adversarial testing and prevention strategies.
Strong communication skills and the ability to document reusable test and prompt strategies effectively.
--
You must verify your mobile number to apply to this job.