"You can't test AI" is what students say before they ship something that breaks on a Tuesday. You can test most of an LLM application exactly as you test any software, and the non-deterministic part you test statistically. FineTune Studio ships with 107 passing tests; here is how the layers divide.
Test the deterministic parts like normal software
- Chunking: given this document, these chunks with these boundaries and this heading context.
- Retrieval plumbing: the query goes to both indexes; results merge with the right fusion.
- Schema validation: malformed model output is caught and reported, not passed downstream.
- Fallback logic: a provider error triggers the next provider; a mid-stream error emits a typed event.
Mock the model here. You are testing your code, not the provider.
Golden prompts with expected properties
For the model-facing parts, keep a set of prompts with properties rather than exact strings: the answer cites at least one chunk; the answer contains the number 5; the answer refuses when the context is empty. Run them against the real model in a scheduled job, not on every commit, and alert on failures.
The evaluation set as a test
Your RAG evaluation set is a test with thresholds: recall@5 must not drop below 0.85, false-refusal rate must stay under 5%. Run it before a deploy that touches retrieval or prompts. When it fails, you have caught a regression before a user did.
In CI
- Unit and schema tests on every push — fast, mocked.
- Golden prompts and the evaluation set on a schedule and before release — slower, real model, budgeted.
- A dataset validation step for fine-tuning projects, so a bad file never reaches a GPU.
What students get wrong
Asserting exact model output, which breaks on every model update; and skipping tests for the deterministic 80% because "it's an AI project". The plumbing is where most bugs live, and it is entirely testable. I teach this as the last hour of the build workshop, because a project with tests is the one a recruiter trusts.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
