Most RAG logging setups capture the full prompt and full response and nothing else, which makes debugging a bad answer nearly impossible six weeks later.
Log this
- The retrieved chunk IDs and their similarity scores, not just the final context text.
- Whether a confidence gate or relevance check fired, and its verdict.
- Latency per stage (embed, search, rerank, generate), not just total time.
- User feedback (thumbs up/down) linked to the exact chunk IDs used.
- Query rewrites, if you decompose or rewrite the original question.
Skip logging every raw embedding vector
Vectors are expensive to store and rarely useful after the fact — you can always re-embed the original chunk text if you need to inspect it. Log the text and the score, not the numbers.
See testing LLM applications, a practical guide for how this logging feeds an evaluation loop.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
