An AI feature that worked in testing produced a wrong answer for one user in production, and there was no way to reconstruct what actually happened — what prompt was sent, what came back, which model version answered.
Log these, structured, on every call
- The exact prompt sent (or a hash plus the template version, if the prompt contains sensitive data).
- Model name and version, temperature and other parameters used.
- Full raw response, latency, and token counts in and out.
- A request ID that threads through your whole system, so one user's report maps to one trace.
Trace, don't just log
A single user action often triggers several LLM calls (retrieval, generation, a safety check). Group them under one trace ID so you can see the whole chain, not disconnected log lines.
See RAG observability: what to log for the retrieval-specific version of this.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
