Students bring me a problem and a preferred solution — usually "fine-tune a model on our data" — before they have asked what kind of problem it is. I have built a production RAG platform and a fine-tuning platform, and they solve different problems. Here is how I decide.
Ask what the model is missing
- Knowledge it never saw — your documents, your prices, last week's policy. This is a retrieval problem. Fine-tuning does not reliably install facts, and it certainly does not update them.
- A behaviour or style — always answer in a strict JSON schema, speak like your brand, follow a domain's conventions. This is a fine-tuning problem, or a prompt problem if the behaviour is simple.
- A skill in a narrow domain — classify support tickets into your 40 categories, extract fields from your invoice layouts. Fine-tuning a small model often beats prompting a large one on cost and consistency.
Why RAG is usually first
RAG works with any model, updates the moment a document changes, and produces citations you can verify. Most business questions are knowledge questions. RAG.NextUpgrad exists because clients wanted answers about their own documents — no fine-tune would have helped them.
Why fine-tuning is sometimes unavoidable
When a prompt grows to two pages of rules and the model still slips, you are paying for those tokens on every request and still getting inconsistency. A QLoRA fine-tune on a few thousand examples bakes the behaviour in. FineTune Studio was built for exactly this: small models, cheap hardware, and an honest base-versus-tuned comparison so you know whether it worked (see how to evaluate a fine-tuned model honestly).
When you need both
A fine-tuned small model for format and tone, with retrieved context for facts. Support assistants often land here: the tune makes it sound like your company; the retrieval makes it correct about your products.
Three questions before spending a GPU hour
- Does the failure go away if I paste the right document into the prompt? If yes, it is retrieval.
- Do I have at least a thousand clean examples of the behaviour I want? If no, fine-tuning will disappoint.
- Will the underlying facts change monthly? If yes, they must live in retrieval regardless.
Pick the tool by the shape of the problem, not by which one sounds more impressive on a resume. Shipping the right one is what impresses.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
