Most student projects have 200 to 2,000 labelled examples, not the tens of thousands a paper assumes. That's enough to teach a narrow behaviour — it is not enough to teach new knowledge or broad reasoning.
What small datasets teach well
- A consistent output format (always reply in this JSON shape, always this tone).
- A narrow classification task with a handful of clear labels.
- A style or persona shift on top of capability the base model already has.
What they can't teach
New factual knowledge the base model doesn't already have, or a reasoning skill it genuinely lacks. For those, RAG or a bigger base model is the right tool, not more fine-tuning epochs on a small set.
See RAG vs fine-tuning: when to use which for that exact decision, and instruction tuning vs domain adaptation for what each small dataset is realistically shifting.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
