Two students both said 'I'm fine-tuning a model' — one meant a sentiment classifier, the other a support chatbot. Both are fine-tuning, but the data, model choice and evaluation are almost unrelated problems.
Classification
Needs a smaller model, a fixed label set, and a metric that's unambiguous: precision, recall, F1 per class. A confusion matrix tells you exactly which labels get confused with which — evaluation here is close to solved.
Open-ended generation
Has no fixed correct answer, so evaluation needs either human review, an LLM-as-judge rubric, or task-specific checks (does the reply contain a required disclaimer, is the format valid JSON). This is the harder, less settled half of the problem.
If your task is really classification, a smaller encoder-only model fine-tuned normally often beats fine-tuning a full generative LLM on cost and latency. See evaluating a fine-tuned model honestly.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
