Every fine-tuning tutorial assumes hardware most students do not have. The choice between LoRA, QLoRA and full fine-tuning is mostly a hardware and data question, so let me lay out what each one actually does before you pick.
Full fine-tuning: every weight moves
You update all parameters of the model. It gives the most capacity to change behaviour and is what labs do for major model versions. It also needs memory for weights, gradients and optimiser states — several times the model size — which puts even a 7B model out of reach for a consumer GPU. For a student project it is almost never the right tool.
LoRA: train small adapters instead
LoRA freezes the base weights and trains low-rank matrices injected into attention (and often MLP) layers. The trainable parameters drop to a fraction of a percent, memory drops with them, and the result is a small adapter file you can swap in and out. Quality on narrow tasks is usually close to full fine-tuning.
QLoRA: LoRA on a quantised base
QLoRA loads the frozen base model in 4-bit precision and trains LoRA adapters on top. That is how FineTune Studio trains Qwen3-1.7B at around 3.2 GB of VRAM — hardware a student can borrow or rent for almost nothing. The adapters are trained in higher precision, so quality holds up well; the cost is slower steps because of dequantisation.
A decision guide
- Under 8 GB VRAM, or a free cloud GPU → QLoRA. Nothing else fits.
- 16–24 GB VRAM and a model up to 7–8B → LoRA in 16-bit for faster steps, QLoRA if you want a bigger base.
- A narrow behaviour (format, tone, classification) → adapters are enough; full fine-tuning is waste.
- Fundamentally new capabilities or a new language → full fine-tuning on serious hardware, and probably not your project this semester.
The part that matters more than the method
Whichever you choose, evaluate base against tuned on held-out examples before you believe anything — the workflow in how to evaluate a fine-tuned model honestly. Most disappointing fine-tunes were not the wrong method; they were the wrong dataset.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
