The hyperparameter page in most guides lists twenty knobs and explains none. After many runs building and testing FineTune Studio on small models, here is the short list of settings that changed outcomes, with defaults that work for 1B–8B models on a single GPU.
Rank and alpha
Rank (r) sets the capacity of the adapter; alpha scales its contribution. For behaviour and format tasks, r = 8 to 16 is plenty. Raise it to 32–64 for tasks that need more change, such as a new output language or dense domain knowledge. A common, sensible pairing is alpha = 2 × r. Doubling rank doubles adapter size but rarely doubles quality.
Target modules
Attaching adapters only to attention projections is the classic setup. Including the MLP layers as well generally improves results at modest memory cost, and is my default for QLoRA on small models where memory is already handled by quantisation.
Learning rate
The single most sensitive knob. Adapters tolerate higher rates than full fine-tuning; 1e-4 to 2e-4 with a short warmup and cosine or linear decay is a reliable range. If the loss curve drops off a cliff then climbs, the rate is too high. If it barely moves over an epoch, too low.
Epochs versus dataset size
- A few hundred examples: 3–5 epochs, watch for the eval loss turning upward.
- A few thousand examples: 1–3 epochs is usually enough.
- Tens of thousands: one epoch, and consider whether you need all of it.
Batch size and gradient accumulation
On a small GPU, per-device batch of 1–4 with gradient accumulation to an effective batch of 16–32 is the normal pattern. Accumulation trades time for memory; it does not change the maths. Sequence length is the hidden memory hog: cap it at what your data needs, not at the model's maximum.
Settings people over-tune
Dropout on adapters, weight decay, scheduler exotica. They matter at the margins. Reading the loss curve — see how to read a loss curve during fine-tuning — matters far more than any of them.
Start with the defaults above, change one thing at a time, and let a held-out evaluation, not the training loss, tell you what improved.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
