The most common message I get about FineTune Studio is some version of "I only have a 4 GB card, is this hopeless?". It is not. Fine-tuning a 1.7B model at about 3.2 GB of VRAM is exactly what the platform does. The trick is knowing where the memory goes and cutting the parts that do not affect quality.
Where the memory goes
- Base weights — a 1.7B model in 16-bit is roughly 3.4 GB; in 4-bit it is under 1.2 GB. Quantising the frozen base is the biggest single saving.
- Adapters and their optimiser state — small by design; tens of megabytes at rank 16.
- Activations — grow with sequence length and batch size. This is the part you control most directly.
- Framework overhead — CUDA context and caches, a few hundred megabytes you cannot avoid.
Settings that fit in 4–6 GB
- 4-bit base with double quantisation, adapters in bf16 or fp16.
- Per-device batch of 1 with gradient accumulation of 16.
- Gradient checkpointing on — slower steps, far smaller activations.
- Sequence length capped at the 95th percentile of your data, not the model maximum.
- A paged 8-bit optimiser so state can spill to CPU under pressure.
Free and cheap GPUs
Free notebook GPUs are enough for a first run on a small model, with two caveats: sessions end, so checkpoint to persistent storage every few hundred steps, and the disk is slow, so keep datasets small and pre-tokenised. FineTune Studio's three inference paths — local, a vLLM server, or a Hugging Face Space — exist because the machine you train on is rarely the machine you serve from.
What you give up, and what you do not
You give up speed: small batches and checkpointing make steps slower. You do not give up much quality for narrow tasks; QLoRA adapters on a quantised base track 16-bit LoRA closely. The honest way to confirm that is the base-versus-tuned comparison in how to evaluate a fine-tuned model honestly.
Hardware is a constraint, not an excuse. The students who ship fine-tuned models on borrowed GPUs learn more than the ones waiting for an A100.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
