A fine-tuned adapter sitting in a folder is not a product. FineTune Studio offers three inference paths — local, a vLLM server, or a Hugging Face Space — because the right one depends on who will call the model and how often. Here is how I choose, and the checks I run before calling it deployed.
Keep the adapter or merge it?
Keeping the adapter separate lets you hot-swap behaviours on one base model and keeps files small. Merging bakes the adapter into the base weights, producing a single model that any serving stack can load without adapter support. Merge when you have one behaviour and want simplicity; keep adapters when you serve several tasks from one base.
Three serving paths
- Local inference — the training machine loads the model and answers requests. Right for demos, evaluation and internal tools; wrong for anything with real traffic.
- vLLM server — batched, fast, OpenAI-compatible API. Right for production on a GPU you control. Supports LoRA adapters directly, so merging is optional.
- Hosted Space — a public or gated endpoint for sharing a demo. Right for portfolios and hackathon judges; be mindful of cold starts and free-tier limits.
Checks before you call it shipped
- The served model reproduces your evaluation outputs. Precision differences and a missing chat template produce a subtly different model.
- The chat template used at serving time is the one used in training.
- Stop tokens are configured; otherwise the model talks until the token limit.
- Throughput and memory are measured with realistic concurrency, not one request at a time.
Operating it
Put the endpoint behind the same reliability layer as any LLM API: timeouts, retries, structured logs with the model version in every record. A fine-tuned model is a dependency you now maintain; version the adapter, the dataset and the evaluation results together so you can answer "what changed?" six months later.
Training is the exciting half. Serving is the half that makes it real, and it is where students separate themselves from tutorial followers.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
