Key takeaways
- Fine-tuning continues training a pretrained model on your examples so it reliably produces the behaviour, format or style you need.
- Use it for consistent formats, narrow tasks and smaller cheaper models; use RAG for knowledge.
- LoRA trains small adapter matrices instead of all weights, and QLoRA adds 4-bit loading so it fits on one GPU.
- The dataset decides the result: hundreds to thousands of clean, consistent examples plus a held-out test set.
- You can fine-tune some OpenAI models through its API and open-weight models such as gpt-oss on your own hardware.
Fine-tuning takes a pretrained model and continues training it on your own examples, so it learns to behave the way you need: follow a format, use a tone, classify into your categories, or handle a narrow task more cheaply. It changes the model's weights, unlike prompting or retrieval. I built FineTune Studio, which runs QLoRA fine-tuning from the browser with live loss curves and base-versus-tuned comparison, so this page answers practical questions: why fine-tune, how, LoRA, datasets, ChatGPT and GPT models, gpt-oss-20b, and the classic four-step recipe.
What does fine-tuning do?
Fine-tuning adjusts a pretrained model's weights using examples of inputs and desired outputs, so the model produces those kinds of outputs more reliably without needing long instructions or examples in every prompt.
It is best at teaching behaviour: a strict output schema, a consistent writing style, domain-specific phrasing, or a classification scheme. It is weaker at teaching facts, because models absorb patterns more readily than specific details, and fine-tuned knowledge is hard to update or cite. That is why the common recommendation is RAG for knowledge and fine-tuning for behaviour, sometimes together: a fine-tuned model that is good at using retrieved context.
Why fine-tune LLMs?
Fine-tune when prompting cannot get consistent results, when you need a smaller, faster or cheaper model to match a bigger one on a specific task, or when outputs must follow a precise format or style every time.
Also asked as: what does fine tune do · why fine tune llms · how to fine tune dataset
Good reasons:
- Consistency: the same format and tone across thousands of requests.
- Cost and latency: a small fine-tuned model replaces a large general one.
- Shorter prompts: behaviour is built in, so prompts and costs shrink.
- Domain language: jargon, abbreviations or local language mixes.
- Privacy: fine-tune an open model and run it on your own hardware.
Before fine-tuning, try better prompts and structured outputs, and measure on a test set; fine-tune only if the gap remains.
How do you fine-tune an LLM?
Collect examples of inputs and ideal outputs, split them into training and test sets, choose a base model, train with a method such as LoRA or QLoRA, then compare the tuned model with the base model on the test set before using it.
Also asked as: how do you fine tune llm
The steps in practice:
- Define the task and a success metric, such as exact-match accuracy or a rubric score.
- Build the dataset in chat format, with consistent instructions.
- Pick a base model that is already decent at the task.
- Train: start with a few epochs and a modest learning rate; watch training and validation loss.
- Evaluate on held-out examples against the base model.
- Deploy the adapter or merged model, and monitor outputs.
FineTune Studio streams the loss curve live so you can stop a run that is diverging or overfitting.
How do I fine-tune with LoRA?
LoRA freezes the original model and trains small low-rank matrices added to some layers, usually the attention projections. With Hugging Face PEFT and TRL, you load a base model, attach a LoRA configuration, and train on your dataset.
Also asked as: how to lora fine tune
from peft import LoraConfig
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset
data = load_dataset("json", data_files="train.jsonl")["train"] # {"messages": [...]} per line
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, target_modules="all-linear", task_type="CAUSAL_LM")
trainer = SFTTrainer(model="Qwen/Qwen2.5-1.5B-Instruct", train_dataset=data,
peft_config=lora, args=SFTConfig(output_dir="out", num_train_epochs=2))
trainer.train()
For larger models on one GPU, add 4-bit loading, which is QLoRA. Save the adapter, which is small, and load it on top of the base model for inference.
How do I build a dataset for fine-tuning?
Write examples that look exactly like real use: the same kind of inputs your app will receive and the ideal outputs, in a consistent format. Quality and consistency matter far more than size.
Guidelines:
- Start with a few hundred high-quality examples; scale to thousands for harder tasks.
- Cover the variety of real inputs, including edge cases and tricky ones.
- Keep the system instruction and output format identical across examples.
- Remove duplicates, contradictions and errors.
- Hold out 10 to 20 per cent as a test set never used in training.
- Include examples where the right answer is to decline or ask for clarification.
You can use a strong model to draft examples, but review them; errors in training data become errors in the model.
How do I fine-tune ChatGPT?
You cannot fine-tune the ChatGPT app itself. You can fine-tune supported OpenAI models through the OpenAI API, then use the resulting custom model in your own application. In ChatGPT, custom GPTs and Projects give customisation through instructions and files, without changing weights.
Also asked as: how to fine tune chatgpt
So the choice depends on the goal. For a personal assistant that follows your instructions and uses your documents, a custom GPT or Project is enough. For an application that needs a model to follow a format reliably at scale, fine-tune through the API with your dataset, evaluate it, and call the fine-tuned model by its ID. Consider whether a strong base model with good prompts and retrieval already meets your quality bar.
Can you fine-tune GPT models?
Yes, some. OpenAI's API supports fine-tuning for selected models, with supervised fine-tuning, preference fine-tuning and, for some reasoning models, reinforcement fine-tuning. The list of supported models changes, so check the fine-tuning guide.
Also asked as: can you fine tune gpt models
The workflow is the same for any supported model: prepare JSONL training data in chat format, upload it, create a fine-tuning job with the base model name, monitor it, and evaluate the resulting model against the base. OpenAI hosts and serves the fine-tuned model, and you do not receive its weights. If you need to own and run the weights yourself, fine-tune an open-weight model instead, such as OpenAI's open-weight releases or Llama, Qwen and Mistral models.
Can you fine-tune GPT-5 mini?
Whether a specific model such as GPT-5 mini can be fine-tuned depends on OpenAI's current list of fine-tunable models, which changes as new models launch. Check the fine-tuning guide and models page for what is supported today.
Also asked as: can you fine tune gpt 5 mini
OpenAI has often made fine-tuning available for smaller, cheaper model variants, because they benefit most and are cheapest to serve. If your target model is not fine-tunable, alternatives include fine-tuning the most similar supported model, using structured outputs and few-shot prompting with the target model, distilling from a strong model's outputs into a supported smaller one, or fine-tuning an open-weight model you host.
How do I fine-tune gpt-oss-20b?
gpt-oss-20b is one of OpenAI's open-weight models, so you can download it and fine-tune it yourself with LoRA using Hugging Face Transformers, PEFT and TRL, or optimised tools, on a GPU with enough memory.
Also asked as: how to fine tune gpt oss 20b
Practical points:
- Use the model's chat format, its harmony response format, for training examples, matching how it will be prompted.
- Train LoRA adapters rather than all weights; the model uses a mixture-of-experts design, so follow recipes published for it.
- Start with a small dataset and short run to confirm loss decreases and outputs improve.
- Evaluate against the base model on held-out examples, including reasoning-heavy cases.
- Respect the model's licence, which is permissive, and test safety behaviour after tuning.
The larger gpt-oss-120b needs data-centre GPUs for fine-tuning.
How do you fine-tune GPT-3.5 Turbo in four steps?
The classic four steps are: prepare training data in chat-format JSONL, upload the file, create a fine-tuning job, then use and evaluate the fine-tuned model. GPT-3.5 Turbo is a legacy model, but the same steps apply to current fine-tunable models.
Also asked as: how to fine-tune gpt-3.5-turbo in four steps
from openai import OpenAI
client = OpenAI()
f = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune") # steps 1-2
job = client.fine_tuning.jobs.create(training_file=f.id, model="BASE_MODEL_NAME") # step 3
# step 4: when the job succeeds, call the model with job.fine_tuned_model
Replace the base model name with a currently supported one from the documentation. Validate your JSONL before uploading, keep a test set aside, and compare the tuned model with the base on it before switching production traffic.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
