What is a fine-tuned model? LLMs, BERT, Llama, Gemma, and loading or merging LoRA

Fine-tuned means a pretrained model has been trained further on a smaller, specific dataset so it performs better at a task, follows a style or knows a domain. Most useful AI models today are fine-tuned versions of larger base models. I built FineTune Studio, which runs QLoRA fine-tuning with Transformers, PEFT and TRL and compares base and tuned outputs, so this page covers fine-tuned models, LLMs and BERT, examples, Llama and Gemma, instruct models as autocomplete, and loading and merging LoRA adapters.

What does fine-tuned mean?

In AI, fine-tuned describes a model that started from pretrained weights and was trained further on task-specific data. In everyday English, fine-tuned means precisely adjusted for best performance, like a fine-tuned engine or argument.

Also asked as: what is fine tuned

The AI meaning follows from the everyday one: pretraining gives broad ability, and fine-tuning makes small, targeted adjustments rather than training from scratch. It is far cheaper than pretraining because the model already understands language or images. Fine-tuning can update all weights or only small added modules, as with LoRA. The term also appears in physics, where the fine-tuned universe argument discusses why physical constants seem precisely set, which is unrelated to machine learning.

What is a fine-tuned model?

A fine-tuned model is a pretrained model adapted to a specific purpose by training on a focused dataset, such as customer support conversations, medical notes or labelled product reviews. It keeps general knowledge while gaining specialised behaviour.

Also asked as: what is fine tuned model

Fine-tuned models appear everywhere: image classifiers trained on a company's product photos, sentiment classifiers for reviews, speech models adapted to accents, and chat assistants trained to follow instructions. They are typically smaller and cheaper to run than general giant models for the same task. A fine-tuned model should be evaluated against its base model on held-out examples, since fine-tuning can also introduce errors or forget skills if the data is narrow or noisy.

What is a fine-tuned LLM?

A fine-tuned LLM is a large language model trained further after pretraining. The most common types are instruction-tuned models that follow requests, and task- or domain-tuned models that specialise in areas such as code, medicine, law or a company's format.

Also asked as: what is fine tuned llm

Methods include supervised fine-tuning on prompt and response pairs, preference tuning such as RLHF or DPO, and parameter-efficient methods like LoRA and QLoRA that train small adapters instead of all weights. Typical goals are consistent output formats, a brand voice, domain terminology, tool-calling reliability or smaller, faster models that match a larger model on one task. For changing factual knowledge, retrieval is usually better than fine-tuning, so many systems combine both.

What are some examples of fine-tuned LLMs?

ChatGPT itself began as a GPT model fine-tuned with instruction data and human feedback. Other examples include Code Llama, fine-tuned from Llama 2 for programming, Google's Med-PaLM 2 for medical questions, and early open models such as Alpaca and Vicuna.

Also asked as: examples of fine tuned llms

More recent examples include the distilled DeepSeek-R1 models, which fine-tuned Qwen and Llama models on reasoning outputs, and countless community fine-tunes on Hugging Face for roleplay, coding, languages and function calling. Companies fine-tune open models for structured extraction, classification and support replies to cut costs and latency. FineTune Studio, which I built, follows the same pattern at small scale: QLoRA training with live loss charts and a side-by-side base-versus-tuned comparison.

What is a fine-tuned BERT model?

A fine-tuned BERT is Google's BERT encoder, pretrained on large text, with a small task head added and trained on labelled data for tasks such as sentiment classification, named entity recognition, question answering or semantic similarity.

Also asked as: what is fine tuned bert

Fine-tuning BERT typically takes a few epochs with a small learning rate, around 2e-5 to 5e-5, using the Hugging Face Trainer and a model class such as AutoModelForSequenceClassification. Well-known variants include DistilBERT fine-tuned for sentiment, BERT fine-tuned on SQuAD for extractive question answering, and domain models like BioBERT and FinBERT. Encoder models like BERT remain efficient choices for classification and extraction, often cheaper than prompting a large LLM.

Can Llama be fine-tuned?

Yes. Meta's Llama models are released with weights you can fine-tune under the Llama community licence, which permits derivative models subject to conditions, such as attribution and naming rules and a large-user threshold requiring a separate licence.

Also asked as: can llama be fine tuned

Practical options: QLoRA with Hugging Face TRL and PEFT on a single consumer GPU for 8-billion-parameter models, Unsloth for faster, memory-efficient training, and Axolotl or torchtune for configurable pipelines. Start from the instruct version for chat-style tasks, format data with the official chat template, and train an adapter for one to three epochs. Read the licence version that applies to your model, since terms differ between Llama releases.

Can Gemma be fine-tuned?

Yes. Google's Gemma models are open-weight and can be fine-tuned under the Gemma terms of use, which allow modification and commercial use with restrictions on prohibited uses. Sizes range from about 1 to 27 billion parameters in Gemma 3.

Also asked as: can gemma be fine tuned

You can fine-tune Gemma with Hugging Face Transformers and TRL, Keras with LoRA support, Unsloth, or Google's Vertex AI. Small sizes, such as Gemma 3 1B or 4B, fine-tune quickly on modest GPUs or free notebook tiers, making them good for learning. Use the instruction-tuned checkpoint as the starting point for chat tasks and apply its specific chat template. Evaluate against the base model, since gains on narrow tasks can come with losses elsewhere.

Can instruct fine-tuned LLMs still autocomplete text?

Yes, partly. An instruct model is still a next-token predictor, so if you send raw text without the chat template, it will continue it. But instruction tuning shifts its behaviour toward answering, so completions may be less natural than from a base model.

Also asked as: can instruct fine-tuned llms still be used to autocomplete text

Instruct models may add commentary, end early, or treat the text as a request. For pure continuation, such as autocomplete, story continuation or code completion in an editor, base models or models trained for fill-in-the-middle usually work better. With an instruct model, you can prompt explicitly, for example asking it to continue the passage exactly in the same style, or use the API's completion mode where available. Test both approaches on your own examples.

How do you load a fine-tuned PEFT or LoRA Llama model?

Load the base model with AutoModelForCausalLM.from_pretrained(base_id), then attach the adapter with PeftModel.from_pretrained(base, adapter_path). Alternatively, AutoPeftModelForCausalLM.from_pretrained(adapter_path) reads the base model name from the adapter config and loads both.

Also asked as: how to load a fine-tuned peft/lora model based on llama with huggingface transformers

Load the tokenizer from the adapter folder if you saved one, since special tokens or chat templates may have changed during training. Use the same dtype and quantization settings as training where possible, for example 4-bit loading for QLoRA adapters when memory is tight. Call model.eval() and generate with the same chat template used in training. If outputs look like the base model, check that the adapter actually loaded and targets the right modules.

How do you merge a fine-tuned adapter into the base model and push to the Hub?

Load the base model in full or half precision, attach the adapter with PeftModel, call merge_and_unload() to fold the LoRA weights into the base weights, then save with save_pretrained or upload with push_to_hub.

Also asked as: how to merge fine-tuned adapter and pretrained model in hugging face transformers and push to hub

Do not merge into a 4-bit quantized base: load the base in float16 or bfloat16 instead, merge, then quantize the merged model later if needed. Save the tokenizer alongside the model, and push both with model.push_to_hub("your-name/model-name") and tokenizer.push_to_hub(...) after logging in with a Hugging Face token. Write a model card describing the base model, data, training settings, evaluation and licence, which the base model's terms may require.

Pranjul Rathour

GenAI Engineer · India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail