Key takeaways
- vLLM and SGLang are open-source engines for serving language models fast on GPUs with OpenAI-compatible APIs.
- vLLM's PagedAttention and continuous batching give high throughput; SGLang adds RadixAttention for prefix reuse and structured generation.
- vLLM began at UC Berkeley's Sky Computing Lab and is Apache 2.0 licensed with a large community.
- llama.cpp targets local and CPU inference; vLLM targets many concurrent users on GPUs.
- Unsloth's core fine-tuning library is open source; open-weight models allow fine-tuning that closed models restrict.
vLLM and SGLang are open-source inference engines that serve large language models efficiently on GPUs, handling many users at once with high throughput and exposing OpenAI-compatible APIs. vLLM is known for PagedAttention memory management; SGLang for fast prefix caching and structured generation. Both suit production serving of open models. I fine-tune models with QLoRA in FineTune Studio, where serving choices follow training, so this page covers vLLM and SGLang, their origins and speed, llama.cpp, build times, Unsloth's licensing, and open versus closed models.
What are vLLM and SGLang?
vLLM is a high-throughput serving engine for LLMs that manages GPU memory efficiently and batches requests continuously, letting one GPU serve many concurrent users. SGLang is a serving framework and language for structured LLM programs, with a fast runtime that reuses shared prompt prefixes.
Also asked as: what is vllm and sglang
Both load models from Hugging Face, support quantised formats such as AWQ, GPTQ and FP8, run across multiple GPUs, and offer OpenAI-compatible servers, so existing client code works by changing the base URL. Teams use them to self-host open models for chatbots, RAG systems and agents, reducing per-request cost at scale compared with simpler serving. Both projects move quickly, adding support for new model architectures within days of release, so check their supported-models lists before choosing.
What is the difference between vLLM and SGLang?
vLLM pioneered PagedAttention and has the broadest model and hardware support with a very large community. SGLang emphasises RadixAttention, which caches and reuses shared prompt prefixes across requests, plus a frontend language for structured generation, making it very fast for agents and multi-turn workloads.
In practice, performance differences depend on model, hardware, batch size and workload: SGLang often excels when many requests share long prefixes, such as system prompts or few-shot examples; vLLM is a safe, well-supported default. Benchmark both with your model and traffic pattern, measuring throughput, latency and memory, before choosing.
Is vLLM open source?
Yes. vLLM is open source under the Apache 2.0 licence, developed on GitHub with contributions from many companies and individuals. You can use it freely, including commercially, under the licence terms.
The project is community-governed and has joined the PyTorch Foundation as a hosted project, which supports long-term neutral stewardship. Commercial companies offer managed services and support built on vLLM, but the engine itself is free. Remember that the models you serve have their own licences, which may restrict commercial use separately from vLLM's licence.
Who built vLLM?
vLLM was created by researchers at UC Berkeley's Sky Computing Lab, led by Woosuk Kwon, Zhuohan Li and colleagues, with professors including Ion Stoica. Their 2023 paper introduced PagedAttention, the memory management technique at vLLM's core.
After release in 2023, vLLM grew rapidly into one of the most widely used open-source LLM serving engines, with contributors from major technology companies, cloud providers and hardware vendors. Its development is now a large community effort, with regular releases adding new models, hardware backends and optimisations.
Why is vLLM faster?
vLLM is fast mainly because PagedAttention stores the attention key-value cache in small blocks, like virtual memory pages, eliminating wasted GPU memory, so far more requests fit in a batch. Continuous batching adds new requests as others finish, keeping the GPU busy.
Also asked as: why vllm is faster
Other optimisations include fused and optimised CUDA kernels, CUDA graphs to cut overhead, prefix caching to reuse shared prompt computations, speculative decoding, chunked prefill and efficient quantised kernels. Together these give much higher throughput than naive serving with Hugging Face Transformers, especially under many concurrent users. For a single user, the speed difference is smaller. Measure tokens per second per user and total throughput separately, since optimising one can hurt the other.
Why is vLLM faster than llama.cpp?
They target different situations. vLLM is optimised for GPU servers handling many concurrent requests, where batching and memory management multiply throughput. llama.cpp is optimised for running quantised models efficiently on CPUs, Macs and consumer GPUs, typically for one or a few users.
Also asked as: why is vllm faster than llama cpp
For a single local user, llama.cpp can be very fast and uses far less memory thanks to aggressive quantisation. For an API serving hundreds of users, vLLM's batching delivers far higher total throughput per GPU. Choose llama.cpp or Ollama for local and edge use, and vLLM or SGLang for production serving on GPUs.
How long does vLLM take to build?
Installing vLLM from prebuilt pip wheels takes a few minutes. Building from source can take much longer, from tens of minutes to over an hour, because it compiles many CUDA kernels; build time depends on CPU cores, memory and which GPU architectures you target.
Also asked as: who build vllm
To speed up source builds, limit target GPU architectures, increase parallel compile jobs if you have enough memory, and use the project's documented build caching. Most users never need to build from source: use the official wheels or Docker images matched to your CUDA version. Build from source only for development or unsupported platforms. Version mismatches between vLLM, PyTorch and CUDA cause most installation problems, so follow the compatibility table in the documentation.
Is Unsloth open source?
Yes, the core Unsloth library is open source under the Apache 2.0 licence. It speeds up fine-tuning of language models with LoRA and QLoRA through optimised kernels, reducing memory use and training time, and works with Hugging Face's TRL.
Unsloth also offers paid tiers for teams needing extra features such as multi-GPU support and enterprise help. For students and individuals, the free library with its notebooks for Google Colab and Kaggle is enough to fine-tune popular open models on a single GPU and export them to formats such as GGUF for Ollama.
Is Unsloth Studio open source?
Unsloth's core library is open source, but licensing can differ for newer tools, interfaces or enterprise products a company releases. Check the specific project's repository and licence file, or Unsloth's official documentation, before assuming any new tool is open source.
As a rule, look for a LICENSE file in the repository, read whether it is a standard open-source licence such as Apache 2.0 or MIT, and check whether some features require a paid key. For production or client work, record the licence of each tool you use. If a tool's licence is unclear, ask the maintainers or choose an alternative with clear terms.
Open-source vs closed-source LLMs for fine-tuning: which is better?
Open-weight models, such as Llama, Qwen, Gemma and Mistral releases, let you download weights and fine-tune freely with LoRA or QLoRA, run them anywhere and keep data private. Closed models offer fine-tuning only through provider APIs, with less control but often stronger base capability.
Also asked as: show hn: finetuning llms: open-source vs. close-source
Choose open models for data privacy, offline deployment, cost control at scale and full customisation, as in FineTune Studio's QLoRA workflow. Choose closed-model fine-tuning when you need the strongest base model and accept provider hosting. Either way, compare the tuned model against a well-prompted base model, since prompting or retrieval sometimes makes fine-tuning unnecessary.
Sources
- Kwon et al., Efficient Memory Management for LLM Serving with PagedAttention (2023)arxiv.org
- vLLM, repositorygithub.com
- Zheng et al., SGLang: Efficient Execution of Structured Language Model Programs (2023)arxiv.org
- SGLang, repositorygithub.com
- Unsloth, repositorygithub.com
- llama.cpp, repositorygithub.com
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
