Key takeaways
- Residual vector quantization encodes a vector in stages, each codebook quantising the error left by the previous one.
- Model quantization stores weights in fewer bits, cutting memory and often speeding up inference.
- Non-uniform levels, as in NF4 or mu-law, fit real data better than evenly spaced ones.
- Quantization usually costs a little accuracy, more for small models, hard tasks and non-English languages.
- AWQ protects the most important weights by scaling them before 4-bit quantization.
Residual vector quantization (RVQ) compresses a vector in stages: a first codebook approximates it, a second codebook encodes what the first missed, and so on, so a few small codebooks represent it precisely. It powers neural audio codecs and compressed vector search. Model quantization applies a related idea to weights. I use 4-bit quantization in FineTune Studio for QLoRA training, so this page covers RVQ and vector quantization, why quantization is needed, sampling and non-uniform levels, its effect on accuracy and multilingual models, and AWQ.
What is residual vector quantization?
Residual vector quantization encodes a vector with a sequence of codebooks: the first picks the nearest codeword, the next encodes the residual error left over, and each later stage refines what remains. The final vector is the sum of the chosen codewords.
With eight stages of 1,024 codewords each, every vector is stored as eight small integers, yet the combinations can represent an enormous number of distinct vectors. RVQ is used in neural audio codecs such as SoundStream and EnCodec, where audio frames become tokens that language models can generate, and in vector search to compress embeddings. Its coarse-to-fine structure is useful: dropping later stages gives a lower bitrate at lower quality, which lets one model serve several bandwidths.
What is vector quantization?
Vector quantization maps each vector to the nearest entry in a fixed codebook of representative vectors, so you store a short index instead of the full vector. The codebook is usually learned with k-means clustering on sample data.
It is lossy compression: the error is the distance between a vector and its codeword. Classic uses include image and speech compression. In vector search, product quantization splits each embedding into sub-vectors and quantises each separately, shrinking memory many times over so very large collections fit in RAM; FAISS implements this. In deep learning, VQ-VAE models learn discrete codes for images and audio, which made token-based generation of non-text media practical.
What does quantization do to a model?
Quantization stores a model's weights, and sometimes activations, in fewer bits, for example 8-bit or 4-bit integers instead of 16-bit floats. The model becomes smaller, needs less memory and often runs faster, at the cost of a small loss in precision.
A 7-billion-parameter model needs about 14 GB in 16-bit precision and roughly 4 GB at 4 bits, the difference between needing a data-centre GPU and running on a laptop. Each weight is rounded to one of a few levels using a scale factor per group of weights, and good methods keep quality close to the original. In FineTune Studio, QLoRA loads the base model in 4-bit NF4 format and trains small LoRA adapters on top, which makes fine-tuning possible on a single modest GPU.
Why is quantization required?
Quantization is required because large models are too big and slow for most hardware at full precision. Lower precision cuts memory, bandwidth and energy use, so models fit on cheaper GPUs, phones and browsers, and serve more users per machine.
Also asked as: why quantization is required
Language model inference is usually limited by memory bandwidth: every generated token reads all the weights. Halving the bytes per weight roughly halves that traffic, so quantised models often generate tokens faster. Quantization also enables on-device AI where data must stay local, and lowers serving costs at scale. It is not free, so measure quality on your own tasks: summarisation might be unaffected while maths or code may degrade more.
Why is non-uniform quantization required?
Non-uniform quantization spaces its levels unevenly, placing more levels where values are common and fewer where they are rare. It is needed because real signals and neural network weights are not spread evenly; most values cluster near zero.
Also asked as: why non uniform quantization is required
In telephone speech, the mu-law and A-law companding schemes give quiet sounds finer steps than loud ones, matching how speech amplitudes are distributed and how we hear. In neural networks, weights roughly follow a bell-shaped distribution, so the NF4 format used by QLoRA places its 16 levels according to a normal distribution, giving lower error than evenly spaced 4-bit levels for the same number of bits. Uniform quantization is simpler for hardware, which is why both remain in use.
Why are sampling and quantization required?
Sampling and quantization turn an analogue signal into digital data. Sampling measures the signal at regular time or space intervals; quantization rounds each measurement to one of a fixed set of levels so it can be stored as a number.
Also asked as: why is sampling and quantization required
Both are required because computers store finite numbers. In images, sampling sets the resolution, the pixel grid, and quantization sets the bit depth, such as 256 levels per colour channel. In audio, CD quality samples 44,100 times a second at 16 bits. Too few samples cause aliasing; too few levels cause banding or noise. Every vision model, including the face pipeline in FaceVision and the document models behind DocuLens AI, starts from images that were sampled and quantised this way.
Does quantization affect accuracy?
Yes, usually a little. 8-bit quantization typically keeps accuracy very close to the original model, while 4-bit methods such as AWQ and GPTQ lose a small amount on most benchmarks. Below 4 bits, losses grow quickly.
The impact depends on the model, the method and the task. Larger models tolerate quantization better than small ones. Reasoning, maths, code and long-context tasks tend to degrade before simple chat. Calibration-based methods that protect important weights outperform naive rounding. The practical rule: evaluate the quantised model on your own test set, compare it with the full-precision version, and choose the lowest precision that meets your quality bar. Perplexity alone can hide task-specific drops.
How does quantization affect multilingual LLMs?
Research suggests quantization hurts non-English languages, especially those in non-Latin scripts, more than English, and that automatic metrics can understate the damage. A 2024 study found human evaluators noticed larger drops than benchmarks showed.
For products serving Indian users in Hindi, Tamil or other languages, this matters: a 4-bit model that scores well in English may give weaker answers in Hindi. Test every language you support separately, include native speakers in evaluation, and consider higher precision or a larger model when multilingual quality is critical. The same study found difficult tasks such as mathematical reasoning degrade fastest, so test those in each language too.
What is AWQ quantization?
AWQ, activation-aware weight quantization, is a 4-bit weight quantization method published by MIT researchers in 2023. It finds the small fraction of weights that matter most, judged by activation sizes, and protects them by scaling before quantising.
Also asked as: awq quantization tutorial
The key observation is that around 1 percent of weights or fewer, those connected to large activations, carry disproportionate importance. Instead of keeping them in higher precision, which complicates hardware, AWQ scales those channels up before quantization and compensates in the activations, reducing their rounding error. It needs only a small calibration set and no backpropagation, so it generalises well across domains. AWQ models are widely shared on Hugging Face and supported by inference engines such as vLLM.
How do I quantize a model with AWQ?
The quickest path is to download an existing AWQ model from Hugging Face. To quantise your own, use a library such as vLLM's LLM Compressor, which took over from the archived AutoAWQ project, with a calibration dataset, then load the result in vLLM or Transformers.
A short tutorial outline:
- Pick a base model and a GPU with enough memory to load it in 16-bit.
- Prepare a few hundred calibration samples similar to your real inputs.
- Run the AWQ recipe with 4-bit weights and a group size of 128.
- Save the quantised model with its configuration.
- Load it and compare outputs with the original on your own test set.
Check the library's current documentation, because quantization tooling changes quickly.
Sources
- Lin et al., AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration (2023)arxiv.org
- Zeghidour et al., SoundStream: An End-to-End Neural Audio Codec (2021)arxiv.org
- Marchisio et al., How Does Quantization Affect Multilingual LLMs? (2024)arxiv.org
- Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (2023)arxiv.org
- Hugging Face Transformers, Quantization overviewhuggingface.co
- Wikipedia, Vector quantizationen.wikipedia.org
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
