# Quantisation

AI Glossary · Deployment, Ops & Cost
Canonical URL: https://aisat.uk/glossary/quantisation
Author: Lucy Gilmore, Assistant General Counsel, Product, Privacy & AI at Sprinklr | Practical Privacy, Product & AI Governance & Defensible Risk Decisions (https://www.linkedin.com/in/lucy-gilmore/)
Breadcrumb: [Glossary](https://aisat.uk/glossary) > [Deployment & Ops](https://aisat.uk/glossary?category=deployment) > Quantisation

## TL;DR

> **Quantisation** is a model compression technique that reduces the numerical precision of a neural network's weights and activations, such as converting 16-bit floating-point numbers (FP16/BF16) into 8-bit or 4-bit integers (INT8/INT4/FP8). It cuts GPU memory footprint and boosts inference speed with minimal accuracy loss.

## How Quantisation works in practice

A language model's parameters are stored as millions or billions of floating-point numbers. In standard 16-bit precision (`BF16` or `FP16`), every billion parameters requires 2 gigabytes of GPU VRAM just to hold the weights, before accounting for the KV cache needed for active context windows: a 70-billion-parameter model needs roughly 140 GB of VRAM, requiring multiple expensive data-centre GPUs. Worse, LLM token generation is typically memory-bandwidth bound rather than compute bound; the GPU cores spend most of their time waiting for weights to move from VRAM into registers.

Quantisation maps those high-precision continuous numbers onto a smaller grid of discrete values using scaling factors and zero-points. Most production teams use Post-Training Quantisation (PTQ) methods such as AWQ (Activation-aware Weight Quantisation), GPTQ, or GGUF, which calibrate a small sample of data to protect the less than 1 per cent of 'outlier' weights that disproportionately affect model accuracy, or native `FP8` execution on modern Hopper and Blackwell GPUs.

For example, a Sheffield software house quantised a 70-billion-parameter open-weight model from BF16 to 4-bit AWQ, shrinking its VRAM footprint from 140 GB to ~38 GB so it runs at 45 tokens per second on a single cloud GPU instance instead of an expensive multi-GPU node.

## Why it matters and trade-offs

Quantisation directly changes the deployment economics of self-hosted and edge AI. Dropping from 16-bit to 8-bit (`FP8` or `INT8`) is nearly lossless on modern benchmarks while nearly doubling throughput and halving hardware costs; 4-bit quantisation enables local coding models on developer laptops and high-density multi-LoRA serving in private clouds.

The trade-off appears when pushing down to aggressive 4-bit or 3-bit schemes on complex reasoning, arithmetic, or long-context retrieval tasks, where subtle perplexity degradation can cause higher hallucination rates or broken JSON formatting. Teams should never trust generic perplexity scores alone; always run your domain-specific eval suite against the quantised checkpoint before promoting it to production.

## Related AI concepts

- [Small Language Model (SLM)](https://aisat.uk/glossary/small-language-model): A Small Language Model (SLM) is a compact neural language model, typically between one billion and ten billion parameters, optimised for low-latency, low-cost inference on a single GPU or on-device NPU for domain-specific workflows.
- [LoRA (Low-Rank Adaptation)](https://aisat.uk/glossary/lora): LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pre-trained model's weight matrices and injects small, trainable rank-decomposition matrices into transformer layers, cutting GPU memory and training cost by over 90%.
- [Large Language Model (LLM)](https://aisat.uk/glossary/large-language-model): A Large Language Model (LLM) is a deep transformer neural network containing billions of parameters trained on massive text and code corpora to comprehend, summarise, translate, reason over and generate human and programming languages.
- [Fine-Tuning](https://aisat.uk/glossary/fine-tuning): Fine-Tuning is the post-training process of updating a pre-trained model's weights on a curated dataset of task-specific input-output examples to lock in domain vocabulary, strict formatting or lower inference cost.