# Small Language Model (SLM)

AI Glossary · LLMs & Generative AI
Canonical URL: https://aisat.uk/glossary/small-language-model
Author: Lucy Gilmore, Assistant General Counsel, Product, Privacy & AI at Sprinklr | Practical Privacy, Product & AI Governance & Defensible Risk Decisions (https://www.linkedin.com/in/lucy-gilmore/)
Breadcrumb: [Glossary](https://aisat.uk/glossary) > [LLMs & GenAI](https://aisat.uk/glossary?category=llms) > Small Language Model (SLM)

## TL;DR

> **A Small Language Model (SLM)** is a compact neural network, typically between 1 billion and 14 billion parameters, designed for fast, low-cost inference on targeted tasks. Unlike frontier models built for broad encyclopaedic reasoning, SLMs run efficiently on single GPUs, edge devices, or private cloud instances.

## How Small Language Model (SLM) works in practice

Early language models relied on brute-force parameter scaling to achieve acceptable accuracy. Modern SLMs, such as Llama 3.2, Qwen, Phi, Mistral, and Gemma families, punch far above their weight class because they are trained on heavily filtered, synthetic, and textbook-grade token corpora, often using knowledge distillation from much larger teacher models during pre-training and post-training.

Because an 8-billion-parameter model quantised to 4-bit precision occupies roughly 5 GB of VRAM, it fits comfortably in the memory of a single commodity server GPU or a modern laptop NPU. In production architectures, teams rarely ask an SLM to hold open-ended knowledge in its weights; instead, they pair the SLM with a tight RAG pipeline, constrained JSON grammar decoding, or a LoRA adapter trained on a single high-volume task such as PII redaction, intent routing, or SQL generation.

For example, an NHS trust's digital triage team runs a locally hosted 7-billion-parameter SLM inside its sovereign UK data centre to classify incoming GP referral letters by clinical urgency and specialty. Processing 50,000 letters a day costs a fraction of commercial API pricing while guaranteeing patient records never leave the trust's network perimeter.

## Why it matters and trade-offs

Most enterprise AI traffic does not require a hundred-billion-parameter frontier model. Using a giant general-purpose model to classify support tickets, extract invoice line items, or route search queries is like hiring a barrister to sort the post: slow, expensive, and hard to scale. Routing 80 per cent of routine requests to fine-tuned SLMs while reserving frontier reasoning models for ambiguous edge cases is the standard pattern for sustainable AI unit economics.

Where SLMs struggle is broad world knowledge, complex multi-step planning across unfamiliar domains, and zero-shot recovery from ambiguous instructions. If a task requires synthesising subtle contradictions across twenty disparate documents without task-specific training data, a frontier model remains necessary.

## Related AI concepts

- [Large Language Model (LLM)](https://aisat.uk/glossary/large-language-model): A Large Language Model (LLM) is a deep transformer neural network containing billions of parameters trained on massive text and code corpora to comprehend, summarise, translate, reason over and generate human and programming languages.
- [Fine-Tuning](https://aisat.uk/glossary/fine-tuning): Fine-Tuning is the post-training process of updating a pre-trained model's weights on a curated dataset of task-specific input-output examples to lock in domain vocabulary, strict formatting or lower inference cost.
- [Quantisation](https://aisat.uk/glossary/quantisation): Quantisation is a model compression technique that converts neural network weights and activations from high-precision 16-bit or 32-bit floating-point numbers into lower-bit representations such as INT8, FP8 or INT4, reducing VRAM footprint and accelerating inference.
- [LoRA (Low-Rank Adaptation)](https://aisat.uk/glossary/lora): LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes a pre-trained model's weight matrices and injects small, trainable rank-decomposition matrices into transformer layers, cutting GPU memory and training cost by over 90%.