Small Language Model (SLM)

Early language models relied on brute-force parameter scaling to achieve acceptable accuracy. Modern SLMs, such as Llama 3.2, Qwen, Phi, Mistral, and Gemma families, punch far above their weight class because they are trained on heavily filtered, synthetic, and textbook-grade token corpora, often using knowledge distillation from much larger teacher models during pre-training and post-training.

Because an 8-billion-parameter model quantised to 4-bit precision occupies roughly 5 GB of VRAM, it fits comfortably in the memory of a single commodity server GPU or a modern laptop NPU. In production architectures, teams rarely ask an SLM to hold open-ended knowledge in its weights; instead, they pair the SLM with a tight RAG pipeline, constrained JSON grammar decoding, or a LoRA adapter trained on a single high-volume task such as PII redaction, intent routing, or SQL generation.

For example, an NHS trust's digital triage team runs a locally hosted 7-billion-parameter SLM inside its sovereign UK data centre to classify incoming GP referral letters by clinical urgency and specialty. Processing 50,000 letters a day costs a fraction of commercial API pricing while guaranteeing patient records never leave the trust's network perimeter.

Most enterprise AI traffic does not require a hundred-billion-parameter frontier model. Using a giant general-purpose model to classify support tickets, extract invoice line items, or route search queries is like hiring a barrister to sort the post: slow, expensive, and hard to scale. Routing 80 per cent of routine requests to fine-tuned SLMs while reserving frontier reasoning models for ambiguous edge cases is the standard pattern for sustainable AI unit economics.

Where SLMs struggle is broad world knowledge, complex multi-step planning across unfamiliar domains, and zero-shot recovery from ambiguous instructions. If a task requires synthesising subtle contradictions across twenty disparate documents without task-specific training data, a frontier model remains necessary.