Evals (Evaluations)

In traditional software, a test passes or fails with binary certainty. In AI applications, changing two words in a system prompt, swapping an embedding model, or upgrading to a new model checkpoint can fix one edge case while quietly breaking twenty others. An evaluation harness replaces guesswork ('vibe checks') with repeatable scoring across a golden dataset of real production traces and adversarial edge cases.

A mature eval pipeline uses three layers of graders. First, fast deterministic code assertions check hard constraints: valid JSON Schema conformance, absence of forbidden regex patterns, citation ID existence, and latency thresholds. Second, LLM-as-a-judge graders use a rubric-prompted model to score subjective dimensions such as answer faithfulness to retrieved RAG chunks, completeness, and tone, calibrated against human annotations. Third, online production monitoring samples live traffic and tracks implicit user feedback such as thumbs-down clicks, prompt re-tries, and copy-paste rates.

For instance, a London fintech running an automated mortgage broker assistant gates every pull request in GitHub Actions against 350 historical broker queries; if context faithfulness drops below 97 per cent or p95 latency exceeds three seconds, the build fails automatically.

Without a bespoke eval suite, AI engineering teams quickly become paralysed: they are afraid to edit system prompts, unable to tell whether a cheaper small language model could replace a frontier API, and blind to retrieval drift when document corpora change. Public leaderboard benchmarks (such as MMLU or SWE-bench) only tell you how a raw model performs on academic tasks, not whether your specific RAG pipeline misreads UK VAT exemptions.

The main trap is building overly complex, uncalibrated LLM-as-a-judge rubrics before looking at real failure traces. Teams should start by manually inspecting 100 real input-output pairs, categorising the exact failure modes they observe, and writing simple binary pass/fail evaluators aligned with those failures.