Evaluation-Driven Development: The Golden Dataset as a CI Gate

Evaluation-Driven Development makes a golden dataset the test suite for your AI: a curated set of labelled cases that every prompt or model change runs against in CI, gated on a regression threshold. Without it, a prompt is code with no tests — you tweak a line, it feels better on the three examples you tried, and you ship a silent regression on the other ninety-seven. The golden set turns "feels better" into a number that either clears the bar or blocks the merge.

A prompt change is a diff without a test

The danger of prompt engineering is how cheap it is to change and how invisible the side effects are. Fix one failing case by adding an instruction, and you may quietly break a category you weren't looking at. Manual spot-checking can't catch it because you check what you expect to change, not what you don't. The only defense is the same one that works for code: a fixed set of cases, run every time, that fails loudly when behavior regresses.

The golden dataset is the asset, not the prompt

Prompts and models churn; the labelled set of inputs and expected outcomes is what compounds. Seed it from real failures — every production bug becomes a permanent case so the same mistake can never ship twice — and grow it toward the long tail and the edge cases that break naive prompts. Each row is an input, a way to judge the output (exact match, a rubric, an LLM-as-judge score), and a label. Curating that set is the real engineering work; the prompt is just the current candidate being measured against it.

// every prompt/model change is scored against fixed cases
const cases = loadGolden("./golden/*.jsonl");
const results = await Promise.all(
  cases.map(async (c) => ({
    pass: await judge(c, await candidate.run(c.input)),
  })),
);

const rate = results.filter((r) => r.pass).length / results.length;
expect(rate).toBeGreaterThanOrEqual(BASELINE); // regression fails CI

Gate the merge, not the vibe

The discipline only bites when the threshold is wired into CI. Store the current pass-rate as the baseline; a pull request that drops below it fails the check and can't merge, exactly like a broken unit test. That single gate changes the culture: prompt changes stop being judged by whoever argues hardest and start being judged by the dataset. "Trust me, it's better" becomes "the pass-rate went from 91% to 94%."

You don't earn confidence in an AI feature by trying a few prompts until one feels right. You earn it with a golden dataset that fails the build the moment a change makes things worse — and grows every time production finds a case you missed.

Eval-driven development is the test harness around the scoring methods in evaluating LLM outputs and the safety net that lets guardrails evolve without silent regressions. Continue on the roadmap.