Eval-Driven Prompt Engineering with Golden Datasets

You tweaked the system prompt to fix one customer's edge case, shipped it, and three days later a different flow is quietly returning garbage — because the "fix" regressed six cases nobody re-checked. Prompts are code with no type system and no test suite by default, so every change is a blind edit to production behavior. You wouldn't merge a refactor without tests. Eval-driven prompt engineering is how you stop merging prompts without them.

Core Architectural Concepts & Trade-offs

The foundation is a golden dataset: a curated set of representative inputs paired with their expected outputs or acceptance criteria. It is the prompt's test suite. The discipline that makes it valuable is sourcing it from reality — real production inputs, especially the failures and edge cases that bit you — not synthetic happy-path examples you invented. Every bug you fix becomes a permanent case in the set, so the same regression can never ship twice. A golden set is a living record of everything your prompt must keep getting right.

Scoring splits by output type. Structured outputs (Lesson 3) get deterministic assertions — exact match, schema validity, a numeric field in range — fast, free, and unambiguous. Open-ended outputs need an LLM-as-judge: a separate model call scoring the response against a rubric (is it grounded? does it answer? is the tone right?). The judge is powerful but fallible, so pin it to a concrete rubric and validate the judge itself against human labels before you trust its scores — an unanchored judge just launders your bias into a number.

The payoff is turning prompt engineering from vibes into a measurable loop. Change the prompt, run the whole golden set, read the pass-rate delta. A change that fixes three cases and breaks five is now visibly a regression instead of a surprise next week. Wire the eval suite into CI as a gate — a prompt change that drops below the pass threshold fails the build, exactly like a unit test — and prompts inherit the same safety net as the rest of your code.

The trade-offs are dataset maintenance and judge cost. A golden set rots if you don't feed it new production failures, and an LLM judge adds a model call per case, so a large suite on every commit costs real tokens and time — route the judge to a cheaper tier (Lesson 3) and reserve the full suite for merge gates while a fast subset runs on every push. The deeper risk is overfitting: optimize too hard against a fixed set and you tune the prompt to the test, not the task. Keep the dataset growing and representative, and it stays an honest proxy for production.

An Eval Harness

Golden cases, deterministic assertions where you can, an anchored judge where you can't, and a pass-rate gate.

// Prompts are code. This is their test suite — sourced from real failures.
type Case = {
  input: string
  assert: (out: Ticket) => boolean   // deterministic where possible
  rubric?: string                    // LLM-judge where output is open-ended
}

// Each entry is a real production case, many born from a past bug.
const GOLDEN: Case[] = [
  { input: "card charged twice",     assert: (o) => o.category === "billing" && o.severity >= 3 },
  { input: "love the new dashboard", assert: (o) => o.category === "feature" }, // regression #214
  { input: "app crashes on upload",  assert: (o) => o.category === "bug" && o.severity >= 4 },
]

async function judge(out: string, rubric: string): Promise<boolean> {
  const res = await anthropic.messages.create({
    model: "claude-haiku-4-5",                 // cheap tier for scoring
    max_tokens: 8,
    system: `Score PASS or FAIL strictly against: ${rubric}`,
    messages: [{ role: "user", content: out }],
  })
  return /PASS/.test(text(res))
}

export async function runEvals(): Promise<number> {
  let passed = 0
  for (const c of GOLDEN) {
    const out = await classify(c.input)        // the prompt under test
    const ok = c.rubric ? await judge(JSON.stringify(out), c.rubric) : c.assert(out)
    if (ok) passed++
    else console.error("FAIL:", c.input, "→", out)
  }
  const rate = passed / GOLDEN.length
  if (rate < 0.95) process.exit(1)             // CI gate: regression fails the build
  return rate
}

Run it in CI and a prompt edit that drops the pass rate below threshold fails the build — the same gate a broken unit test would trip. The prompt can no longer regress silently.

The Eval Gate

With behavior measurable and gated, the agent is finally trustworthy enough to run unattended. The series closes there: autonomous agent routines on a cron. Or browse the full roadmap.