FinOps for AI: Routing, Caching, and Cost Attribution

FinOps for AI is treating token spend as a unit-economics problem with an owner, not a surprise on the monthly invoice. At scale, cost per request is an architectural property you design — with three levers in order of leverage: a semantic cache so you never pay for the same answer twice, a router so easy queries don't hit a frontier model, and per-tenant attribution so every dollar has a name. Optimizing a single prompt is tactics; governing the cost curve is architecture.

The cheapest token is the one you never spend

Real traffic is repetitive — the same question phrased a hundred ways. A semantic cache embeds the query and returns a prior answer when a near-identical one exists, which collapses your most common questions to roughly zero cost and zero latency. It's the first lever because it's pure subtraction: a cache hit isn't a cheaper model call, it's no model call at all. Everything downstream only sees genuine cache misses.

Route to the cheapest model that can answer

Sending every request to your most capable model is paying frontier prices for questions a small model answers perfectly. A router classifies difficulty first and sends the easy majority to a cheap, fast model, escalating only the hard minority. If a small model handles 80% of traffic at a fraction of the per-token price, your blended cost drops by most of that 80% — without the users on the easy path noticing anything but lower latency. Quality gates on the cheap path catch the rare misroute and escalate it.

// don't pay twice · don't overpay · always know who pays
const hit = await cache.lookup(q);
if (hit) return ledger.record(tenant, "cache", 0), hit;

const tier = classify(q);        // "easy" | "hard"
const model = tier === "easy" ? CHEAP : FRONTIER;
const res = await model.generate(q);

ledger.record(tenant, model.name, cost(res.usage)); // per-tenant showback
return res;

Attribution turns a bill into a budget

An unlabeled invoice is undebuggable: you can see spend doubled but not which tenant, feature, or model did it. Tag every call with the tenant and write a ledger entry, and cost becomes a dimension you can group by — showback per customer, a budget alert before a runaway loop empties the account, a clear answer to "is this feature profitable?". Attribution is the difference between knowing your costs and merely paying them.

AI cost isn't a number you discover at month end — it's one you architect. Cache so you never pay twice, route so you never overpay, and attribute so every token traces to an owner.

FinOps is the governance layer over the tactical token economics of a single app, and it leans on the same router pattern that directs multi-agent work. Continue on the roadmap.