Claude Architecture: Context Windows, Output Limits & Token Physics

Your agent worked perfectly in testing, then fell apart on a real codebase: halfway through a long task it started forgetting decisions it made twenty messages earlier, truncating files mid-function, and contradicting its own plan. Nothing in your code changed. What changed is that you ran out of a budget you were never tracking — the context window. Before you tune a single prompt, you have to understand the model as what it actually is: a fixed-size attention budget, and a separate, smaller budget for everything it's allowed to say back.

Core Architectural Concepts & Trade-offs

A Claude model is stateless. It has no memory between calls; the only thing it "knows" on any given turn is whatever you pack into the context window — the system prompt, your tool definitions, the full conversation transcript, any extended-thinking blocks, and the space reserved for the reply. Every one of those competes for the same token pool. On the 2026 lineup that pool is large — Opus 4.8 ships a 200K-token default window with a 1M-token extended mode, and Sonnet 4.6, Haiku 4.5, and Fable 5 sit at their own tiers — but "large" is not "infinite," and the failure mode when you hit the ceiling is silent degradation, not a clean error.

The trap is treating input and output as one quantity. They are not. The context window caps what the model can read; a separate, much smaller max-output limit caps what it can write in a single turn. You can hand Opus a 300K-token monorepo (in extended mode) and still only get back tens of thousands of tokens of generated code per response. That asymmetry is why "summarize this whole repo into one file" quietly truncates: the read fit, the write didn't. Output is the scarce resource, and it is the one developers forget to budget.

Adaptive extended thinking adds a third claimant. When a request is hard enough to warrant it, the model spends tokens on internal reasoning blocks before it answers — and those blocks come out of the same window. A generous thinking budget buys accuracy on genuinely hard problems and wastes latency and tokens on easy ones; the engineering decision is matching the thinking budget to the problem, not maxing it globally. The related cost lever is the cached prefix: the static head of your context (system prompt, tool schemas, a stable document) can be cached so you don't re-pay full price to re-read it every turn. Keep that prefix stable and put it first; anything you mutate near the top invalidates the cache below it.

Put together, these constraints define an operating envelope. The window is RAM, not disk — the moment a transcript no longer fits, something gets evicted, and you'd rather choose what than let attention quietly thin out across a bloated history. The discipline that follows is mechanical: measure tokens before you send, reserve headroom for output and thinking, and treat the window as a budget you actively manage rather than a bucket you fill until it overflows.

Budgeting the Window in Code

Don't guess at token counts — measure them. Anthropic's SDK exposes a token-counting endpoint so you can price a request before you spend output on it. Here is a type-safe budget guard: it counts the input, subtracts a reservation for thinking and output, and refuses to send a request that can't fit a usable reply.

import Anthropic from "@anthropic-ai/sdk"
import type { MessageParam } from "@anthropic-ai/sdk/resources/messages"

const MODEL = "claude-opus-4-8"
// Opus 4.8 operating envelope. Extended mode lifts the window to 1M.
const CONTEXT_WINDOW = 200_000
const MAX_OUTPUT = 32_000      // reply ceiling — the scarce budget
const THINKING_RESERVE = 16_000 // headroom for adaptive thinking

const anthropic = new Anthropic()

export async function guardedSend(system: string, messages: MessageParam[]) {
  // Price the request before spending a single output token.
  const { input_tokens } = await anthropic.messages.countTokens({
    model: MODEL, system, messages,
  })

  const reserved = MAX_OUTPUT + THINKING_RESERVE
  const remaining = CONTEXT_WINDOW - input_tokens - reserved

  if (remaining < 0) {
    throw new Error(
      `Context over budget: ${input_tokens} in + ${reserved} reserved \`
      `> ${CONTEXT_WINDOW}. Compact the transcript or split the task.`,
    )
  }

  return anthropic.messages.create({
    model: MODEL,
    max_tokens: MAX_OUTPUT,
    thinking: { type: "enabled", budget_tokens: THINKING_RESERVE },
    system, messages,
  })
}

The shape that matters: countTokens is a cheap metering call, not the expensive generation. You spend a few milliseconds to learn the price, reserve explicit headroom for max_tokens and the thinking budget, and fail loudly the instant a request can't fit a real answer — instead of discovering it as a truncated file three function calls later.

The Window as a Budget

Visualize one turn's context as a single bar. Everything below shares the same token pool; the model can only generate into whatever slice you leave unspent.

Internalize that bar and most "the model got dumber" mysteries dissolve into budget math. The next lesson builds directly on it: XML-tag structural prompting — how to organize what you put inside the window so the model parses it deterministically instead of guessing at structure. Or jump to the full roadmap.