One Model, Two Jobs: Single-Model RAG for Embeddings and Generation

The reflexive RAG architecture reaches for two models: a specialised embedding model for retrieval and a separate, larger model for generation. It is defensible — best-of-breed at each step. It is also two providers, two SDKs, two sets of credentials, two failure surfaces, and a standing risk that your query embeddings and your corpus embeddings drift into subtly different spaces. On the streamerOS Support Agent I made the opposite call: one model — gemini-flash-latest — does both.

The complexity you delete

Every provider you add is not one integration but a tax on all of them: another key to rotate, another rate limit to reason about, another client to bundle into a constrained Worker, another SDK quirk to learn. Folding embedding and generation into a single model removes an entire axis of operational surface. The architecture diagram loses a box, and so does the on-call runbook.

The subtle win: one latent space

Retrieval works by comparing the query vector to the chunk vectors. That comparison is only meaningful if both vectors live in the same space. When two different models embed the corpus and the query, you are quietly trusting that their geometries align well enough — a trust that erodes the moment either model is versioned underneath you. Use one model for both and the guarantee is free: query and corpus are embedded by the same function, so they are comparable by construction.

// same model, same call — used at index time AND query time
async function embed(text: string, env: Env) {
  const { embedding } = await ai.embed({
    model: "gemini-flash-latest",   // 768-dim
    value: text,
  });
  return embedding;
}

// index time: embed(chunk)  ·  query time: embed(question)
// identical function → guaranteed-comparable vectors

When the trade-off flips

This is not dogma. If your corpus is enormous and retrieval recall is the whole product, a dedicated embedding model tuned for that may earn its keep. If your generation needs frontier reasoning that a flash-tier model cannot provide, split it. The point is to make the choice deliberately: the dual-model setup is the default people reach for without pricing the complexity, and for a bounded, documentation-grounded support agent that complexity buys almost nothing.

Best-of-breed is a real strategy, but so is fewest-moving-parts. For a bounded RAG agent, one model that is good at both beats two models you have to keep in sync.

This unification is one of the load-bearing decisions behind the agent — see the project teardown — and it pairs naturally with an edge-native runtime where every extra dependency has a real cost.