Streaming LLM Responses to React with Server-Sent Events

A model that takes eight seconds to answer feels broken if the user stares at a spinner the whole time, and instant if the words appear as they are generated. The perceived latency of an LLM feature is almost entirely a streaming problem. Get the tokens onto the screen as they arrive and a slow model feels alive.

Why you must stream

Generation is sequential — the model produces tokens one at a time, and you can have them the moment each is ready instead of waiting for the whole completion. Buffering the full response before rendering throws away that free latency win and turns a responsive interface into a loading screen. For any user-facing generation, streaming is not an optimization; it is the baseline.

SSE vs. WebSockets for token flow

Token streaming is one-way: server to client. That is exactly what Server-Sent Events are for — a single HTTP response that stays open and pushes chunks, with automatic reconnection and no protocol ceremony. Reserve WebSockets for genuinely bidirectional needs; for streaming a completion, SSE is simpler and fits the request/response model you already have.

// server: pipe model tokens straight to the client
for await (const chunk of llm.stream(prompt)) {
  res.write(`data: ${JSON.stringify(chunk)}\n\n`);
}
res.end();

// client: append each token as it lands
const es = new EventSource("/api/stream");
es.onmessage = (e) => setText((t) => t + JSON.parse(e.data));

Render without thrashing

Tokens can arrive faster than the eye needs and faster than React wants to reconcile. Append into a single piece of state and keep the streaming text isolated so only that component re-renders, not the whole page. The goal is a smooth crawl of text, not a stutter — buttery output is part of why the feature feels premium.

Users do not experience your model's latency. They experience the moment the first token appears. Stream, and that moment comes almost immediately.

For true bidirectional, high-frequency data, the WebSocket telemetry pattern picks up where SSE leaves off.