Blog5 min read

How to Stream LLM Responses in the Browser (fetch, ReadableStream, and SSE Explained)

How to Stream LLM Responses in the Browser (fetch, ReadableStream, and SSE Explained)

The first time I built a chat UI on top of an LLM, I did the obvious thing. I called the API, awaited the response, and rendered it.

It worked. It also felt terrible. Users stared at a spinner for 8-15 seconds, and then a wall of text appeared all at once. Nobody thought it was broken, but everybody thought it was slow.

That typewriter effect in ChatGPT isn’t an animation. It’s the actual response arriving token by token over a stream. Once you stream, time to first visible text drops from “however long the whole answer takes” to “however long the first token takes”, which is usually well under a second or two.

Here’s how to build it properly, and the bugs I hit along the way.

The big picture

Flow of a streamed LLM response from the browser through your server to the LLM provider and back as SSE events

Three pieces:

  • Your server calls the LLM provider with streaming turned on, and re-emits each chunk to the browser as Server-Sent Events (SSE).
  • The browser reads the response body as a stream using fetch and ReadableStream.
  • The UI appends tokens to state, batched so you don’t re-render 50 times a second.

The API key stays on your server. I’ll keep repeating this because I keep seeing it violated: never call an LLM provider directly from the browser with your secret key. I wrote a whole post on securing AI features on the frontend.

Why not just use EventSource?

The browser has a built-in SSE client called EventSource. It handles parsing and reconnection for you, so it’s tempting.

The problem is it only supports GET requests, and you can’t set custom headers like Authorization. A chat request needs to POST a conversation history that can be tens of kilobytes. You don’t want that in a URL.

Approach POST body Custom headers Auto reconnect Good for
EventSource No No Yes Notifications, live feeds
fetch + ReadableStream Yes Yes No (you build it) LLM chat, anything request/response
WebSocket Yes (messages) Only via handshake workarounds No True bidirectional, like voice or collaboration

For LLM responses, fetch with a streamed body is the sweet spot. It’s one request in, one stream out, and it works with your existing auth, cookies, and HTTP infrastructure.

The server: re-emit tokens as SSE

Here’s a minimal Express route. The llm.stream call stands in for whichever provider SDK you use. Most of them expose streaming as an async iterator.

app.post('/api/chat', requireAuth, async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('X-Accel-Buffering', 'no'); // stop nginx from buffering the stream
  res.flushHeaders();

  const controller = new AbortController();
  // If the user closes the tab or hits Stop, stop paying for tokens.
  res.on('close', () => {
    if (!res.writableEnded) controller.abort();
  });

  try {
    for await (const token of llm.stream(req.body.messages, { signal: controller.signal })) {
      res.write(`data: ${JSON.stringify({ token })}\n\n`);
    }
    res.write('data: [DONE]\n\n');
  } catch (err) {
    if (!controller.signal.aborted) {
      res.write(`event: error\ndata: ${JSON.stringify({ message: 'Generation failed' })}\n\n`);
    }
  } finally {
    res.end();
  }
});

Two things here that cost me real debugging time:

  • Proxy buffering. Everything worked locally, then in production the whole response arrived at once. Nginx was buffering the response. The X-Accel-Buffering: no header fixes it for nginx; other proxies and some compression middleware have their own equivalents.
  • Cancelling upstream. If you don’t abort the provider call when the client disconnects, you keep generating (and paying for) tokens nobody will see.

The client: reading the stream

SSE is a simple text format. Each event is one or more lines starting with data:, and events are separated by a blank line. The tricky part is that network chunks don’t respect those boundaries. One chunk might contain half an event, or three and a half events.

So you keep a buffer, split on blank lines, and always hold back the last piece because it might be incomplete:

export async function streamChat(messages, { signal, onToken }) {
  const res = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
    signal,
  });
  if (!res.ok || !res.body) throw new Error(`Request failed: ${res.status}`);

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    // stream: true keeps multi-byte characters (emoji, accents) intact across chunks
    buffer += decoder.decode(value, { stream: true });

    const events = buffer.split('\n\n');
    buffer = events.pop(); // the last item may be a partial event

    for (const event of events) {
      for (const line of event.split('\n')) {
        if (!line.startsWith('data:')) continue;
        const data = line.slice(5).trim();
        if (data === '[DONE]') return;
        onToken(JSON.parse(data).token);
      }
    }
  }
}

That { stream: true } flag is easy to miss. Without it, an emoji split across two chunks turns into garbage characters. You won’t see it in testing with plain English, and then a user in Japan files a bug.

The SSE spec also allows \r\n line endings. If you don’t control the server, normalize with buffer.replace(/\r\n/g, '\n') before splitting.

Rendering tokens without killing performance

A fast model can push dozens of tokens per second. If every token triggers a React state update and a markdown re-parse, the main thread gets busy and the page starts to feel sticky. Typing in the input lags, and scrolling stutters.

The fix is to batch tokens and flush them at most once per frame:

function createTokenBatcher(flush) {
  let pending = '';
  let frame = 0;

  return {
    push(token) {
      pending += token;
      if (frame) return;
      frame = requestAnimationFrame(() => {
        const chunk = pending; // capture before resetting
        pending = '';
        frame = 0;
        flush(chunk);
      });
    },
    cancel() {
      cancelAnimationFrame(frame);
      frame = 0;
    },
  };
}

// In a React component:
const batcher = useMemo(
  () => createTokenBatcher((chunk) => setText((prev) => prev + chunk)),
  [],
);

See the “capture before resetting” line? My first version did setText(prev => prev + pending) and then pending = ''. React can run that updater function later, during render, and by then pending was already empty. Tokens just vanished. It’s a nice little closure bug, and honestly a good interview question.

Two more rendering tips:

  • Markdown. Re-parsing the entire message on every flush is O(n) per flush, which gets slow for long answers. Split the message into completed blocks (memoized) and one “live” block that you re-parse.
  • Auto-scroll. Only stick to the bottom if the user is already near the bottom. If they scrolled up to read something, yanking them back down on every token is infuriating.

If you want to go deeper on keeping interactions snappy while this is happening, read my post on debugging INP.

Stop, retry, and errors

Streaming adds new failure modes. The request can fail before the first token, or halfway through a sentence.

const controller = new AbortController();
stopButton.onclick = () => controller.abort();

try {
  await streamChat(messages, { signal: controller.signal, onToken: batcher.push });
  setStatus('complete');
} catch (err) {
  if (err.name === 'AbortError') {
    setStatus('stopped'); // keep the partial text, it's still useful
  } else {
    setStatus('error'); // keep partial text too, and offer Retry
  }
}

The rule I follow is that partial output is never thrown away automatically. If the user stopped it, they probably got what they needed. If it failed, they should see what arrived and decide whether to retry. I go into the full state machine in UX patterns for AI features.

Interview angle

This is showing up more and more in frontend interviews, either as “build a ChatGPT clone” in an app round or as a system design question. You can practice the coding version with Build ChatGPT and the design version with Build ChatGPT (Frontend).

Here’s what typically separates answers on this topic:

Junior answers Senior answers
“I’d use WebSockets” Compares SSE, fetch streaming, and WebSockets, then picks one with a reason
Calls the provider from the client Proxies through a server, explains why
setState on every token Batches per frame, mentions markdown re-parse cost
Spinner until done Stop button, partial output kept, retry on error
Doesn’t mention cancellation AbortController on the client and upstream abort on the server

“I’d stream with fetch rather than EventSource because I need to POST the conversation and send auth headers. On the client I’d parse SSE frames with a buffer, batch tokens per animation frame, and use an AbortController so Stop also cancels generation upstream.”

That’s one sentence, and it covers the protocol, the parsing, performance, and cost. That’s the kind of answer that gets a nod.

And that’s it, folks. Streaming is one of those features that looks like a CSS animation and turns out to be a networking, parsing, and rendering problem all at once. If you’re prepping for interviews, the rest of my free material is on frontendlead.com.