Streaming LLM Responses: Render Tokens in a Chat UI Without Jank

Streaming LLM responses looks easy on a fast laptop with English text, and then breaks for real users. Hindi turns into black diamonds, words go missing, the page stutters, and the window keeps dragging readers back down.

Picture a common setup. A homework-help app for school students in India answers in Hindi and English, and each answer appears word by word. The setup is made up, but every decoding and parsing bug below was reproduced on Node.js 22.

Here is what its first version got wrong:

• Garbled letters. A student asks in Hindi and sees न���स्ते instead of नमस्ते, because a network piece ended in the middle of a character.
• Broken lines. One line of the stream arrives in two pieces, and the code reads only the first half.
• Jank, the stutter you see when the browser misses screen redraws. The app rebuilt the whole answer for every token, so scrolling felt sticky on a budget phone.
• Scroll fights. A student scrolls up to re-read step 2, and each new word yanks the view back to the bottom.
• A stop button that only hid the text while the server kept writing.

Each problem has a small, known fix. This guide builds them in order on one running example: the app answering "What is photosynthesis?" in Hindi and English.

You need seven words: token, byte, UTF-8, chunk, stream, SSE and frame. Each gets one plain meaning here.

Token. A token is the small piece of text a model writes in one step, often a word or part of a word. Servers usually send one or a few tokens at a time.

Byte and UTF-8. Networks carry bytes, not letters. UTF-8 is the standard rule for turning characters into bytes. An English letter takes 1 byte, a Devanagari character like न takes 3, and the waving-hand emoji takes 4.

Chunk. A chunk is one batch of bytes the network hands your code. Its size is decided by the network and the servers along the way, not by where letters, words or lines end.

Stream. A stream is data you read piece by piece as it arrives, instead of waiting for all of it. In the browser, fetch gives you the response body as a ReadableStream, a stream object you read one chunk at a time.

SSE and frame. Server-sent events, or SSE, is a plain-text format for sending many small events down one long HTTP response. HTTP is the protocol browsers use to talk to servers. A frame is one redraw of the screen. Many screens redraw 60 times a second, about every 16.7 milliseconds.

The server keeps one HTTP response open and writes a small event into it each time the model produces tokens. The browser reads that response chunk by chunk, turns bytes into text, cuts text into events, and paints the new words.

Follow the diagram from top to bottom. Notice that three boxes in the middle each keep a small leftover between chunks. Those leftovers are where most streaming bugs live.

An everyday picture helps with the leftovers. Think of a courier bringing a long handwritten letter in sealed boxes of random sizes. A word, or even one letter of a word, may start in one box and finish in the next.

Here is how the picture maps. The boxes are chunks. The letter is the event stream. Your unpacking table, where you keep a torn half until its other half arrives, is the leftover inside the decoder and the parser.

Where the picture breaks: a real courier can lose or reorder boxes. On one HTTP connection, bytes arrive complete and in order. What can fail is the whole connection, which the resume section covers.

Use fetch with a stream reader for most AI chats. A chat usually sends a POST request with the question and history as JSON, a common text format for data, plus an Authorization header. EventSource can do neither: its only option is withCredentials, and it makes GET requests.

We checked this on Node.js 22.22.0, which ships an EventSource behind the --experimental-eventsource flag. We passed a POST method and an Authorization header to the constructor. The server still saw a GET with no Authorization header.

EventSource still fits some jobs, such as a GET status feed where a login cookie proves who the user is. Setting withCredentials to true also sends cookies to another site, a cross-origin request. That server must reply with Access-Control-Allow-Credentials: true, or the browser reports a network error. Putting a token in the URL also works, but URLs often land in server logs.

SSE stays useful even when you use fetch. It is a simple, well-specified text format, so the server can keep sending it and the browser can parse it by hand. For a fuller comparison with WebSockets, see the related guide on real-time options.

Create one TextDecoder per response, call decode(chunk, { stream: true }) for every chunk, and call decode() once with no arguments at the end. With stream: true, the decoder keeps the bytes of an unfinished character and joins them to the next chunk.

Pause and predict. The text नमस्ते, a space and the waving-hand emoji make 23 bytes. If a chunk ends after byte 4, what does a fresh decoder show for each half?

const bytes = new TextEncoder().encode("नमस्ते \u{1F44B}"); // \u{1F44B} is the waving-hand emoji
console.log(bytes.length); // 23 bytes for 8 code points

// cut after byte 4: all of न, plus 1 of the 3 bytes of म
const first = bytes.slice(0, 4);
const second = bytes.slice(4);

// naive: a fresh decode for each chunk
const wrong = new TextDecoder().decode(first) + new TextDecoder().decode(second);
console.log(wrong); // न���स्ते and the emoji: three U+FFFD marks

// correct: one decoder, stream: true, then one final flush
const decoder = new TextDecoder();
let right = decoder.decode(first, { stream: true });
right += decoder.decode(second, { stream: true });
right += decoder.decode();
console.log(right === "नमस्ते \u{1F44B}"); // true

What just happened? The naive line decoded each half on its own. The first half ended with 1 of the 3 bytes of म, so the decoder wrote U+FFFD, the replacement mark that shows as a black diamond. The 2 orphan bytes at the start of the second half became two more.

With stream: true, the same decoder kept that lone byte and finished the character when the next chunk came. The Encoding Standard describes exactly this: call decode with stream set to true zero or more times, then once without it.

This bug hides in plain ASCII English testing, because every ASCII character is 1 byte and a cut never lands inside one. Curly quotes, the rupee sign and emoji in English answers take 3 or 4 bytes, so they can still be split.

In our run, a local server sent a short Hindi greeting in 5-byte chunks. The naive version damaged 2 characters and showed 5 replacement marks. The stream: true version damaged none.

• TextDecoder removes a byte order mark by default. That is an invisible marker some tools add at the start of text, and the SSE spec says to strip it too.
• If the stream ends in the middle of a character, the final decode() outputs one U+FFFD instead of silently dropping the bytes. That is a useful signal that the server cut off mid-answer.
• response.body.pipeThrough(new TextDecoderStream()) does the same job as a stream. We fed it the same split bytes and got the greeting back intact.

Keep a text buffer, cut it into complete lines, and save the last unfinished piece for the next chunk. A blank line ends an event, and only then do you hand the event to the UI.

The SSE format in five rules

These rules come from the WHATWG HTML Standard's server-sent events section, the official spec that browsers follow.

• Each line is a field name, a colon and a value, such as data: hello. One space after the colon is removed, and field names are case-sensitive.
• A line starting with a colon is a comment. Servers send these as keep-alives, and the spec suggests one every 15 seconds or so for old proxies.
• All data lines in one event are joined with a newline. The event field sets a custom type, such as tool; without it the type is message.
• The id field sets the last event ID used for resuming. The retry field sets EventSource's reconnection delay in milliseconds.
• Lines may end with CRLF (carriage return plus line feed), LF alone or CR alone. If the stream ends before an event's blank line, that event is dropped.

Why the naive parser fails

The naive version splits each chunk on newlines and reads every line that starts with data. It is wrong, because chunks do not respect lines. In our test, the line data: {"t":"Hel"} was split after He, and the naive code produced an event holding only {"t":"He, which is not even valid JSON.

The diagram shows the hardest case: an event whose CRLF line ending is itself cut in two. Look at the decision box. It is the one step most hand-written parsers leave out.

A parser that survives any split

Here is the parser. It follows the spec's rules for the data, event and id fields. It skips retry, which only matters if you build your own reconnect timer.

function createSseParser(onEvent) {
  let buffer = "";
  let data = [];
  let eventType = "";
  let lastId = "";

  function handleLine(line) {
    if (line === "") {
      // blank line: the event is complete
      if (data.length) {
        onEvent({ type: eventType || "message", data: data.join("\n"), id: lastId });
      }
      data = [];
      eventType = "";
      return;
    }
    if (line.startsWith(":")) return; // comment, often a keep-alive
    const colon = line.indexOf(":");
    const field = colon === -1 ? line : line.slice(0, colon);
    let value = colon === -1 ? "" : line.slice(colon + 1);
    if (value.startsWith(" ")) value = value.slice(1);
    if (field === "data") data.push(value);
    else if (field === "event") eventType = value;
    else if (field === "id" && !value.includes("\0")) lastId = value;
  }

  function feed(text) {
    buffer += text;
    // a CR at the very end may be the first half of CRLF, so keep it
    const hold = buffer.endsWith("\r") ? "\r" : "";
    if (hold) buffer = buffer.slice(0, -1);
    const lines = buffer.split(/\r\n|\r|\n/);
    buffer = lines.pop() + hold;
    lines.forEach(handleLine);
  }

  function end() {
    // a lone CR at the very end really was a line ending
    if (buffer.endsWith("\r")) {
      buffer.slice(0, -1).split(/\r\n|\r|\n/).forEach(handleLine);
    }
    buffer = ""; // anything else is an unfinished event: drop it
  }

  return { feed: feed, end: end };
}

What just happened? The feed function joins new text to the saved leftover and splits on any of the three line endings. The last piece may be unfinished, so it goes back into the buffer. The end function drops a half-written event, as the spec says. With servers that end lines with CR alone, a finished event waits until the next byte arrives, which keep-alive comments supply.

The hold step fixes a bug we reproduced. CRLF is two characters. If a chunk ends right after the CR, a parser without the hold treats the next chunk's LF as a second, empty line. That fake blank line ends the event too early.

In our test, the stream data: a, CRLF, data: b, CRLF, CRLF was split after the first CR. The parser without the hold produced two events, a and b. This parser produced one event, a and b joined by a newline, which is correct.

To check the parser, we fed a test stream at every one of its 72 possible split points, and again one character at a time. Every run gave the same events. We also streamed a tricky input one byte at a time to Node.js 22's built-in EventSource, and both produced identical events.

Pass an AbortController's signal to fetch, and call abort() when the user presses stop. The waiting read then fails with a DOMException, the browser's standard error object, named AbortError. So your code can tell a stop apart from a real failure.

First, the reading loop. It sends the question as a POST with JSON and a token header, then feeds each chunk through the decoder and the parser.

async function streamAnswer(url, payload, token, onEvent, signal) {
  const res = await fetch(url, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": "Bearer " + token,
    },
    body: JSON.stringify(payload),
    signal: signal,
  });
  if (!res.ok) throw new Error("Server said " + res.status);

  const decoder = new TextDecoder("utf-8");
  const parser = createSseParser(onEvent);
  const reader = res.body.getReader();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    parser.feed(decoder.decode(value, { stream: true }));
  }
  parser.feed(decoder.decode()); // flush any bytes still held
  parser.end();
}

A few lines matter most. res.ok is true only for status codes 200 to 299, so errors do not get parsed as events. reader.read() resolves with done and value, and value is a Uint8Array, an array of raw bytes. The last two calls flush the decoder and the parser.

Now the stop button. Each answer gets its own controller, called mine, and the button aborts whichever answer is current.

let controller = null;

async function ask(question) {
  if (controller) controller.abort(); // a new question stops the old answer
  const mine = new AbortController();
  controller = mine;
  try {
    await streamAnswer(API, { question: question }, token, handleEvent, mine.signal);
    finishMessage("done");
  } catch (err) {
    if (err.name === "AbortError") finishMessage("stopped");
    else finishMessage("error");
  } finally {
    mine.abort(); // closes the request even after an error
    if (controller === mine) controller = null;
  }
}

stopButton.addEventListener("click", function () {
  if (controller) controller.abort();
});

In our test, pressing stop 150 milliseconds in ended the call with the stopped status, and the words already shown stayed on screen. Keep them there and add a small "stopped" label, because a student may stop once they have read enough.

Two details stop quiet leaks. A new question aborts the old answer, and finally clears controller only if it still points at this answer. In our test, two overlapping questions both ended stopped, with no stream left open.

The mine.abort() call in finally also closes the request when something throws mid-stream, such as bad JSON in an event. Without it, our test request stayed open after the error. In our test, calling abort() after an answer had fully arrived changed nothing.

Stopping the browser side is only half the job. Aborting cancels the request. Over HTTP/1.1, as in our Node.js test, that closes the connection: the server's close event fired at once and it stopped writing. Over HTTP/2, many requests share one connection, so the server sees just that request cancelled. Your server must listen for that close and cancel the model call. Otherwise generation keeps running and may still be billed.

Collect tokens in a string and write them to the page at most once per frame with requestAnimationFrame. Tokens can arrive faster than the screen redraws, and every extra page update is work the user never sees.

The DOM is the browser's live tree of page elements, and each change to it can make the browser redo layout and painting. requestAnimationFrame runs your function just before the next repaint, which is exactly one place per frame to make changes.

let pending = "";
let frameRequested = false;

function onToken(text) {
  pending += text;
  if (!frameRequested) {
    frameRequested = true;
    requestAnimationFrame(flush);
  }
}

function flush() {
  frameRequested = false;
  if (pending === "") return;
  target.appendData(pending); // one DOM write per frame
  pending = "";
  afterPaint(); // scroll if the reader is following
}

We tested this logic with a fake frame clock. 300 tokens arrived every 4 milliseconds while a 60 Hz screen redrew every 16.7 milliseconds. The code made 72 DOM writes instead of 300, and the final text was identical.

• requestAnimationFrame is one-shot, as MDN notes. Each flush needs a new request, which onToken makes when the next token arrives.
• Most browsers pause requestAnimationFrame in background tabs. Call flush() yourself when the stream ends, or the last words wait until the student switches back.
• While the answer streams, add plain text through a text node, for example with appendData. Plain text cannot run a script, which matters because model output is untrusted input.

Expensive, and it gets worse as the answer grows. If you re-parse the whole message for each token, token 1,000 re-reads the 999 before it, so total work grows with the square of the answer's length.

We measured this with the marked markdown library, version 18.1.1, on Node.js 22.22.0. The test answer had 10,904 characters with paragraphs, lists and code blocks, sent as 2,726 tokens of 4 characters each.

Read the middle column, not the times. Times were measured on a server processor, and a budget phone will be slower. The amount of text parsed is the part that carries over to every device.

Partial markdown changes shape as it streams

Half-written markdown also renders differently from the finished version, which makes text jump. With marked 18.1.1, a word with only its opening pair of asterisks shows them as plain text. When the closing pair arrives, the word suddenly turns bold.

An unclosed code fence is worse. A code fence is the line of three backticks that starts a code block. Until the closing fence arrives, everything after the opening one renders as code, including paragraphs that will end up outside it.

Re-parse only the unfinished tail

A blank line outside a code fence usually ends a markdown block. So in most answers, everything before the last such line is finished and will not change. Parse that part only when it grows, about once per paragraph, and re-parse just the short tail on each frame.

function splitAtLastSafeBreak(text) {
  const lines = text.split("\n");
  let inFence = false;
  let lastBreak = -1;
  lines.forEach(function (line, i) {
    if (line.trimStart().startsWith("```")) inFence = !inFence;
    else if (!inFence && line.trim() === "") lastBreak = i;
  });
  if (lastBreak === -1) return { stable: "", tail: text };
  return {
    stable: lines.slice(0, lastBreak).join("\n"),
    tail: lines.slice(lastBreak + 1).join("\n"),
  };
}

For our test answer, rendering the stable part and the tail separately matched a full render exactly. That is how the third row of the table got its numbers.

This is a shortcut, not a markdown parser, so know where it breaks. It only knows backtick fences, not tilde fences. A list with blank lines between its items rendered as two separate lists in our test. Do one full parse when the stream ends to fix any difference.

Scroll to the bottom after each paint only if the reader was already near the bottom. Check that whenever they scroll, and stop following the moment they move up to re-read.

const NEAR_BOTTOM_PX = 40;
function isNearBottom(el) {
  const gap = Math.max(0, el.scrollHeight - el.clientHeight - el.scrollTop);
  return gap === Math.min(gap, NEAR_BOTTOM_PX);
}

let follow = true;

chatList.addEventListener("scroll", function () {
  follow = isNearBottom(chatList);
});

function afterPaint() {
  if (follow) chatList.scrollTop = chatList.scrollHeight;
}

Here gap is how many pixels of content sit below the visible area. The Math.min line simply asks whether gap is at most 40 pixels. It is written that way because this page cannot display a less-than sign.

MDN explains why you need a threshold at all. scrollTop can be a fraction while scrollHeight and clientHeight are rounded, so an exact equality check fails at random. Math.max(0, ...) only clamps a tiny negative gap from a fractional scrollTop; MDN's Math.abs version works just as well.

When follow is false, show a small "Jump to latest" button over the chat. It tells the student new text is arriving without moving their view.

Send tool steps as their own named SSE events, such as event: tool, and show them in a status line that has its space reserved from the start. A tool call is when the model asks your server to run something, such as a search, before it answers.

The parser already hands you the event type, so one small function can route each event. The JSON shapes here are our own example; your server's format will differ.

function handleEvent(e) {
  const payload = JSON.parse(e.data);
  if (e.type === "message") onToken(payload.t);
  else if (e.type === "tool") statusLine.textContent = "Looking up: " + payload.name;
  else if (e.type === "tool_done") statusLine.textContent = "";
  else if (e.type === "done") flush();
}

In our test, a tool event set the status line to Looking up: class 7 science notes, and the tool_done event cleared it. The Hindi tokens that followed reached onToken in order.

Give the status line a fixed height in CSS even when it is empty. Then a status appearing or disappearing does not push the answer text down while the student is reading it.

Do not make the growing answer a live region that reads out every token. A live region is a part of the page whose changes a screen reader announces. Mark the message as busy while it streams, then announce once, politely, when it is done.

The aria-live attribute sets how a region is announced. MDN describes polite as waiting for a graceful moment, such as the end of the current sentence. Assertive interrupts at once, and MDN warns against it unless the interruption is imperative.

1. 1

   Start with an empty status region

   Put an empty element with aria-live set to polite in the page's first HTML. MDN calls this the most reliable way to make sure screen readers register it.

2. 2

   Mark the message busy

   When streaming starts, set aria-busy to true on the answer. MDN describes this as telling assistive technology to wait until the changes are complete.

3. 3

   Announce once at the end

   When the stream ends, set aria-busy to false and write a short line such as Answer ready into the status region.

4. 4

   Keep the stop button reachable

   Use a real button element with a clear label like Stop answer, so keyboard and screen reader users can reach it while text streams.

EventSource reconnects by itself and sends a Last-Event-ID header. With fetch, you do the same by hand. Either way, resuming only works if the server stored the events and can replay the ones after that ID.

We watched this happen on Node.js 22's EventSource. After the server closed the stream, the client reconnected with Last-Event-ID: 41 and received event 42. A 404, a 204 or a wrong content type closed it for good, matching the spec's rule that any status except 200 fails the connection.

With fetch, send Last-Event-ID as a normal request header. It is not on MDN's list of forbidden request headers, and our Node.js test server received it. CORS is the browser's set of rules for requests to another site. On a cross-origin request, the Authorization, Last-Event-ID and JSON Content-Type headers are all outside MDN's list of CORS-safelisted headers. So the browser first sends a preflight, a check request, and your server must allow them.

Wait a random time between tries, up to 1 second, then up to 2, then up to 4, with a cap. The randomness spreads students out, so an outage is not met by everyone reconnecting together. Do not retry after the student pressed stop, and do not retry after an HTTP error such as 401, which needs a fresh login instead.

Usually a proxy is buffering the response. nginx, a common web server and proxy, buffers proxied responses by default. It first collects the response in its own buffers, so small events can reach the browser late and bunched together.

The nginx documentation gives two fixes. Set proxy_buffering off for the streaming route, so nginx passes data on as soon as it arrives. Or have your app send the response header X-Accel-Buffering: no. Other layers between server and browser can also hold data back, so test the real path.

To check, run curl -N against the streaming URL; the -N flag turns off curl's own buffering. You should see events appear one by one. If they arrive all together after a pause, something in between is buffering.

Long answers have a second risk: quiet gaps while the model runs a tool. The SSE spec notes that old proxies can drop idle connections and suggests a comment line every 15 seconds or so. The parser above already ignores those lines.

Walk the answer from server to screen, naming one failure and its fix at each step.

1. 1

   Pick the transport

   Say you use one long HTTP response with SSE as the text format, read through fetch, because the chat needs POST and an Authorization header.

2. 2

   Decode safely

   One TextDecoder per response with stream: true, because Hindi characters and emoji span several bytes and chunks split them.

3. 3

   Parse with a buffer

   Keep the unfinished line between chunks, dispatch on a blank line, and handle a CRLF split across chunks.

4. 4

   Paint per frame

   Buffer tokens and write once per animation frame. Cache finished markdown blocks and re-parse only the tail.

5. 5

   Respect the reader

   Follow the bottom only when the reader is there, mark the message aria-busy, and announce politely when done.

6. 6

   Handle stop and drops

   AbortController for stop with a server-side cancel, event IDs for resume, and turn off proxy buffering on the route.