Mastering Real-Time AI: The Ultimate Guide to Streaming LLM Responses in React

In the current landscape of generative AI, user experience is defined by speed. If you have used ChatGPT, Claude, or Perplexity, you have noticed the characteristic "typing" effect where the response appears word-by-word. This isn't just a stylistic choice; it is a technical necessity. Large Language Models (LLMs) are computationally expensive and can take several seconds—or even minutes—to generate a complete response.

Waiting for the entire JSON payload to finish generating before showing anything to the user results in a "dead" UI and high drop-off rates. Streaming solves this by leveraging Server-Sent Events (SSE) or the Fetch API’s ReadableStream to pipe data to the client as it is generated. For Indian developers building the next generation of SaaS tools, mastering streaming is the difference between a clunky prototype and a world-class AI product.

Why Streaming is Non-Negotiable for AI Applications

When building AI-powered features, the most critical metric is Time to First Token (TTFT). In a traditional request-response cycle, the user triggers an action and waits until the server receives the full output from the LLM (like GPT-4 or Llama 3), packages it into a JSON object, and sends it back.

Improving Perceived Performance

Streaming reduces the perceived latency to almost zero. As soon as the model generates the first few tokens, the user sees text on their screen. This keeps the user engaged and provides immediate feedback that the system is working. In the competitive Indian tech ecosystem, where mobile networks can vary in stability, providing a lightweight, incremental data flow is significantly more resilient than a heavy, single-shot request.

Handling Long-Form Content

If your application generates long-form reports, code blocks, or creative writing, waiting for a 1,000-word response can take 30 seconds or more. Without streaming, the connection might even time out at the gateway level (like Nginx or Vercel functions). Streaming maintains an open connection, trickling data through and preventing timeout errors while providing a smooth reading experience.

The Mechanics of Streaming: ReadableStream and SSE

To implement streaming in React, we need to understand how data travels from the LLM provider to the browser. There are two primary ways to handle this: Server-Sent Events (SSE) and the Fetch API with ReadableStream.

Server-Sent Events (SSE)

SSE is a standard that allows servers to push data to web pages over HTTP. It is a uni-directional stream where the server keeps the connection open. While robust, it can sometimes be restrictive with headers and custom POST bodies.

The Fetch API and ReadableStream

This is the modern standard for most AI applications. When you call an LLM API (like OpenAI's /v1/chat/completions) with stream: true, the response body is not a blob of text but a ReadableStream. In your React frontend, you can hook into this stream using a reader and process chunks of data as they arrive.

Comparing Implementation Strategies

| Feature | Traditional JSON | Streaming (Fetch/ReadableStream) |
| :

---

| :

---

| :

---

|
| Time to First Token | High (3-10+ seconds) | Very Low (< 500ms) |
| User Experience | Static Loading Spinners | Dynamic, Interactive |
| Implementation | Simple fetch | Complex (Buffer & Stream handling) |
| **Connection Type** | Short-lived | Persistent until completion |
| **Best Use Case** | Summaries, Data Tagging | Chatbots, Content Generation |

Building the Backend: A Next.js Route Handler

Before we jump into the React code, we need an endpoint that supports streaming. If you are using Next.js (the weapon of choice for many Indian startups), the App Router makes this incredibly straightforward.

Setting up the Stream

Using the OpenAI SDK as an example, you can initialize a stream and return a StreamingTextResponse.

import { OpenAIStream, StreamingTextResponse } from 'ai';
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req) {
  const { messages } = await req.json();
  const response = await openai.chat.completions.create({
    model: 'gpt-4',
    stream: true,
    messages,
  });

  const stream = OpenAIStream(response);
  return new StreamingTextResponse(stream);
}

This backend setup ensures that the tokens are forwarded to the client as soon as they are emitted by the model, rather than being buffered on the server.

Frontend Architecture: Implementing the Stream Handler in React

Now for the core task: consuming that stream in your React component. We will avoid heavy libraries for a moment to understand the "vanilla" logic required to read a stream.

Reading the Stream Chunks

The fetch API returns a response.body which implements the ReadableStream interface. You use a TextDecoder to convert binary chunks into strings.

const handleStream = async () => {
  const response = await fetch('/api/chat', {
    method: 'POST',
    body: JSON.stringify({ messages: [{ role: 'user', content: 'Hello!' }] }),
  });

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let done = false;

  while (!done) {
    const { value, done: doneReading } = await reader.read();
    done = doneReading;
    const chunkValue = decoder.decode(value);
    // Update your React state here with chunkValue
    setCompletion((prev) => prev + chunkValue);
  }
};

Managing State Transitions

When streaming text, you are updating the state hundreds of times per second. If your component is complex, this can lead to performance bottlenecks. It is often better to use a useRef to track the accumulated text and a debounced state update, or utilize useReducer to manage the message history effectively.

Creating a Robust Custom Hook for LLM Streams

To make this reusable across your application, it is best to encapsulate the logic in a custom hook. This hook should handle the connection, errors, and the accumulated response state.

The useLLMStream Hook

This hook provides a clean interface for any component that needs AI generation.

import { useState, useCallback } from 'react';

export const useLLMStream = () => {
  const [data, setData] = useState('');
  const [isLoading, setIsLoading] = useState(false);

  const generate = useCallback(async (prompt) => {
    setIsLoading(true);
    setData('');
    
    try {
      const response = await fetch('/api/generate', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ prompt }),
      });

      const reader = response.body?.getReader();
      const decoder = new TextDecoder();

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        const text = decoder.decode(value, { stream: true });
        setData((prev) => prev + text);
      }
    } catch (error) {
      console.error("Streaming error:", error);
    } finally {
      setIsLoading(false);
    }
  }, []);

  return { data, generate, isLoading };
};

Integrating with UI Components

Once the hook is ready, using it in a component like a chat window or a blog post generator becomes trivial. You can map the data state directly to your UI, ensuring that as data grows, the UI reflects the new tokens immediately.

Handling Markdown and Code Syntax in Real-Time

One challenge with streaming is that the raw text often contains Markdown. Since the stream is incomplete, a Markdown parser might struggle with "half-written" syntax (e.g., a code block that hasn't been closed with triple backticks yet).

Using react-markdown

Libraries like react-markdown are surprisingly resilient. They will attempt to render whatever valid Markdown they can find and treat the rest as plain text until the closing tags arrive.

Code Highlighting Challenges

For code syntax highlighting (using react-syntax-highlighter), you might notice a flickering effect as the language is identified. To prevent this, ensure your LLM is prompted to specify the language immediately after the backticks (e.g., ```javascript). You can also implement a "buffer" that waits for a newline before attempting to render a complex block, though for most use cases, react-markdown's default behavior is sufficient.

Auto-Scrolling for UX

As the text streams down, the container height increases. To provide a professional feel, you should implement an "auto-scroll" effect. Using a useEffect that monitors the length of your streamed data and triggers a scrollIntoView on a dummy div at the bottom of the list is a common and effective pattern.

Best Practices for Error Handling and Connection Resiliency

Streaming connections are more prone to interruptions than standard REST calls. A brief flicker in the user's internet connection can break the ReadableStream.

1. Graceful Degradation: If the stream fails, provide a "Retry" button that picks up from the last successful state if possible, or restarts the prompt.
2. Abort Controller: Always implement an AbortController. If a user navigates away from the page or starts a new prompt while the current one is streaming, you must cancel the previous fetch request to save on API costs and prevent state collisions.
3. Visual Cues: Use a blinking cursor or a subtle pulse effect at the end of the streaming text. This signals to the user that the model is still "thinking" and the response isn't finished yet.
4. Token Limits: On the frontend, keep an eye on the length of the data. While rare, runaway loops in LLM generation can lead to massive state objects that crash the browser tab.

Conclusion

Streaming LLM responses is no longer a "nice-to-have" feature; it is the industry standard. By moving away from static JSON responses and embracing the ReadableStream API, you provide your users with a fast, interactive, and modern interface. Whether you are building an AI coding assistant in Bangalore or a customer support bot for a global audience, these implementation patterns will ensure your React application remains performant and user-centric.

The AI landscape is moving incredibly fast, and staying ahead requires more than just reading documentation—it requires a community.

Join the AIxTribe community today. Connect with top AI engineers, access exclusive workshops on advanced LLM patterns, and help us build the future of AI in India. Let’s innovate together!