Streaming AI Replies: Server-Sent Events vs WebSockets for Chatbots

When you watch a chatbot answer, the words appear one after another, not all at once after a long pause. That streaming effect is more than a nicety. Language models produce text token by token, and a full answer can take several seconds; showing words as they are generated makes the wait feel short and lets the visitor start reading almost immediately. Under the hood, streaming needs a connection that stays open while the server sends pieces of the reply. Two technologies dominate: Server-Sent Events (SSE) and WebSockets. Which one you use affects hosting compatibility, complexity, reliability behind proxies and the cost of running the chat. This guide explains both in plain language, compares them for AI chat, describes how streaming interacts with serverless hosting, error handling and reconnection, and explains why many chat widgets, including Crescelo's, use SSE. It is written for developers and for owners who want to understand what their vendor chose and why.

Why streaming matters

Perceived speed matters more than total time. A reply that starts instantly and takes six seconds to finish feels faster than one that arrives complete after four.

Server-Sent Events (SSE)

SSE is a simple standard: the browser opens a normal HTTP request, the server keeps it open and sends text messages down it as they are ready. It is one-way, from server to browser, which suits a reply being generated. The visitor's message is sent with an ordinary request, and the reply comes back as an event stream.

• Simple: plain HTTP; no special protocol upgrade.
• Works through most proxies and firewalls since it looks like a long response.
• Automatic reconnection in the browser's built-in support, with a way to resume.
• Easy to implement on most server frameworks.
• Fits serverless and edge hosting that support streaming responses.

Limits of SSE

• One direction only: the client cannot push events on the same connection.
• Browsers limit the number of simultaneous connections per domain on older protocols.
• Some proxies buffer responses unless configured not to.
• Text-based, so binary data needs encoding.

WebSockets

A WebSocket upgrades an HTTP connection into a two-way channel. Both sides can send messages at any time with low overhead. It is ideal for real-time apps such as multiplayer games, collaborative editing and live dashboards.

• Two-way: client and server both send at will.
• Low latency for frequent, small messages.
• Binary support.
• Suitable for presence, typing indicators and live agent chat.

Limits of WebSockets

• Needs a long-lived connection to a server that holds state, which is awkward on serverless platforms.
• Some proxies, corporate networks and older infrastructure interfere with upgrades.
• You must build reconnection, heartbeats and message ordering yourself.
• More moving parts to scale and monitor.

Side by side for an AI chatbot

For the core job of streaming a model's answer to a visitor, SSE does everything needed with less machinery.

Streaming and serverless hosting

Many small businesses' tools run on serverless platforms where each request is a short-lived function. Streaming responses are supported by modern platforms, but functions have time limits and cold starts, and some intermediaries buffer output. WebSockets require a persistent connection that most function platforms do not hold open. That is a major reason SSE is favoured on such hosting. If you need two-way real-time features, they are usually handled by a dedicated real-time service.

How a streamed reply flows

1. The visitor sends a question to the server.
2. The server retrieves relevant passages from the knowledge base.
3. It calls the language model with streaming turned on.
4. As each token or small group of tokens arrives, the server forwards it to the browser as an event.
5. The widget appends the text as events arrive.
6. At the end, the server sends a final event with metadata such as sources, and closes the stream.

Crescelo's chat endpoint streams replies as Server-Sent Events: token events as the text is generated, followed by source and completion events.

Handling errors and cut-offs

• Connection drops mid-reply: the client should show what it has, mark the reply incomplete and offer a retry.
• Model stops early: a robust server detects a reply that ends mid-sentence and asks the model to continue, so the visitor never sees half a sentence.
• Provider error: fall back to another provider or a friendly message; never a raw error.
• Timeouts: set sensible limits and send keep-alive messages so proxies do not close idle streams.
• Duplicate sends: protect against retries creating two replies.

Crescelo's server continues a reply that stops unfinished and closes cleanly if it cannot. See how to stop a chatbot from making things up for the accuracy side.

Behind proxies and CDNs

Streaming can be broken by intermediaries that buffer responses. Configure servers and proxies to disable buffering for stream endpoints, set appropriate headers (such as no-cache and an event-stream content type) and avoid compression that waits for the full response. Test through the same route your visitors use, including your CDN, not only on your laptop.

Accessibility and widget behaviour

• Announce new text to screen readers without repeating the whole reply on every token; batch updates.
• Respect reduced-motion preferences: avoid typing animations that add delay.
• Keep scroll behaviour predictable while text streams.
• Show a clear indicator when the reply is complete.

See chatbot accessibility and WCAG.

When to choose WebSockets

• You need live, two-way chat with human agents, typing indicators and presence.
• You have a persistent server and the operational capacity for it.
• Message frequency is high in both directions.
• You need binary data.

Many products combine approaches: SSE for AI streaming and a real-time service for live agent features.

A simple decision guide

Frequently asked questions

See the website widget feature, read performance of embedded chat widgets and how a chat widget is built, and try streaming in the playground.

Further reading

• MDN: Server-sent events: How servers stream updates to a browser.
• MDN: WebSockets API: Two-way real-time connections in the browser.