Cruxly › AI engineering › AI-Native Interfaces

Streaming Interfaces That Show Their Work

SSE plumbing, confidence surfacing, and a stream that fails halfway

Prompt: design the frontend for an AI feature that streams responses as they generate, shows the user what the system is doing (retrieving, reasoning, answering) and how confident it is, and handles a stream that drops or errors partway through without leaving the user staring at a frozen half-answer.

40 minutes first. The streaming plumbing is a third of the work; the trust- surfacing and the failure handling are the other two-thirds.

Streaming plumbing: the transport and the partial-response problem

Server-sent events (SSE) or a streaming SDK, not polling, and not waiting for the complete response. The transport streams tokens from the model through your backend to the client as they generate. The backend's role here is not to buffer the whole response and forward it — that reintroduces exactly the latency streaming exists to hide — but to pass tokens through as they arrive while still applying any necessary processing (safety filtering, formatting) incrementally.

Handle the partial response as a first-class state, not an edge case. At any moment the client holds an incomplete response, and the UI has to render that incomplete state coherently — a half-finished sentence should look like it's still being written (a cursor, a subtle animation), not like a finished answer that happens to end mid-thought. The distinction between "still streaming" and "done" must be visually unambiguous, because a user acting on a half-streamed answer thinking it was complete is a real error the interface can prevent.

Backpressure and cancellation both matter on the client side. If the user navigates away or asks a new question mid-stream, the old stream needs to be cancelled cleanly — both to stop rendering it and to signal the backend to stop generating (and stop paying for) tokens nobody will read. A streaming interface that can't cancel is one that keeps spending money and rendering into the void after the user has moved on.

Showing the work: process visibility as the trust mechanism

Surface the phases of the system's work as distinct, visible states: retrieving sources, reasoning, generating the answer. This is not decoration — it's what lets a user understand that an answer is grounded in retrieved sources (and see what those sources were) versus generated from the model's own parametric knowledge, which is a trust-relevant distinction they cannot make from the answer text alone.

Show retrieved sources before or alongside the answer, not buried in a footnote. If the answer is based on specific documents, surfacing those documents prominently lets the user verify the answer against its actual basis — turning a "trust me" interaction into a "here's my evidence, check it" one. This is the single highest-leverage trust feature in a RAG-backed interface, and it's often relegated to a collapsed section where users never see it.

Distinguish "I found relevant sources and this answer is grounded in them" from "I found nothing relevant and I'm answering from general knowledge" explicitly in the UI. These are radically different trust situations for the user, and an interface that renders both identically is hiding exactly the information the user most needs to calibrate how much to trust a given response.

Be careful what "showing the reasoning" promises. A model's displayed chain of thought isn't guaranteed to be the process that actually produced the answer — faithfulness research (2025) found stated reasoning frequently omits factors that demonstrably influenced the output — and frontier APIs often expose only a summary of the model's thinking, not the raw trace. So build trust on the cited evidence (sources the user can open and check), and present any reasoning as the model's explanation, not as proof.

Tell users they're talking to an AI. Under the EU AI Act's Article 50 transparency duty (in force since 2 August 2026 — it was not delayed by the Digital Omnibus), a chat interface must make clear that the user is interacting with an AI system unless that's already obvious — a small, persistent label is enough, and it's good practice everywhere.

Confidence and severity: calibrated signal, not decoration or noise

Confidence indicators have to reflect a real signal, not a fabricated one. If the backend can produce a genuine confidence measure (retrieval score, model-reported uncertainty, an evaluation check), surface it honestly. If it cannot, do not invent a confidence number that looks precise but means nothing — a fake "94% confident" badge is worse than no badge, because it manufactures false trust. Surface confidence only where there's a real signal behind it.

Severity indicators (for a system that flags alerts, anomalies, or issues) need visual distinction that survives accessibility requirements. Color alone is insufficient — a high-severity flag distinguished only by red versus green is invisible to a colorblind user and to a screen reader. Severity needs a redundant signal (an icon, a text label, a shape) beyond color, which is both an accessibility requirement and, honestly, a clearer design for everyone.

Calibrate the noise floor: if everything is flagged, nothing is. An interface that surfaces a confidence or severity indicator on every single response trains users to ignore all of them. Reserve prominent indicators for the cases where they carry real information — a genuinely low- confidence answer, a genuinely high-severity alert — and let the common, routine, high-confidence case be visually calm. The signal is only useful if it's rare enough to notice.

Failure handling and accessibility: the two-thirds everyone skips

A stream that fails partway needs a defined recovery, not a frozen half-answer. When a stream drops at token 40 of an expected 200, the interface should: clearly mark the response as incomplete (not let it masquerade as a finished short answer), and offer a recovery.

The production answer is a resumable stream. Decouple generation from the client's connection: the server keeps generating into a buffer keyed by a response ID, each streamed chunk carries a sequence number (SSE's event id), and a client that reconnects sends Last-Event-ID to resume from where it dropped. The model never restarts, no tokens are paid for twice, and a phone that loses signal in a tunnel picks up mid-sentence. Without this, "retry" means regenerating from scratch — a different answer, paid for again. The failure state has to be as designed as the success state, because streaming failures are common enough (flaky mobile connections, backend timeouts) to be a routine part of the experience, not a rare exception.

Accessibility for streaming content requires announcing the completed response, not every token. A screen reader that announces each token as it streams is completely unusable. Use an ARIA live region configured to announce the response once it's complete (or at meaningful chunk boundaries), keep all streaming-control affordances (stop, retry) keyboard-navigable, and ensure confidence/severity indicators meet contrast requirements. The naive streaming implementation is inaccessible by default — accessibility here is deliberate work, tested with actual assistive technology, not an attribute added at the end.

Test the interface under realistic conditions: slow streams, dropped streams, very long responses, and rapid successive queries. A streaming UI that only works on a fast connection with short, successful responses is a demo. The real experience includes the mobile user on a train whose stream drops twice, the query that returns a 2,000-token response, and the impatient user who fires three questions in five seconds — and each of those has to degrade gracefully, which is exactly what separates a shipped AI interface from an impressive one.

How to say it in the interview

"Streaming doesn't make the model faster - it makes the wait feel like progress instead of a frozen screen, and it lets the user start reading and catch a wrong answer earlier. But a streamed interface has failure modes a request-response UI never had: a stream that dies at token 40 of 200 leaves a confident-looking half-answer, and that has to be handled as deliberately as the happy path."

Checklist

• Streamed tokens through the backend without buffering the whole response first
• Rendered the partial/streaming state as visually distinct from a completed answer
• Implemented clean client-side cancellation that also stops backend generation
• Surfaced retrieved sources prominently, not in a collapsed footnote
• Distinguished grounded-in-sources from answered-from-general-knowledge in the UI
• Surfaced confidence only where a real signal backs it, never a fabricated number
• Gave severity a redundant signal beyond color, and kept a calm default state
• Designed the stream-failure recovery as deliberately as the success path
• Built resumable streams: server-side generation keyed by ID, resumed with Last-Event-ID
• Grounded trust in cited evidence rather than displayed reasoning, and disclosed that the user is talking to an AI
• Made streaming content accessible by announcing the complete response, not every token

Related lessons

• Model Selection: Cost, Latency, and Capability Trade-offs
• Multi-Turn Conversation State and Context-Window Limits