Design for the answers your model gets wrong
Most AI UX is designed around the demo, where the model is right. The product people trust is designed around the moments it isn't. Five patterns that help.
On this page 7 sections
- Assume a fixed error rate and design the experience of being wrong, not just of being right.
- Show sources and confidence where a wrong answer is costly; hide the machinery where it isn't.
- Make correction cheaper than starting over, and capture every correction as data.
- Let the cost of a mistake decide how much autonomy the feature gets.
Every AI feature demo shows the model being right. The prompt is clean, the answer is crisp, the room nods. Then it ships, and the first impression a lot of users get is the one time in ten it’s confidently wrong.
That moment decides whether they come back. Trust in AI products isn’t built by the right answers; users expect those. It’s built or lost in how the product behaves when it misses.
Start from the error rate, not the happy path#
Deterministic software trained us to treat errors as bugs to be eliminated. With LLMs, some error rate is a property of the system. You can push it down with better retrieval, prompts and evals, but you won’t take it to zero.
So the honest design brief is: this feature will be wrong some percentage of the time. What should that feel like?
A useful exercise before any mockups: take your eval set, pick the five worst failures, and storyboard what the user sees for each. If the answer is “a confident paragraph of nonsense with no way to tell,” you’ve found your real design problem.
Pattern 1: Show your work where it matters#
When a wrong answer is costly, let users check it in one glance:
- Inline citations that link to the exact passage, not just the document.
- Quoted source text next to extracted values, so a number can be verified without opening anything.
- “Based on” lines that state what the model looked at and, just as important, what it didn’t.
The goal isn’t transparency for its own sake. It’s making verification so cheap that users do it on the answers that matter, and skip it on the ones that don’t.
Pattern 2: Calibrate, don’t decorate#
Confidence scores like “87% confident” rarely help. Users don’t know what to do with them, and raw model probabilities are poorly calibrated anyway.
What works better is behaviour that changes with certainty:
| Situation | Better behaviour |
|---|---|
| Strong evidence in sources | Answer directly, with citation |
| Partial evidence | Answer the part you can, flag the gap explicitly |
| No evidence | Say so, and offer the next best action |
| Request outside scope | Decline plainly and route to a person or tool |
“I couldn’t find this in your documents” is a feature. Users forgive a product that knows its limits much faster than one that bluffs.
Pattern 3: Make correction cheaper than starting over#
When the output is 80% right, the worst UX is a regenerate button that throws away the good 80%. Better options:
- Edit in place, with the model respecting the edit in what follows.
- Targeted refinements (“shorter”, “more formal”, “use the Q3 numbers”) that change one thing.
- Accept / reject per item for list outputs, instead of all-or-nothing.
Every one of these corrections is also the best training and eval data you’ll ever get: a real user telling you precisely what was wrong. Log them, review them weekly, and turn the recurring ones into eval cases.
The edit a user makes to your AI’s output is the most honest product feedback you will ever collect.
Pattern 4: Match autonomy to the cost of a mistake#
Not every feature should act on its own. A simple way to decide:
- Low cost, easily reversed (drafting a reply, suggesting tags): let it act, show an undo.
- Medium cost (sending an email, updating a record): prepare the action, require a click.
- High cost or irreversible (moving money, deleting data, anything regulated): recommend only, a human decides, and the system keeps a record of who approved what.
Teams usually get this backwards: they ship maximum autonomy for the demo and then add guardrails after the first incident. Starting conservative and loosening as the eval numbers earn it is slower for a month and much faster for a year.
Pattern 5: Fail in the open#
When something goes wrong at the system level (a timeout, a retrieval miss, a tool error), say what happened in plain words and offer a path forward. A silent fallback to a generic answer is worse than an honest “I couldn’t reach your calendar just now. Try again, or paste the dates here.”
The checklist#
Before your next AI feature review, ask:
- What are the five worst failures in our eval set, and what does the user see for each?
- Can a user verify the high-stakes parts of an answer in under ten seconds?
- Does the product behave differently when it has weak evidence?
- Is fixing an 80%-right answer cheaper than regenerating it, and are we logging the fixes?
- Does the feature’s autonomy match the cost of its mistakes?
If you can answer all five, you’re designing the product users will actually live with, not the one in the demo.