Every AI feature will give a wrong answer at some point. Not because the team did a bad job, but because that is how probabilistic systems work. The difference between a feature people trust and one they abandon is rarely the model. It is what the product does when the model is wrong.
A better prompt is not a strategy
When an AI feature misbehaves, the first reflex is to tune the prompt. Add a rule, add an example, add “IMPORTANT” in capitals. It helps for a while. Then the next edge case shows up, and the prompt turns into a legal document nobody dares to touch.
Prompts matter, but they are the last line of defence, not the first. The first line is the design of the feature around the model.
Four questions I ask before building
- What does a wrong answer cost? A clumsy product description costs a little. A wrong refund decision costs a lot. The cost decides how much checking you need.
- Can the user see and fix it? A draft that a person reviews before sending is a very different risk from an action the system takes on its own.
- Can we check the output with plain code? If the model returns an amount, compare it with the invoice. If it returns a product ID, look it up. Cheap checks catch expensive mistakes.
- Where does it go when it’s unsure? “I don’t know, let me connect you with a colleague” is a perfectly good answer. A confident guess is not.
Suggest first, automate later
My default is to launch an AI feature as a suggestion: the model proposes, a person decides. That gives you two things at once. Users stay in control, and every accept or reject is a label you can use to measure quality.
Once the numbers show the suggestions are right often enough for a specific type of case, you automate that type of case. Not everything at once. Trust is earned per use case, not per model.
Always keep the boring path
Whatever you build, keep a non-AI path that works. A search that falls back to keyword matching. A form that still works when the assistant is down. A queue a human can pick up. Providers have outages, rate limits and bad days, and your product shouldn’t have them too.
Designing for the wrong answer sounds pessimistic. In practice it is what lets you ship AI features with confidence, because you already know what happens on the bad day.
