Name the assumption that would kill your product bet
Roadmap bets often list upside and effort while hiding the belief they depend on. Write the disconfirming observation first, then design the smallest test that could change your plan.
On this page 5 sections
- Separate observed facts from assumptions before estimating a bet.
- State what evidence would make you stop or change direction.
- Choose the smallest test that can produce that evidence.
A roadmap proposal can look rigorous and still rest on one sentence nobody has tested: users will change their behavior if we make this easier. The estimate may be detailed; the dependency is not. Before I argue for a bet, I want to name the observation that would make me stop believing in it.
Separate the evidence from the belief#
A product bet usually mixes three things: an observed problem, an explanation for it, and a proposed response. They deserve separate lines because each has a different confidence level.
Consider a hypothetical scheduling tool. A team sees that people abandon a booking form. That is an observation, assuming the event is measured correctly. “They abandon because the form asks for too much detail” is an explanation. “A shorter form will increase completed bookings without creating costly follow-up” is a bet.
Those statements can all be plausible while only the first is directly evidenced. If a roadmap discussion jumps from abandonment to shorter form, it quietly treats the explanation as fact and the outcome as guaranteed.
Write the chain explicitly:
- Observed: What did we see, and where did the evidence come from?
- Believed: What explanation are we using?
- Bet: What change do we expect to improve, for whom?
- Risk: What else might happen if the belief is wrong?
The purpose is not to make a proposal sound cautious. It is to locate the uncertainty that could invalidate the work. A team can then debate the actual belief rather than arguing about a feature as if it were the only possible response.
Write the disconfirming observation#
“Validate the idea” is too vague to guide a test. Ask instead: what could we observe that would make us change the plan?
For the shorter booking form, one disconfirming observation might be that people who abandon do so before reaching the long section. Another might be that users complete a short version but then require enough follow-up that the total task becomes harder. These observations point in different directions: the first challenges the diagnosis; the second challenges the proposed trade-off.
A useful statement has four parts:
If we observe [specific evidence] in [defined context], we will [change, pause or stop] because it weakens [assumption].
For example: “If people who leave the current form mostly exit before seeing the detailed questions, we will not build a shorter-form project around those questions as the primary cause.” That does not prove what the cause is. It prevents the team from treating one untested explanation as settled.
Make the threshold concrete enough that two people would interpret the result similarly. “Users dislike it” is not a threshold. “In five moderated sessions, at least four participants reach the same point of confusion without prompting” is more observable, though it is still a small qualitative signal, not a population estimate. The method and the claim must stay proportional to each other.
Test the uncertainty, not the solution polish#
When the risky assumption concerns the cause of a problem, a polished prototype may answer the wrong question. It can show that someone can use the proposed interface while telling you little about why the existing behavior occurs.
Start with the least expensive test that can distinguish between plausible explanations. Depending on the question, that could be:
- Inspecting a small, clearly defined sample of session replays or support cases, if they are available and appropriate to use.
- Asking users to complete the current task while narrating what they expect at the point of hesitation.
- Showing a low-fidelity alternative only after understanding the current behavior.
- Running a manual concierge version of the proposed change, with a clear record of what was done by a person.
A small sample is useful for finding a mechanism or a surprising failure. It does not establish prevalence. If the decision depends on how common a behavior is, plan a measurement that can answer that question rather than stretching a handful of interviews into a percentage.
Write down the test before running it: target participant or case, task, evidence to capture, stopping condition, and the decision that follows each plausible result. If every result leads to the same roadmap item, the test is not informing the decision. Either the bet is already committed for another reason, or the experiment needs a different question.
Keep the cost of being wrong in view#
Not every uncertain belief deserves an experiment. The cost of delay and the cost of a wrong bet both matter. A reversible copy change may justify a quick test; a workflow that can exclude eligible users or create substantial manual work deserves stronger evidence and a safer rollout.
Use a simple two-axis check:
| If the assumption is wrong… | Cost to reverse is low | Cost to reverse is high |
|---|---|---|
| Impact is limited | Ship narrowly and observe | Test the risky edge before expanding |
| Impact is substantial | Set a visible guardrail and rollback | Pause for stronger evidence or redesign |
This is a conversation aid, not a scoring formula. “High impact” should refer to a concrete consequence for users, operations, or the business. “Easy to reverse” should include hidden costs such as migration, retraining, or trust lost after a bad experience.
Also name the downside you will monitor. If shortening the form reduces effort but increases incomplete submissions, the primary success signal cannot be the only metric on the launch readout. A guardrail makes the trade-off visible before the team starts celebrating the number that moved.
Turn the result into a decision#
A test should end in a decision, not a slide labeled “learnings.” Before starting, write the possible outcomes and the action attached to each:
- The explanation holds up: proceed to a bounded test of the proposed change.
- A different cause is more plausible: revise the problem framing and test that cause.
- Evidence is mixed or too weak: reduce the scope, gather a more discriminating signal, or defer the bet.
- The downside appears: stop or add a control before expanding.
Keep the record short: the original assumption, evidence collected, limitation, decision, and owner of the next step. If the result changes the roadmap, preserve the reason alongside the change. That helps the team avoid repeating the same debate later, and gives future reviewers a fair account of what was knowable at the time.
Before you commit a roadmap bet, ask: what specific observation would make us change our mind, what is the smallest credible way to look for it, and what will we do if we find it?