Confirmation Bias

011 min

Confirmation Bias at a Glance

• What it is — favoring evidence that fits a belief you already hold, and discounting evidence that doesn't.
• Origin — Peter Wason's 1960 rule-discovery experiment, the 2-4-6 task.
• Where it bites — a PM who treats praise for a launch as proof and complaints as noise.
• Guard against it — write the strongest opposing argument before the decision is final.

021 min

Where Confirmation Bias Shows Up

A product manager ships a redesigned onboarding flow, confident it will raise signups. Reviewing the first week of user interviews, they take detailed notes on every comment that praises the new flow and skim past the ones that don't, telling themself the people who struggled are edge cases. When a teammate points out that three of five interviewees couldn't find the "skip" button, the PM asks for a bigger sample before believing that. They do not ask for a bigger sample before believing the praise.

The same asymmetry shows up again a month later, after the launch, when signups turn out flat. The PM reviews the same interview notes and now reads the earlier praise as people being polite, and the earlier complaints as the real signal — the explanation changes to fit the new outcome, not to fit what the interviews actually said.

The pattern generalizes past this one launch: whoever holds a belief tends to ask harder questions of evidence against it than of evidence that agrees with it, and updates less than the new evidence would justify either way.

032 min

Why Confirmation Bias Happens

The clearest account of why confirmation bias happens comes from how people naturally test an idea. Raymond Nickerson's 1998 review of the research literature describes an experiment in which people were shown a triple of numbers, such as 2-4-6, and asked to discover the rule that produced it by proposing further triples and getting a yes-or-no answer for each one. Most people guessed a rule such as "even numbers increasing by two," then tested that guess by proposing only more triples that fit it, like 8-10-12 and 20-22-24. Every test came back "yes," which appeared to confirm the guess, even though it ruled nothing out: the actual rule was often broader, such as "any three ascending numbers," and only a test that could come back "no" could have shown the guess was too narrow.

Researchers call this a positive test strategy: checking a hypothesis by looking for cases that would confirm it, instead of cases that would rule it out. It is not laziness or a failure to understand logic. A test that comes back positive does support the hypothesis to some degree — the error is stopping there, and never running the test that could have said no.

A second, motivational explanation works alongside the strategy one. People find it easier to believe a claim they would like to be true than one they would prefer false, a pattern Nickerson links to a general preference for pleasant beliefs over unpleasant ones. This overlaps with cognitive dissonance: rejecting evidence that a recent decision was wrong is one way to avoid the discomfort of holding two conflicting beliefs about that decision at once. Nickerson's review is explicit, though, that confirmation bias also shows up when nothing is at stake — when someone tests an idea they have no reason to prefer either way. Both explanations are supported by separate lines of evidence; how much each contributes in any one case is a question the field has not settled.

041 min

Where Confirmation Bias Comes From

The 2-4-6 task described above is Peter Wason's own experiment, published in 1960 in the Quarterly Journal of Experimental Psychology under the title "On the Failure to Eliminate Hypotheses in a Conceptual Task." Nickerson's 1998 review summarizes the result: most participants tested their guessed rule only with triples that were consistent with it, and so never discovered that a broader rule — one their own test triples could not distinguish from their guess — was the actual answer.

The term "confirmation bias" itself is younger than the experiment. It became the field's standard name for this pattern through Nickerson's 1998 review, which surveyed the phenomenon across memory, perception, hypothesis testing, and everyday reasoning, and gave it the definition still in use today.

051 min

Individual Effects

For one person, confirmation bias shows up as a specific decision that goes wrong in a specific direction: an estimate that never moves from the number a plan started with, a hiring call that keeps looking better the longer an interviewer has already vouched for the candidate, a feature bet that survives one more release than the data justified because every dashboard check lands on the one metric that is still climbing.

The direction of the error tracks whatever was already believed, not the truth: an optimistic forecaster's confirmation bias makes the forecast more optimistic, and a pessimist's makes it more pessimistic. The bias does not create the direction of a decision on its own; it makes whatever direction was already there more extreme.

061 min

Systemic Effects

Confirmation bias compounds once it operates inside a process instead of inside one person's head. The clearest documented case is the U.S. Senate Select Committee on Intelligence's 2004 report on the run-up to the Iraq war. The committee concluded that "intelligence analysts, in many cases, based their analysis more on their expectations than on an objective evaluation of the information in the intelligence reporting," and that evidence contradicting the assumption Iraq had active weapons programs "was often ignored."

What made the bias systemic rather than individual was the missing check: the report found that formal mechanisms built for exactly this problem — devil's-advocate review, red teams, alternative analysis — existed inside the intelligence community but "were not utilized" on this assessment. A single analyst's confirmation bias is a bounded error a colleague can catch by asking a different question. An organization's confirmation bias is what happens when nobody in the process is assigned to ask it.

072 min

Examples

082 min

How Confirmation Bias Shows Up in Product, Design, and AI

In design and product work, confirmation bias usually enters through user research. A team that already believes a feature is the right bet reads five interview transcripts and remembers the two comments that praised it; the three lukewarm ones get filed as "not the target user." This is the accidental version — nobody set out to mislead anyone, and the researcher believed they read the interviews fairly.

The deliberate version is the dark pattern: a company frames its own usage numbers or customer testimonials to answer the question it already wants answered, then presents that framing to a board or to users as if it were neutral evidence, knowing which comparison will look most favorable. The difference between the two versions is intent, not effect — both end with evidence that only ever seems to confirm what someone already wanted to believe.

In AI and data work, confirmation bias shows up in how a team reads an evaluation. When a new model version scores worse than expected on a benchmark, the postmortem often finds a reason the benchmark doesn't count — the test set is stale, the metric is noisy, the failures are edge cases. The same scrutiny rarely gets applied when the new version scores better than expected. A benchmark result means something only if the team would have accepted a result that went the other way, and applied the same checks to it either way.

091 min

How to Guard Against Confirmation Bias

Awareness by itself does not reliably fix confirmation bias. Nickerson's review treats simply knowing the bias exists as a small help at best — enough to make someone "a little cautious about making up one's mind quickly," not enough on its own to change how someone weighs evidence. Two techniques that change the decision process itself have better evidence behind them.

The first is arguing the other side in writing before deciding. Nickerson cites experiments that trained people to generate reasons why the opposite of their working hypothesis might be true, not just reasons to support it, and reports that "the evidence provides reason for optimism that the approach can work." The output has to be a real counter-argument, not one token line acknowledging that one exists.

The second is assigning the disagreement to a role, rather than hoping any one person raises it. The 2004 Senate Intelligence Committee report on Iraq's weapons assessment traced part of the failure to exactly this gap: formal "red team" and devil's-advocate review existed as intelligence-community practice but "were not utilized" on that assessment. A structured dissent step works because it does not depend on whoever is most confident being willing to volunteer doubt about their own conclusion.

101 min

Common Misunderstandings

The most common misunderstanding is that confirmation bias means someone is being dishonest, or is unusually closed-minded. Nickerson's review is explicit that the effect shows up even when a person has no stake in the answer and no intention to mislead anyone — it operates on people who are trying, in good faith, to reach the right answer.

A second misunderstanding treats every case of discounting new evidence as confirmation bias. Sometimes discounting is the right response: if a single new anecdote contradicts a large body of well-tested evidence, giving it less weight is appropriate skepticism, not bias. Nickerson made this point about science itself, arguing that a degree of institutional conservatism "plays a stabilizing role in science and guards the field against uncritical acceptance" of claims that later don't hold up, citing the widely publicized but short-lived 1960s claims about magnetic monopoles and "polywater" as cases where the field's doubt turned out to be correct.

The difference is whether the standard for evidence changes depending on which conclusion it points toward. If it does, that's confirmation bias. If the standard stays the same either way, it's ordinary judgment.

111 min

Confirmation Bias vs. Nearby Concepts

Motivated reasoning is the closest neighbor of all. Some researchers treat it as the umbrella term and confirmation bias as one mechanism inside it, since motivated reasoning also covers biased interpretation of evidence that's already in hand, not only biased search for new evidence.

121 min

Related Concepts

• Anchoring — distorts a number, not a search for evidence.
• Availability Heuristic — judges likelihood by ease of recall.
• Cognitive Dissonance — the discomfort confirmation bias often protects against.
• Hindsight Bias — rewrites memory of a past prediction, not present search for evidence.
• Framing Effect — the deliberate version shows up as which framing gets shown.

?5 questions

Questions people ask

▶3 videos

Watch

• 

  Understanding Confirmation Bias

• 

  Confirmation bias | Extremely ziddi log | explained in hindi | Vikas Choudhary

  YourVikas

• 

  Confirmation Bias: Your Brain is So Judgmental | Heidi Grant Halvorson | Big Think

  Big Think

§6 sources

Sources