Planning Fallacy

Also called planning bias, optimistic time prediction. Also in Product, Business

Quick answer

The planning fallacy is the habit of predicting that your own task or project will take less time, cost less and run into fewer problems than similar ones did before. It happens because people imagine one smooth plan for the work in front of them and give little weight to how long comparable work really took.

~20 sec read

011 min

Where the planning fallacy shows up

Some years ago, the psychologist Daniel Kahneman joined a team of academics and teachers who were writing a school curriculum for a new subject in Israeli high schools. In time, the team began to talk about how long the job would take. Each member of the team wrote on a slip of paper how many months the project needed. The guesses ran from 18 to 30 months.

Then another member of the team asked the group's expert on curriculum design to think back. How long had similar teams taken, at the same stage? The expert said that about 40 percent of those teams never finished. Of the teams that did, he could name none that took less than seven years and none that took more than ten.

The team's own guesses were far below that range. The team finished the project eight years later.

That gap between the plan and the record is the planning fallacy at work. The expert had the numbers, and the team was made of people who study teaching for a living. The mistake did not come from a lack of knowledge. It came from how the forecast was made.

team guesses18 to 30 months7 to 10 years
Look at the two units. The team's guesses are counted in months, and the expert's range is counted in years.

021 min

Where the planning fallacy comes from

Kahneman and his colleague Amos Tversky gave the pattern the name planning fallacy in 1979. They described people who firmly believe that their own project will go to plan, while they know that most similar projects ran late.

The first direct test came in 1994. Roger Buehler, Dale Griffin and Michael Ross phoned 37 psychology students at the University of Waterloo in Canada who were writing an honours thesis, and asked when each one would hand it in. Their average answer was 33.9 days, and the average thesis took 55.5 days.

The researchers also asked each student for a worst-case date, assuming everything went as badly as it could. Fewer than half of the students finished by even that date.

Other people's projects

The students were not simply hopeful about everything. In a later study in the same paper, people who predicted how long someone else would take guessed too long, not too short. The error shows up when people forecast their own work.

032 min

Why the planning fallacy happens

Kahneman and Tversky's explanation is that a forecaster can look at a project in two ways. In the inside view, the forecaster studies the project's own details and unusual features and tries to predict what will happen. In the outside view, the forecaster sets those details aside and asks how similar projects turned out. Lovallo and Kahneman link the planning fallacy to the inside view: the forecaster studies the parts of one specific plan, not how similar plans turned out. The curriculum team used the inside view. Their expert, when challenged, used the outside view.

The inside view feels like careful work, because it uses everything the forecaster knows. Its weak point is that a plan lists the steps that go right. Delays come from events that are not on the list, such as a reviewer who is busy or a test that fails. Any one of those events is unlikely. Some of them are likely to happen in a long project.

Past delays do not fix this. Buehler and colleagues argue that people see their own earlier delays as caused by circumstances. If each delay was bad luck, the record seems not to apply to the next project.

What people think about while estimating

Buehler's team asked students to think aloud while they predicted when a school assignment would be done. Nearly all of them, 93.5 percent, described their plans for the assignment. In an earlier think-aloud study, 71 percent of what people said was about their plans, and only 3 percent was about what might go wrong.

Bad news gets discounted

Is it enough to ask people to think of problems first? Newby-Clark and colleagues tested this in 2000. They had people generate pessimistic scenarios before they predicted. The scenarios did not change the predictions or make them more accurate. People also rated the pessimistic scenarios as less plausible than the optimistic one.

041 min

The Sydney Opera House: the same gap at the scale of a city

The curriculum team lost years. Larger projects lose years and money, and the best-known case is a building. In 1957 the plan was to finish the building early in 1963 for 7 million dollars, but it opened in 1973 in a scaled-down form at 102 million. That is more than 14 times the first estimate. All the amounts are in Australian dollars.

The same pattern across transport projects

The Opera House is not a lone case. Bent Flyvbjerg studied cost forecasts for transport projects in 2006. Cost forecasts for rail projects were wrong by 44.7% on average, for bridges and tunnels by 33.8%, and for roads by 20.4%. Over the 70 years for which cost data exist, forecasts did not become more accurate.

Concrete roof shells of the Sydney Opera House partly covered in scaffolding, with cranes behind them, on a platform above the water.
Fig. 1The roof shells were still being built in the mid-1960s, after the date first planned for the finished building.My Grandfather · CC0 · Wikimedia Commons
Sydney Opera Housefirst estimate: 7 millionfinal cost: 102 million
The two bars share one scale. The first estimate is a small part of the final cost.

052 min

How the planning fallacy shows up in product, design and engineering

A transport project is far from a sprint board, but the pattern is not limited to it. Flyvbjerg notes that what he found in transport projects also applies to many other project types, including IT systems.

Picture a tech lead in sprint planning who is asked when a payment-provider migration will ship. They list the steps: write the adapter, change the checkout, test it, release it. Each step looks small, so the total comes to six weeks, and they say six weeks. The list has no line for the days when a reviewer is busy, for a test environment that breaks, or for a bug that only appears with real customers. A sales team then tells customers about the six-week date.

The same shape appears in other roles:

  • A designer promises a new component for the next sprint after listing the visual work, and leaves out accessibility checks and the edge cases that engineers raise.
  • A product manager commits a roadmap quarter by adding up the estimate for each feature. The sum assumes that no feature grows and that no team is pulled onto an urgent fix.
  • A data scientist plans a model launch around training and evaluation time, and leaves out the weeks spent finding and cleaning the data.

In all four cases the person is not lying. The estimate is an honest sum of the steps they thought of. Sometimes a short estimate is chosen on purpose, to win approval for a project. That is a different problem, and it comes up under common misunderstandings below.

062 min

How to guard against the planning fallacy

Knowing about the fallacy is not enough to avoid it. In Buehler's classroom survey, students said they had finished about two thirds of their past projects later than they expected, and their predictions for new projects were still too early. So the fix is a change in how the estimate is made, not a reminder to be careful.

Start from similar finished projects

The outside view gives a way to do this, and Flyvbjerg calls it reference class forecasting. It has three parts: pick a group of similar finished projects, look at how their outcomes were spread, and place the new project in that spread.

Go back to the tech lead. Say they first look up the last three migrations. They took nine, eleven and fourteen weeks (these numbers are made up for this example). The lead now tells sales that the work will probably take around eleven weeks and is unlikely to take under nine. If no past project matches, they use the closest ones and say that the range is wide.

A reference class can also show how often projects never finish: in the curriculum story, about 40 percent of similar teams gave up. Using one is still rare. Flyvbjerg reviewed several hundred forecasts and found none that was a reference class forecast. In his UK road work, the reference class was 172 finished road projects, and it gave a probability for each size of cost overrun.

Tie the estimate to your own record

In Buehler's fourth study, students were told to link each prediction to relevant past experience. The optimistic bias disappeared. Keep a simple log with the estimate, the actual result and the date. After a few projects it is your own reference class.

Decide when and where you will start

Sander Koole and Mascha van't Spijker studied students in the Netherlands in 2000. All of them had to write a report. Some also said exactly when and where they would write it. These students predicted earlier finishes, but even more of them finished, so the gap between prediction and result got smaller. The students with a plan also reported fewer interruptions while they worked.

Break the work into steps and give a range

List the steps and estimate each one, then give a range instead of one date. The list will not catch everything, but it shows how many places there are for a delay. If the sum of the steps and the outside view differ a lot, trust the outside view more, because it includes the delays the list left out.

072 min

Common misunderstandings about the planning fallacy

These methods work best when people know what the fallacy is and what it is not.

It is not procrastination

Procrastination is delaying a task you mean to do. The planning fallacy is a wrong forecast of how long the task will take, and it appears in people who start on time. The two can combine, because a late start leaves less room for the delays the forecast missed.

It is not only about time

Flyvbjerg gives the wider version: people underestimate the costs, completion times and risks of planned actions, and they overestimate the benefits.

Not every low estimate is a mistake

Some short estimates are chosen. Flyvbjerg describes forecasters and managers who deliberately underestimate costs and overestimate benefits so that their project gets approved and funded. He calls this strategic misrepresentation. The outside view helps when forecasters are honestly trying to see the future. When estimates are shaded for approval, he argues that it also needs rewards for accurate forecasts and penalties for inaccurate ones.

Expertise does not protect

People often expect experts to be immune. In a 1970 study by Kidd, professional engineers consistently underestimated when equipment overhauls at power stations would finish. The curriculum team was also a team of experts.

The evidence is mixed

The planning fallacy is well documented, but it is not a law. In 2012, Torleif Halkjelsvik and Magne Jørgensen reviewed studies of time predictions. Underestimation was reported more often than overestimation in studies from engineering and management, but not in studies from psychology. For a practitioner this means measuring instead of assuming. Your own log of estimates and results tells you which way your estimates lean.

Small and large tasks

Halkjelsvik and Jørgensen also question a common finding, that small tasks are overestimated and large tasks are underestimated. They say random error alone can produce part of that pattern. So a large task that ran late does not prove that large tasks are always underestimated.

081 min

Planning fallacy vs. nearby concepts

Several neighbours describe a part of the same experience. Each one is separated from the planning fallacy by what it explains.

  • Optimism bias is a general expectation that good things are more likely for you than for others. The planning fallacy is narrower: a forecast of the time, cost and risk of one specific plan.
  • Hofstadter's Law says that work takes longer than you expect, even when you allow for this law. It describes the result, and the planning fallacy explains one reason for it.
  • Parkinson's Law says that work grows to fill the time you give it. It describes work that takes the time allowed, while the planning fallacy is about allowing too little time.
  • Present bias is a pull toward rewards that come sooner. It explains why people start late, not why they guess a wrong duration.
  • Anchoring is the pull of the first number someone hears. A first date said in a meeting can anchor a team's estimate, so the two can work together.

?4 questions

Questions people ask

Do experts avoid the planning fallacy?

No. Professional engineers in a 1970 study by Kidd consistently underestimated when equipment overhauls at power stations would finish. In the curriculum story, a team of experts also guessed far too low.

If people usually meet deadlines, how can the planning fallacy be real?

A deadline is fixed by someone else, and a prediction is your own guess. In Buehler's survey, students finished about three quarters of their projects on the deadline day, yet about two thirds finished later than predicted.

How much time should I add to an estimate?

There is no fixed percentage. Base the number on how long your own comparable projects took, and add time where that record shows a gap. A log of past estimates and actual results makes this simple.

How do I estimate a project nobody on my team has done before?

Use the closest finished projects you can find, from your team or from others, and give a wide range instead of one date. Say plainly that the range is wide because the match is loose.

§6 sources

Sources

Show all 6 sources
  1. Buehler, R., Griffin, D. and Ross, M. (1994). Exploring the "planning fallacy": Why people underestimate their task completion times. Journal of Personality and Social Psychology 67(3), 366-381.

  2. Flyvbjerg, B. (2006). From Nobel Prize to project management: Getting risks right. Project Management Journal 37(3), 5-15.

  3. Newby-Clark, I. R., Ross, M., Buehler, R., Koehler, D. J. and Griffin, D. (2000). People focus on optimistic scenarios and disregard pessimistic scenarios while predicting task completion times. Journal of Experimental Psychology: Applied 6(3), 171-182.

  4. Halkjelsvik, T. and Jørgensen, M. (2012). From origami to software development: A review of studies on judgment-based predictions of performance time. Psychological Bulletin 138(2), 238-271.

  5. Koole, S. L. and van't Spijker, M. (2000). Overcoming the planning fallacy through willpower: Effects of implementation intentions on actual and predicted task-completion times. European Journal of Social Psychology 30(6), 873-888.

  6. Guinness World Records. Largest proportional increase in cost for a building project (Sydney Opera House).

Keep reading

More from Psychology

All of Psychology
All of Psychology

Missing a concept?

Tell us what to write next. We’ll email you when it’s live.

Several? Separate them with commas.

Only to tell you when it’s live.

How we use your email: privacy.