011 min
Why abductive reasoning begins with a surprise
In 1847, the Vienna General Hospital ran two maternity clinics side by side. In the First Clinic, doctors and medical students delivered the babies. In the Second Clinic, midwives did. Women in the First Clinic died of childbed fever, an infection after birth, at about three times the rate of women in the midwives' ward. Ignaz Semmelweis, a doctor in the First Clinic, could not say why.
He ruled out the obvious causes one at a time. Overcrowding could not be the reason, because the Second Clinic was always more crowded. The climate could not be the reason, because the two clinics stood close together. The gap was a surprise, since nothing he knew had predicted it.
Then a close colleague, Jakob Kolletschka, died after a student cut him with a scalpel during an autopsy. The examination of the body showed damage like that found in the women dying of childbed fever. Semmelweis proposed that the doctors and students carried particles from corpses on their hands from the autopsy room to the women they examined in the First Clinic. The student midwives in the Second Clinic never worked with corpses, which matched their much lower death rate.
Semmelweis had gone from a surprising fact to a cause that would make the fact expected. That step has a name.
021 min
How abductive reasoning works
That step is abduction. Semmelweis did not count thousands of similar cases and average them. He also did not start from a known law and work out what must follow. He started from one odd fact and asked what would explain it. The pattern has three parts.
A surprising fact is observed. If a certain explanation were true, the fact would be expected. So there is reason to suspect that the explanation is true.
The last part says reason to suspect. It does not say proof. A plain case shows why. You wake up and see a wet lawn. Rain would make a wet lawn expected, so rain is a good explanation. A sprinkler would do the same job, and so would morning dew. The wet lawn cannot choose between them, so you keep rain as the working answer until you look at the street, the neighbours' lawns or the sky.
Three things follow. Abduction reasons back from an effect to a possible cause. Its conclusion is plausible and not certain. And it adds a new idea, because the explanation was not contained in the evidence. Logicians say that reasoning of this kind can be withdrawn: a new fact can remove the conclusion, even though no earlier fact was wrong.
031 min
Where the term abductive reasoning comes from
The American philosopher Charles Sanders Peirce formulated abduction in the second half of the 19th century. He also called it retroduction. His interest was in where a scientific idea comes from before anyone tests it.
In 1903 Peirce wrote the pattern as three lines: a surprising fact is observed, the fact would be expected if an explanation were true, so the explanation deserves suspicion. That is the form shown above.
The word has drifted since. Philosophers use the word in two senses: the first is about producing hypotheses, and the second is about justifying them. Peirce meant the first. Most modern writing means the second, and calls it inference to the best explanation. A reader should check which sense an author has in mind.
041 min
Abductive reasoning vs deduction and induction
Abduction is easiest to place beside the two kinds of reasoning most people already know, deductive reasoning and inductive reasoning.
| Deduction | Induction | Abduction | |
|---|---|---|---|
| Starts from | A general rule | Many observed cases | One surprising observation |
| Moves to | A specific result | A general rule | A cause that would explain it |
| Conclusion | Certain, if the premises are true | Probable | Plausible |
| Lawn example | It rained, so the lawn is wet | Lawns checked after rain were wet, so rain wets lawns | The lawn is wet, so it probably rained |
The difference that matters in practice is how each can fail. A deduction with true premises cannot fail. An induction fails when the next case differs from the earlier ones. An abduction fails when the real cause was not the one chosen, even though every observation was correct. That is why an abduction needs a test before anyone acts on it.
051 min
How abductive reasoning picks one explanation
Usually several explanations fit the same surprise, so the reasoner has to choose. Three tests are common.
- Fewer assumptions. The explanation that needs fewer unproven things is preferred. Occam's razor is the usual name for this rule.
- How likely it was beforehand. An explanation can fit every symptom and still be extremely rare. A common cause that explains most of the evidence is usually the better first choice.
- Reach. A good explanation also predicts things the observation did not mention, and those predictions can be checked.
How Semmelweis ruled out rival explanations
Semmelweis used all three tests. He dropped overcrowding because the Second Clinic was busier and had fewer deaths. He dropped climate because the clinics shared one location. He changed the position in which women gave birth, and he tried the idea that a priest bringing last rites frightened women into fever. Each rival that failed made the explanation about corpse particles the better choice, and it was the only one that also predicted a clear action: remove the particles.
062 min
Testing a conclusion from abductive reasoning
A conclusion that is only plausible has to be tested, and the explanation itself says how. If particles from corpses carry the fever, then removing the particles should lower the deaths. Drawing a prediction from an idea in this way is deduction. Semmelweis told staff to wash their hands in a chlorinated lime solution between autopsy work and the examination of patients. Semmelweis showed that making staff in obstetric clinics disinfect their hands drastically reduced infection.
The three kinds of reasoning work as a cycle. Abduction supplies the idea. Deduction draws a prediction from it. Observation checks the prediction, and that part is induction. If the prediction fails, the reasoner goes back to the surprising fact and looks for a new explanation. The result is a hypothesis that has survived one test, which is stronger than a hypothesis that has only been proposed.
A working explanation before its mechanism is known
Semmelweis could act before he understood the cause. The germ theory of disease had not yet been accepted in Vienna, so he had no way to say what the particles were. His explanation was still useful, because it predicted an action and the action worked. An abductive conclusion can guide work while its details remain open, as long as it is labelled as a working answer.
Why a sound conclusion can still be rejected
Semmelweis had no accepted theory behind his result. Some doctors were also offended by the suggestion that they should wash their hands, and they mocked him for it. An explanation can be right and still fail to persuade when it has no accepted mechanism and implies that colleagues were at fault.
The iridium layer
Geology gives a second case, far from a hospital. In 1980, a team led by Luis and Walter Alvarez found that sediment layers at the end of the Cretaceous period, all over the world, held iridium hundreds of times above the normal amount. Their explanation was the impact of a giant asteroid that ended the age of the dinosaurs. This became known as the Alvarez hypothesis.
An explanation that waits for its test
Unlike Semmelweis, the Alvarez team could not run the test the next week. An impact of that size should leave a crater, and finding it took years of further work. By 2010 a panel of international scientists had endorsed the Chicxulub impact as the cause of the extinction. That is thirty years between the abduction and a widely accepted confirmation. Some abductions can only be tested by waiting for evidence that nobody has yet found.
072 min
Where abductive reasoning shows up at work
The same move appears wherever people reason back from a symptom to a cause.
- Medicine. A doctor takes the symptoms and the history, then names the condition that would explain them best. The same method is used in legal judgments, where a jury weighs which account of events fits the evidence.
- Detective stories. Sherlock Holmes calls his method deduction, but it runs from clues to the most likely cause, which makes it abduction.
- Incident response. Say an on-call engineer sees the error rate climb a few minutes after a release. The release is one candidate. A database that filled up at the same time is another. The engineer lists both and checks which one also explains the other symptoms. This is the first move of root cause analysis.
- Qualitative research. A researcher coding interviews meets an answer that existing theory does not predict. They propose an explanation, return to the data to test it and revise. Researchers who report such a finding as if it had been predicted in advance are doing what is called HARKing, short for hypothesizing after the results are known.
Machines are tested on the same skill. In 2019, Bhagavatula and colleagues tested language models on abductive reasoning, and the best model reached 68.9% accuracy where humans reached 91.4%. The test asked a model to pick the more likely explanation for what happened between two events.
Design abduction
Designers meet a variation. They are not handed a surprising fact. They are handed a value to reach, such as an easier task for a user, and must find both a working principle and an object that would deliver it. Kees Dorst, as described by Kroll and Koskela, sees novice designers generating proposals for both the working principle and the object almost at random, then looking for a pair that reaches the wanted value. The lesson for a team is to settle how the thing will work before deciding what it looks like.
082 min
When abductive reasoning goes wrong
Because the answer is only plausible, its failures follow a few known patterns.
The true explanation was never on the list
Philosopher Bas van Fraassen argued in 1989 that choosing the best explanation only works if the true explanation is among the candidates already thought of. If it is not, the reasoner picks the best of a weak set and feels confident. A useful habit is to ask what would have to be true for none of the candidates to be right.
More than one cause at once
In medicine, Hickam's dictum warns that a patient's symptoms can come from several diseases at once. A search for the single best explanation will miss the second cause. The tell is a treatment that works partly: the main complaint improves and a smaller one remains.
Stopping at the first fit
Graber and colleagues reported in 2005 that premature closure, which means failing to keep considering alternatives after a first diagnosis, was the most common cause among the diagnostic errors they studied. Once an explanation is chosen, confirmation bias makes later evidence look like support for it. The tell is a conclusion reached in the first few minutes that nobody has tried to disprove.
Telling the story after the fact
An explanation built from the data cannot also be a test of the data. Presenting it as a prediction made in advance makes the evidence look stronger than it is. The honest label is exploratory, not confirmed.
091 min
How to use abductive reasoning on a surprising result
Say a product manager sees sign-ups on one browser fall by a third the week after a release. The numbers here are made up for the example. The manager can use abductive reasoning in five steps.
- State the surprise in one sentence with its numbers: sign-ups on one browser fell by a third after the release.
- List at least three explanations that would make the drop expected: a bug in the release on that browser, a change in the ads that bring visitors, or a fault in how sign-ups are counted.
- For each explanation, write down one other thing it predicts. A bug predicts failed attempts in the error logs. An ad change predicts fewer visits. A counting fault predicts that new accounts still appear in the database.
- Run the cheapest check first. Reading a log takes minutes, and it can remove two candidates at once.
- Keep the survivor as a working answer. Say how sure you are, and name the result that would change your mind.
Treat the answer as a lead until a prediction has held. If the manager acts on the first plausible explanation, a fix for the wrong cause ships and the real one keeps running.
?4 questions
Questions people ask
Is abductive reasoning just guessing?
Who invented abductive reasoning?
How is abductive reasoning different from inference to the best explanation?
Can a conclusion from abductive reasoning be certain?
§9 sources
Sources
Show all 9 sourcesHide sources
"Abductive reasoning", Wikipedia (a summary of Peirce's account and later uses).
Igor Douven, "Abduction", Stanford Encyclopedia of Philosophy (covers the modern sense and van Fraassen's objection).
"Ignaz Semmelweis", Wikipedia (a biography covering the Vienna clinics, 1847).
"Alvarez hypothesis", Wikipedia (a summary of the 1980 iridium finding and the Chicxulub impact).
"Hickam's dictum", Wikipedia (a short entry on the medical principle).
Mark L. Graber, Nancy Franklin and Ruthanna Gordon, "Diagnostic error in internal medicine", Archives of Internal Medicine, 2005 (abstract on the RTI publication page).
Ehud Kroll and Lauri Koskela, "On Abduction in Design", Design Computing and Cognition '14, Springer, 2015.
Chandra Bhagavatula and colleagues, "Abductive Commonsense Reasoning", arXiv:1908.05739, 2019.
CASRAI, "Inductive vs. Deductive vs. Abductive Reasoning" (a guide for researchers, covering HARKing).
