011 min
Why recall matters
A team builds a model that flags fraudulent payments. After a month, someone reports that the model is 99% accurate. The same month, an analyst finds that the model caught only 3 of the 40 fraudulent payments. Both statements are true, and only one of them matters to the person whose money was stolen.
Recall is the number that answers the second statement. It asks one question: of all the cases that really were positive, how many did the model find? A PM who ships a screening feature, an engineer who tunes a spam filter, and a founder who buys a search tool all face a decision where a missed case costs more than a false alarm. Recall measures that cost directly.
The risk of ignoring it is silent failure. A missed case produces no alert, no error and no complaint. The fraud, the disease or the relevant document is simply absent from the output, and nobody sees what is not there. This article explains how recall is calculated, why it trades off against precision, and where the number can mislead you.
022 min
What recall actually measures
Recall compares two groups. The first is every case that is truly positive, such as every real spam email, every patient who has the disease, or every document that is relevant to a search. This group is fixed by reality and does not depend on the model. The second is the set of cases the model labelled positive. Recall is the part of the first group that the second group contains.
To count this, you sort every prediction into one of four cells. A true positive (TP) is a real positive that the model flagged. A false negative (FN) is a real positive that the model missed. A false positive (FP) is a real negative that the model flagged. A true negative (TN) is a real negative that the model left alone. The table of these four counts is called a confusion matrix.
Recall = TP / (TP + FN)
The denominator, TP + FN, is the number of actual positives. The scikit-learn documentation gives the same formula. The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. It also gives the plain reading: The recall is intuitively the ability of the classifier to find all the positive samples.
Recall always falls between 0 and 1, and it is often written as a percentage. A value of 1 means the model missed nothing. A value of 0 means it found none of the real positives.
Recall has many names, because several fields invented it separately. Fawcett (2006) calls it the true positive rate, also known as the hit rate. Powers (2011) adds that psychology and medicine call it sensitivity. Google's machine learning course says that the true positive rate (TPR), or the proportion of all actual positives that were classified correctly as positives, is also known as recall. It adds one more name: probability of detection. All of these are the same ratio.
Notice what the formula leaves out. False positives and true negatives do not appear in it. A model can raise its recall by flagging more cases, and no false alarm will lower the number. This is the property that makes recall useful, and it is also the reason recall can never be read alone.
032 min
A worked example with an email filter
Take an inbox of 1,000 emails, of which 40 are spam. The numbers below are invented for this calculation, so they illustrate the arithmetic and are not results from a real filter.
A spam filter flags 50 emails. Of those, 30 are spam and 20 are legitimate. It missed the other 10 spam emails. The four counts are TP = 30, FN = 10, FP = 20 and TN = 940.
- Recall = 30 / (30 + 10) = 30 / 40 = 0.75
- Precision = 30 / (30 + 20) = 30 / 50 = 0.60
- Accuracy = (30 + 940) / 1,000 = 0.97
Now the team makes the filter more aggressive. It flags 100 emails: 36 are spam and 64 are legitimate. The counts become TP = 36, FN = 4, FP = 64 and TN = 896.
| Filter setting | Emails flagged | Recall | Precision | Accuracy |
|---|---|---|---|---|
| Cautious | 50 | 0.75 | 0.60 | 0.970 |
| Aggressive | 100 | 0.90 | 0.36 | 0.932 |
| Flags nothing | 0 | 0.00 | undefined | 0.960 |
The aggressive filter finds 6 more spam emails. Recall rises from 0.75 to 0.90. The cost is 44 more legitimate emails in the spam folder, and precision falls from 0.60 to 0.36. Accuracy falls too, from 0.970 to 0.932, so a team that watched only accuracy would reject the change that found the most spam.
The last row shows the opposite problem. A filter that flags nothing has an accuracy of 0.96, because 960 of 1,000 emails are legitimate. Its recall is 0, since it finds no spam at all. Precision is undefined here, because it divides by the number of flagged emails, and that number is zero. Accuracy rewards this filter, and recall exposes it.
041 min
Where recall comes from
Recall came out of library and document-retrieval research, not machine learning. In 1955, Kent, Berry, Luehrs and Perry published "Machine literature searching VIII. Operational criteria for designing information retrieval systems" in the journal American Documentation. The paper described how to judge a system that retrieves documents for a query. Its two ratios, one measuring how many relevant documents were found and one measuring how many retrieved documents were relevant, became recall and precision. Manning, Raghavan and Schütze list this paper in the reference list of their textbook Introduction to Information Retrieval (2008).
The two measures then spread to other fields under other names. Medicine and psychology use sensitivity. Signal detection and ROC analysis use true positive rate and hit rate. Machine learning took all three. Powers (2011) traces these overlapping names and shows that they describe the same ratio.
Manning, Raghavan and Schütze define the two measures for a search that returns a set of documents. Precision is the fraction of retrieved documents that are relevant. Recall is the fraction of relevant documents that are retrieved. They add that an IR system can be thought of as a two-class classifier, which is why the same formulas apply to spam filters and medical tests.
052 min
Recall and the threshold
Most classifiers do not output a label directly. They output a score, and a threshold turns the score into a decision: everything at or above the threshold is flagged positive. The threshold is a setting that the team chooses, and recall depends on it.
Lower the threshold and the model flags more cases. Every extra flag can only add true positives or add false positives. It cannot remove a true positive. So recall never goes down when the threshold goes down. Manning, Raghavan and Schütze state this for search: recall is a non-decreasing function of the number of documents retrieved.
Precision usually moves the other way. Manning, Raghavan and Schütze write that in a good system, precision usually decreases as the number of documents retrieved is increased. The two measures therefore trade off against each other.
The extreme case shows the trade-off clearly. Manning, Raghavan and Schütze note that you can always get a recall of 1 (but very low precision) by retrieving all documents for all queries. A system that returns everything finds every relevant document, and it is useless.
This is why recall is rarely reported alone. Two common ways to report both are:
- A precision-recall curve, which plots precision against recall as the threshold changes.
- The F1 score, which combines the two into one number with the harmonic mean, a kind of average that stays low when either input is low.
The ROC curve plots recall (as the true positive rate) against the false positive rate, and AUC summarises that curve. Both depend on recall, but they answer a different question about the threshold.
062 min
Why accuracy hides low recall
Recall matters most when positives are rare. Many of the problems people care about are like this: fraud, disease, defects, security attacks and relevant documents in a large archive. Accuracy counts every correct decision equally, so on rare-positive data it is dominated by the easy negatives.
Manning, Raghavan and Schütze describe the extreme case for web-scale search. They write that in almost all circumstances, the data is extremely skewed: normally over 99.9% of the documents are in the nonrelevant category. They then spell out the consequence: a system tuned to maximize accuracy can appear to perform well by simply deeming all documents nonrelevant to all queries.
Google's machine learning course reaches the same conclusion for classifiers. It says that in an imbalanced dataset where the number of actual positives is very low, recall is a more meaningful metric than accuracy because it measures the ability of the model to correctly identify all positive instances.
The email example above shows the arithmetic. With 40 spam emails in 1,000, a filter that does nothing scores 0.96 accuracy and 0 recall. If the spam share drops to 1 in 1,000, doing nothing scores 0.999.
The remedy is not to drop accuracy. It is to read the four counts of the confusion matrix and pick the measure that matches the cost of each error. When a miss is expensive, accuracy is the wrong headline number, and recall belongs next to it.
072 min
How recall shows up in product and engineering work
Recall is a decision about which mistake is cheaper. The same model needs a different recall target in different products.
- Medical screening. A screening test is meant to find every person who may have a condition. Powers (2011) writes that in a medical context recall is regarded as primary, as the aim is to identify all real positive cases. A false alarm leads to a follow-up test. A miss may lead to no treatment.
- Legal and compliance search. Manning, Raghavan and Schütze write that professional searchers such as paralegals and intelligence analysts are very concerned with getting as high recall as possible, and will tolerate fairly low precision results in order to get it. A reviewer would rather read 500 extra documents than miss the one that matters.
- Web search. The same textbook says typical web surfers want every result on the first page to be relevant. That is a precision goal. A person searching the web has no interest in seeing every relevant document.
- Content moderation and security. A moderation team that must remove all copies of a banned image needs high recall. Missed copies stay online. A team that auto-bans accounts needs high precision, because a wrong ban harms a real user.
A PM can turn this into a requirement. Write the cost of a miss and the cost of a false alarm next to each other, and set the minimum recall before the team starts tuning. A sentence such as "the model must catch at least 95% of confirmed fraud, and analysts can review up to 200 alerts per day" gives the engineer both a recall target and a limit on false alarms.
Recall also needs enough positives to measure. If a test set holds 8 real positives, one miss moves recall from 1.00 to 0.875. Report the count of positives next to the recall value so a reader can judge how stable the number is.
082 min
Recall with more than two classes
Many models choose among several labels, such as the topic of a support ticket. Recall is then calculated for each class separately, as if each class were the positive one and all other classes were negatives. The scikit-learn documentation describes this approach. Support beyond binary targets is achieved by treating multiclass and multilabel data as a collection of binary problems, one for each label.
That gives one recall value per class. To report one number, the values have to be averaged, and the method matters.
| Averaging method | How it works | What it hides |
|---|---|---|
| Macro | Average the per-class recalls with equal weight | Nothing about class size, so a rare class counts as much as a common one |
| Micro | Pool all true positives and false negatives, then calculate once | Poor results on rare classes, because common classes dominate |
| Weighted | Average per-class recalls, weighted by class size | Poor results on rare classes, in a similar way to micro |
A support team with 900 billing tickets and 20 security tickets shows the difference. A model that finds every billing ticket and no security ticket has a macro recall of 0.5 across those two classes. Its micro recall is about 0.98, because most tickets are billing tickets. Both numbers are correct. The team must pick the one that matches the question, and for security tickets, the per-class recall is the number that matters.
092 min
When recall misleads
Recall has four known failure modes, and each has a tell.
- Flag everything. A system that labels every case positive has a recall of 1. The tell is a very low precision or a very high rate of flags. Never accept a recall figure without the matching precision.
- Unknown denominator. The formula needs the number of all real positives. In a fraud or search setting, nobody knows the true total, because the missed cases were never found. Test collections for search solve this by having people label the relevant documents ahead of time, but in live use recall is an estimate. The tell is a recall number from production data with no description of how the positives were counted.
- Ignoring chance and base rates. Powers (2011) argues that commonly used evaluation measures including recall, precision, F-measure and rand accuracy are biased and should not be used without clear understanding of the biases. His example is that a system that is worse in an objective sense can appear to perform better under any of these measures. Recall does not correct for how often the model would hit positives by chance.
- Treating all misses as equal. Recall counts a missed minor case and a missed critical case the same way. The tell is a good recall value alongside a complaint about one specific miss. If misses differ in cost, report recall for the critical segment on its own.
Recall also tells you nothing about how well the model handles negatives. The related measure for that is specificity, the share of real negatives that the model leaves alone. A full evaluation reports both, or uses a measure such as AUC that covers the whole range of thresholds.
102 min
Recall vs precision, sensitivity and specificity
The nearest neighbour is precision. The deciding fact is the denominator. Recall divides by the actual positives, so it asks how many real cases were found. Precision divides by the predicted positives, so it asks how many flags were right. A model with few flags has a high chance of high precision and low recall.
| Measure | Formula | Question it answers |
|---|---|---|
| Recall | TP / (TP + FN) | Of the real positives, how many were found? |
| Precision | TP / (TP + FP) | Of the flagged cases, how many were real? |
| Specificity | TN / (TN + FP) | Of the real negatives, how many were left alone? |
| False positive rate | FP / (FP + TN) | Of the real negatives, how many were flagged? |
Recall and sensitivity are the same number under two names. Specificity is the same idea applied to the negative class. The false positive rate is one minus specificity, and it is the horizontal axis of the ROC curve, where recall is the vertical axis.
The word also has a second meaning outside this article. In psychology, recall is the act of retrieving a memory without a cue, as in a free-recall test. That meaning has no formula and no relation to the ratio above. This entry covers only the retrieval and classification metric.
111 min
Recall questions people ask
Questions people ask about recall are answered below, each in its first sentence.
?8 questions
Questions people ask
What is recall in machine learning?
What is the difference between recall and precision?
What is a good recall score?
How do you calculate recall?
How do you increase recall?
Can recall be 100% and the model still be bad?
Is recall the same as sensitivity?
When should you use recall instead of accuracy?
§6 sources
Sources
Show all 6 sourcesHide sources
Manning, C. D., Raghavan, P. and Schütze, H. (2008). Introduction to Information Retrieval, section 8.3, Evaluation of unranked retrieval sets. Cambridge University Press.
scikit-learn developers. sklearn.metrics.recall_score documentation.
Google for Developers. Machine Learning Crash Course: Classification, accuracy, precision and recall.
Powers, D. M. W. (2011). Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies 2(1), 37-63.
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters 27(8), 861-874.
Kent, A., Berry, M. M., Luehrs, F. U. and Perry, J. W. (1955). Machine literature searching VIII. Operational criteria for designing information retrieval systems. American Documentation 6(2), 93-101.





