Error Prevention

011 min

Error prevention and the 2018 Hawaii false alert

On 13 January 2018 at 8:07 a.m., a warning officer at the Hawaii Emergency Management Agency sent a message to phones across the state. It said a ballistic missile was inbound and that this was not a drill. No missile was coming. The agency had meant to run an internal exercise. The US Federal Communications Commission (FCC) later found that it took 38 minutes to get a correction out through the official alert channels.

The officer worked in alert software with a drop-down menu of message templates. Test and live missile alerts sat in one drop-down menu, side by side. After the officer chose a template, the software asked for a confirmation. The question was identical for a test and a real alert, and it never showed the officer the message that was about to go out.

Most headlines blamed one employee who pushed the wrong button. The interface deserves a closer look. It let one person send a state-wide emergency alert in a few clicks, with almost nothing between the exercise and the real thing. Designers have a name for the work of putting something there: error prevention. The rest of this article uses the Hawaii alert as its running case.

021 min

What the error prevention heuristic says

Error prevention comes from the usability heuristics that Jakob Nielsen and Rolf Molich first published in 1990, and that Nielsen refined in 1994. A heuristic evaluation is an expert review that checks an interface against a list like this one. (The same words also name a habit in software engineering, where it means catching defects in code. This article is about the people who use an interface.)

The heuristic gives a designer two options. Remove the condition that makes the error likely. Or detect the condition and ask for confirmation before the action goes through. Nielsen's position is that even a well-written error message comes second to a design that stops the problem from happening. A message arrives after the person has already acted, and fixing the problem takes them away from their task.

Prevention also changes who is at fault. The phrase "user error" assigns the fault to the person. The Nielsen Norman Group (NN/G), the research firm Nielsen co-founded, argues the opposite: when an interface makes an error easy, the interface is the cause. The safety researcher James Reason drew the same line in a 2000 paper. He called the first view the person approach, which blames forgetfulness and inattention. He called the second the system approach, which accepts that people are fallible and builds safeguards around them.

032 min

Slips and mistakes in error prevention

The Hawaii case also shows why designers split errors into two kinds. Don Norman, in The Design of Everyday Things, calls them slips and mistakes.

A slip happens when a person means to do one thing and does another, usually similar, thing. Typing "i" instead of "o" is a slip. Slips are common in tasks that a person knows well and does on autopilot. A mistake is different. The person forms the wrong goal, often because they misread what the system is doing, so even a perfect run of steps gives the wrong result.

The two need different fixes. Constraints, defaults and layout reduce slips. Clear labels, previews and a design that matches what people expect reduce mistakes.

Was the Hawaii alert a slip or a mistake?

Many commentators, NN/G among them, read the false alert as a slip: the officer picked the live template instead of the drill template. The FCC reached a different finding. The FCC found that the officer chose the live alert on purpose, because of a belief that Hawaii was under missile attack.

The recorded drill message had included the phrase this is not a drill, which the drill procedure did not call for. The officer says they did not hear the word "exercise" read out. The day-shift supervisor was also away from the watch center, because the two shift supervisors had misunderstood each other about who would run the drill.

So the root error was a mistake, not a slip. It still belongs in an error-prevention article, because catching a wrong belief is exactly what a good confirmation step is for. This one could not do that. The same question appeared for test and live alerts, and no message text was shown. Keeping both kinds of alert on one screen also makes a [mode error] more likely(/concept/mode-error), where a person acts as if the system is in one state when it is in another.

042 min

Preventing slips with constraints, defaults and distance

Slips are the easier half, because the fix is usually structure rather than persuasion.

Constrain what can be entered, where the rule is real. A flight-booking calendar can refuse a return date that falls before the departure date. The person cannot slip into an impossible trip.

Offer suggestions. Search suggestions mean fewer typed characters, so fewer typos. A reminder app that offers "tomorrow" and "in one hour" means nobody types a wrong date.

Choose the default with care. People tend to keep the value they are given, a pattern covered under the default effect, so the default should be the safe choice.

Forgive the format. Accept a phone number with or without spaces, then show it grouped as the person types. They can see a typo at a glance.

Keep consequential options away from routine ones

Slips also come from layout. NN/G described a Firefox spell-check menu that put "add to dictionary" directly next to the suggested spelling, so one stray click taught the browser the typo for good. Place consequential options far from routine ones, and make them look different in more than colour.

Engineers worked this out long before screens. During the Second World War, Boeing B-17 bombers kept crashing on landing even though the planes worked and the pilots were well trained. Alphonse Chapanis, the first psychologist at the Army Air Force's aero medical laboratory, saw that the landing gear lever and the flap lever were nearly identical and sat close together. In the rush of landing, pilots often retracted the gear when they meant to raise the flaps. He fixed a rubber wheel to the landing gear lever and a wedge to the flap lever, so the two could be told apart by touch. That kind of pilot error almost disappeared.

In 1947, Paul Fitts and Richard Jones analysed 460 reported errors in operating aircraft controls. They concluded that pilots of every level of skill made such errors, and that better design and placement of controls could cut accidents sharply.

When a constraint rejects real input

A constraint fails when its rule is wrong. In 2010, the programmer John Graham-Cumming complained that a system had rejected his surname as containing invalid characters. Patrick McKenzie, who wrote about the case, listed assumptions that programmers make about names, and called all of them wrong. A return date cannot come before a departure date, but no pattern can decide what counts as a real name. Constrain only where the rule holds for everyone.

051 min

Preventing mistakes with expectations and previews

Mistakes come from a gap between what the person expects and what the system does. Norman describes two gulfs. The gulf of execution is the difficulty of working out how to make a tool do what you want. The gulf of evaluation is the difficulty of telling what your action just did. Prevention means making both gulfs narrow.

1. 1

   Follow conventions.

   People have used thousands of other interfaces, and they expect the same controls to behave the same way. A design that breaks the convention makes a wrong goal more likely.

2. 2

   Show how a control works.

   A raised button looks pressable and a recessed field looks fillable. When a control gives no such cue, novices set it wrongly several times before they learn it.

3. 3

   Preview slow or wide changes.

   NN/G points to the iOS accessibility zoom setting, which needs a restart to apply, so the phone first shows what the screen will look like.

4. 4

   Keep the needed facts on screen.

   Under recognition rather than recall, a person should never have to remember a choice from an earlier step. A useful test is to imagine a phone call interrupting the person after every step, and ask whether the screen would let them resume.

The Hawaii software did the opposite. NN/G judged its template names cryptic and close to each other, so the person choosing had to work out which was which.

062 min

Confirmation dialogs and undo in error prevention

A confirmation dialog asks the person to check an action before it happens. It suits actions with serious consequences, such as deleting work, and it interrupts the task. For that reason, the question has to say what will happen. A dialog that names the file and shows the count of items helps. A bare "Yes / No" does not.

The Hawaii prompt failed on both counts. After the incident, the agency's own report recommended that the prompt name the exact action to be confirmed. Other emergency-alert systems, according to the FCC, host the live system apart from the test system, or use visual cues such as watermarks and colour coding to tell the two apart.

072 min

Layers of error prevention beyond the screen

No single check is enough. Reason compared a system's safeguards to slices of Swiss cheese. Every safeguard has weak points. Harm reaches people only when the weak points in several safeguards line up at once.

In Hawaii on the day of the false alert, the safeguards were few. One credentialed officer could send the alert alone, the confirmation prompt told them nothing, and the supervisor who might have checked the drill was away. After the incident, the agency began requiring two people to confirm any alert, whether a test or a real one.

081 min

Error prevention in accessibility standards

The Web Content Accessibility Guidelines (WCAG) 2.2, published by the W3C, turn error prevention into testable rules. Success Criterion 3.3.4, Error Prevention (Legal, Financial, Data), is level AA. It applies to pages where a person makes a legal commitment or a financial transaction, changes or deletes data they control, or submits test answers. Such a page must meet at least one condition: the submission can be reversed, the entered data is checked and the person can correct it, or a step lets the person review and confirm before finishing.

Success Criterion 3.3.6, Error Prevention (All), is level AAA. It extends the same three conditions to every page that asks the person to submit information.

The W3C explains who this protects. People with reading disabilities may transpose numbers and letters, and people with motor disabilities may hit keys by mistake. A design that checks, confirms or reverses helps them first, and everyone else as well.

091 min

Error prevention vs. nearby concepts

Error prevention sits next to several ideas that people mix up. The table names the axis that separates each one.

101 min

Applying error prevention to a design

Start with a short audit, then work through the actions it finds.

1. List the actions that destroy work, spend money, send something to other people, or cannot be reversed. A rare, high-cost error outranks a common, trivial one.
2. For each action, decide which kind of error is likely. A familiar, repeated task makes a slip likely. A new or unclear task makes a mistake likely.
3. Remove the error-prone condition first. Separate risky options from routine ones, constrain input where the rule is real, and make the safe choice the default.
4. If the action can be reversed, offer undo. If it cannot, use a specific confirmation that shows what will happen. For the rarest and most dangerous actions, ask for something unusual, such as typing a word.
5. For the highest stakes, add a second person or a separate environment, as Hawaii did after the false alert.
6. Watch real people use the design while they are hurried or interrupted. Slips appear under those conditions, and a calm test hides them.

If you have time for only one step, find the one action that would hurt most and make it harder to reach by accident.

?4 questions

Questions people ask

§15 sources

Sources