Gulf of Evaluation

Quick answer

The gulf of evaluation is the effort a person must spend to work out what state a system is in and whether it did what they wanted. The gulf is narrow when the display shows the result in the person's own terms. It is wide when they must guess, calculate, or look somewhere else to find out.

~20 sec read

011 min

The gulf of evaluation in one Bluetooth switch

In 2018 Kathryn Whitenton of the Nielsen Norman Group bought a Bluetooth headset and could not connect it to her computer. She spent over an hour on the problem. She re-read the manual, checked that both devices were on, read reviews to confirm the headset worked with her computer, and tested it on a second computer.

The answer came from a help page written by another hardware maker. It showed a screenshot of the Windows 10 Bluetooth switch in its On position. She then saw that the settings screen she had looked at for the past hour showed that Bluetooth was off.

The switch was visible, and next to it was a label that said Off. The label named the current state of the switch. A label beside a switch can also be read as the name of the position the switch will move to, and she had taken the screen to mean that Bluetooth was on. Seeing the switch was easy. Knowing what it said about the computer was the hard part.

That difficulty has a name: the gulf of evaluation. It is a problem of reading, not of skill. People who meet it repeat steps that already worked, as Whitenton did when she checked that both devices were on, or they give up and return the product.

Bluetooth is onOff
Compare the thought with the label on the screen. The person believes one thing, and the display states the opposite.

022 min

Where the gulf of evaluation comes from

The name comes from a 1985 paper by Edwin Hutchins, James Hollan and Donald Norman, who worked at the University of California, San Diego. It appeared in the journal Human-Computer Interaction under the title "Direct Manipulation Interfaces". The authors wanted to explain why some interfaces feel direct, as if the person handles the things the task is about and not a program between them and those things.

Their explanation was distance. An interface adds distance to the extent that a person's goals and knowledge differ from the level at which the system describes things. They called the distance on the way in the gulf of execution and the distance on the way back the gulf of evaluation. The more mental effort a person spends to operate and read a system, the less direct it feels.

The paper is a framework, not a measurement. It does not give a scale for how wide a particular gulf is. It gives designers a way to ask where a person's effort goes. That makes the gulf of evaluation a design principle in the sense of practitioner convention, not a research finding with a formula, as Fitts's Law is.

Norman took the idea to a wide audience in his 1988 book The Psychology of Everyday Things, later retitled The Design of Everyday Things. There he tied the two gulfs to a model of how a person carries out any action, which he called the seven stages of action.

Why sources give 1985 and 1986

Some sources date the terms to 1986. The journal paper says that a version of it also appears as a chapter in User Centered System Design, a book edited by Norman and Draper in 1986. Both dates point to the same work. This entry uses 1985, the year of the journal paper.

032 min

How the gulf of evaluation works

After a person acts, three steps decide whether they understand the result. They perceive what the system now shows. They interpret what they perceived. Then they compare that state with the goal they started with. Norman's seven stages begin with the goal, plan the action and carry it out, and end with exactly these three steps. A gulf of evaluation exists when any of them takes effort the display could have saved.

The three steps can fail one at a time. Whitenton perceived the switch and the label without trouble, and she failed at interpretation. Another display fails earlier, because it never shows the state. A third fails last: the person reads the state correctly but cannot tell whether it matches the goal. A fix that only makes the status more visible helps in the first case and does nothing in the other two.

The effort comes from translation. When a display describes the system in one set of terms and the person's goal is in another, the person has to convert between them. Hutchins, Hollan and Norman call the size of that conversion semantic distance. In their account, the gulf narrows when the display gives a good conceptual model of the system, one that is easy to perceive, interpret and evaluate.

The water tank: the right information in the wrong form

The 1985 paper gives a plain case. A person wants to control how fast the water level in a tank rises. They act and then read the display. If the display shows only the current level, they must watch the level over time and work out for themselves how fast it is changing. The information they need is on the screen, but not in a form that answers their question. The authors show a second display that draws the rate of change directly, and it takes that calculation away from the person. Both displays come from a working system called Steamer, described by Hollan, Hutchins and Weitzman in 1984.

041 min

Three Mile Island: a display that answered a different question

A wide gulf costs most where a wrong reading is dangerous. At about 4 a.m. on 28 March 1979, a relief valve at the top of the pressurizer in Unit 2 of the Three Mile Island nuclear plant opened to release pressure. It should have closed again when the pressure fell, but it stuck open. Instruments in the control room indicated that the valve was closed, so the staff did not know that cooling water was escaping through it.

The control room had a lamp for the valve. Operators took the lamp going out to mean that the valve had closed. In fact the lamp only showed that the pilot valve's solenoid, an electromagnet, was powered, not where the main relief valve was. The operators did not diagnose the problem correctly for several hours.

Compare this with Whitenton's switch. In both cases the display was visible and was read with confidence. In both cases the display reported something other than what the person was asking about. The switch label gave the current state to a reader who was looking for the effect of an action. The lamp gave the command to a reader who was looking for the result of the command. In the settings screen the mistake cost about an hour. In the control room it went on for hours.

A close view of a control panel in the Three Mile Island Unit 2 control room, cropped from a photograph of a visit by President Carter.
Fig. 1Reading the state of the plant meant reading panels such as this one.NRC · Public domain · Wikimedia Commons
valve closedlamp outvalve stuck opencoolant escapes
The lamp is out, so the operator believes the valve is closed. Look at what the valve is doing on the right.

052 min

The gulf of evaluation in everyday interfaces

The same question can be asked of many displays: what does this tell the person about what is true now?

DisplayWhat the person wants to knowGulf
A light switch and the lamp it controlsIs the light on?Narrow. The lamp is the result.
A spinning wheel with nothing elseIs it working, and how much is left?Wide. It shows activity, not progress.
A progress barHow much is left?Narrower. Progress is stated in the unit the person cares about.
A Submit button with no responseDid my data arrive?Wide. Nothing says whether to wait, try again or leave.
A search page that does not show which filters are onWhich filters apply now?Wide. The person must remember or test.
An appliance with one light that means several thingsWhat state is it in?Wide. The person must learn a code.

Norman's own film projector shows the problem with a physical device. Even after the film was in the projector, it was hard to tell whether it had been threaded correctly.

The Nielsen Norman Group compares the Windows 10 switch with the checkbox that Windows 7 used for the same setting. It judged that the checkbox demanded much less evaluation effort from the person, because the state depends only on whether the box holds a mark, not on colour or on where a label sits.

From line editors to screen editors

Hutchins, Hollan and Norman list the move from editors that work on lines to editors that work on the whole screen as another case of matching the display to what the person means. A screen editor shows the result of each editing command at once, so the person does not have to picture the document.

061 min

Response time and the gulf of evaluation

A display can show the right thing and still arrive too late. Jakob Nielsen described three response-time limits in his 1993 book Usability Engineering. About 0.1 second feels instant, and no special feedback is needed beyond showing the result. About one second is the longest delay that leaves a person's flow of thought unbroken, and about ten seconds is the longest that keeps their attention on the task.

Beyond ten seconds people turn to other work, so Nielsen says they should be told when the computer expects to finish. For long operations he recommends a percent-done indicator. This explains the table above: a spinning wheel says that something is happening, but a person cannot tell a slow task from a stuck one.

071 min

What the gulf of evaluation does not mean

It is not only feedback after a click

It is easy to hear the gulf of evaluation as the question did my click work. The authors draw it more widely. The gulf of evaluation runs from the state of the system to the person's goals. It therefore applies when someone returns to a screen after a break and needs to know where things stand, or reads a dashboard without having acted at all.

081 min

Gulf of evaluation vs. nearby concepts

The nearest neighbour is the gulf of execution. The two sit on opposite sides of an action, and they depend on each other. Once Whitenton read the switch correctly she knew at once how to turn Bluetooth on, and any plan built on a wrong reading of the state fails.

ConceptQuestion it answersDirection
DesignGulf of executionFrom goal to actionQuestion it answers: What can I do, and how?
This pageGulf of evaluationFrom system state to goalQuestion it answers: What happened, and is it what I wanted?
DesignVisibility of system statusA design rule that narrows the gulfQuestion it answers: Is the state shown at all?
Not in the library yetSignifiers and affordancesExecution sideQuestion it answers: Where can I act?

Visibility of system status is a rule for designers. The gulf is the measure of how well a design follows it for a given person. A display can follow the rule, as Whitenton's did with its Off label, and still leave the gulf wide. Signifiers and affordances belong to the execution side: they tell a person where an action can be taken.

092 min

Narrowing the gulf of evaluation

Whitenton's switch and the Three Mile Island lamp suggest a short list of changes.

  1. Show the result, not the command.

    A display driven by the signal sent to a part says that a request was made. A display driven by the part itself says what the part did.

  2. Use the person's terms.

    If they are asking how fast something is changing, show the rate.

  3. Borrow a mental model they already have.

    A mental model is a person's theory of how a system works. A checkbox works because it comes from paper forms: a mark means chosen, no mark means not.

  4. Give feedback after every action and keep the current state visible.

    This is the rule behind visibility of system status.

  5. Answer within the limits above,

    and for long waits say how much is left.

  6. Write errors in plain language

    that says what failed and what to do next. After a risky action such as deleting, name what was deleted.

  7. Show fewer states.

    Every state a person never needs to know about is one more thing to read.

To check the result, an expert review such as a heuristic evaluation finds missing status displays quickly. Usability tests find misreadings: ask a person what state the system is in and what just happened, then compare their answer with the truth. An A/B test can compare two wordings of one message.

Two routes to a narrower gulf

Hutchins, Hollan and Norman name two routes. The designer changes the system so that the display moves toward the person, or the person builds new skill and learns to read the system on its own terms. The first costs the designer effort once. The second costs every user effort, and only helps those who get the chance to learn. At Three Mile Island the operators had not been trained to understand that the lamp was ambiguous, and the lamp gave no direct indication of the valve's position.

101 min

Where narrowing the gulf of evaluation goes wrong

More display is not always a narrower gulf. A screen that announces every internal event makes the one result the person wants hard to find. The width of a gulf also depends on the person and the goal, not on the screen alone: a trained operator reads a panel that a newcomer cannot.

Specialised displays lose range

A display built around one question answers that question well and others poorly. Hutchins, Hollan and Norman make the point about interface languages in general: a language that matches one domain closely loses generality, and tasks outside the domain become hard to express. The rate display for the tank is a small case. It helps a person who asks about speed and gives no help to one who needs a different quantity, such as the total volume. A designer has to choose which questions the display answers.

111 min

How to spot a wide gulf of evaluation

A wide gulf shows up in recordings, logs and support tickets.

  • The same form is submitted twice, or the same button is pressed several times in a row.
  • A person leaves the screen to confirm the result somewhere else, such as opening an inbox to see whether a message was sent.
  • Support requests ask did it work or what state is it in, not how do I do this.
  • A page is reloaded straight after an action.
  • In a usability test, a person says they are not sure whether anything happened, or states the system's state wrongly. The wrong statement is the best evidence.

The same gulf appears on a phone, a website or a desktop program. Only the places where the state can be shown differ.

121 min

The gulf of evaluation and accessibility

A display that works for sighted people can leave a screen reader user with no way to evaluate. A message that appears after an action, such as a note that an item was added, is a change of state. If it appears without moving keyboard focus and is not marked up for assistive technology, a screen reader user may never learn that it happened.

WCAG 2.2 Success Criterion 4.1.3, Status Messages (Level AA), requires that a status message can be presented by assistive technology without taking keyboard focus. The W3C names blind and low-vision users of screen readers as the intended beneficiaries.

Two neighbouring criteria apply to the same reading task. SC 3.3.1, Error Identification (Level A), requires that an error found in input is identified and described in text. SC 1.4.1, Use of Color (Level A), requires that colour is not the only visual means of conveying information, so a red error or green success needs a word or a symbol beside it.

?4 questions

Questions people ask

How do you measure the width of a gulf of evaluation?

There is no standard unit. Teams ask people what state the system is in and what just happened, then count wrong answers, long pauses and repeated actions. Each one marks a place where the display fails.

How do slips and mistakes relate to the gulf of evaluation?

A slip is a correct goal carried out wrongly, and a mistake is a wrong goal or plan. A wide gulf can cause a mistake: the person acts on a false belief about the state, as the operators did.

Does the gulf of evaluation apply to AI tools?

Yes. A 2026 research preprint on generating interface designs says users struggle to evaluate and refine results, and calls this a gulf of evaluation. The person must judge whether generated output matches their intent.

Do the seven stages always happen in order?

No. The stages are a simplified model. A person may start from something they see, not from a goal, and several stages can happen together. The model is useful for finding where the effort goes.

§12 sources

Sources for the gulf of evaluation

Show all 12 sources
  1. Hutchins, E. L., Hollan, J. D. and Norman, D. A. (1985). "Direct Manipulation Interfaces". Human-Computer Interaction, 1(4), 311-338.

  2. Whitenton, K. (2018). "The Two UX Gulfs: Evaluation and Execution". Nielsen Norman Group.

  3. Wikipedia, "Three Mile Island accident": the control room lamp and what it indicated.

  4. U.S. Nuclear Regulatory Commission, "Backgrounder on the Three Mile Island Accident": the stuck valve and the control room instruments.

  5. Nielsen, J. (1993). "Response Times: The 3 Important Limits". Nielsen Norman Group.

  6. W3C, "Understanding Success Criterion 4.1.3: Status Messages", WCAG 2.2.

  7. W3C, "Understanding Success Criterion 3.3.1: Error Identification", WCAG 2.2.

  8. W3C, "Understanding Success Criterion 1.4.1: Use of Color", WCAG 2.2.

  9. Wikipedia, "The Design of Everyday Things": a summary of Norman's 1988 book, its seven stages and the film projector example.

  10. Wikipedia, "Human action cycle": the seven steps and which belong to evaluation.

  11. Park, S. et al. (2026). "Bridging Gulfs in UI Generation through Semantic Guidance". arXiv:2601.19171.

  12. Dix, A. (2003). Course extract on Norman's model, from the teaching notes for Human-Computer Interaction, chapter 3 "The interaction": expected state against presented state, and slips against mistakes.

Keep reading

More from Design

All of Design
All of Design

Missing a concept?

Tell us what to write next. We’ll email you when it’s live.

Several? Separate them with commas.

Only to tell you when it’s live.

How we use your email: privacy.