011 min
Why a fixed reward fades and variable reward does not
Picture a product manager at a language-learning app. Each finished lesson gives the learner 10 points, every time. The numbers in this example are made up.
In the first weeks, learners come back daily. Later, the same learners skip days, and the points counter has stopped changing what anyone does. The manager's first fix is to raise the reward to 20 points. That fails for the reason the first number failed: the learner already knows what a lesson pays.
A reward that can be predicted exactly gives the learner nothing new to look forward to. Animal laboratories found this decades before apps existed, and they also found what to do about it.
021 min
Where variable reward came from
B. F. Skinner and Charles Ferster tested the question with animals in laboratory chambers. They set a rule for when a response, such as pressing a lever, earned food. Then they watched what the animal did under each rule. They called each rule a schedule.
Their 1957 book Schedules of Reinforcement records a distinct pattern of responding for each rule. One rule stood out. Under the variable-ratio schedule, food arrives after an unpredictable number of responses, around a fixed average. Of the simple rules, it made animals respond fastest, and their responding lasted longest after the food stopped.
Slot machines pay out on this kind of schedule. That is why they are the usual real-world example of the rule.

033 min
How variable reward works
The slot machine shows the idea, but the full picture needs the other rules for contrast. A schedule links responses to rewards in two choices. The reward can follow a count of responses (a ratio) or the passing of time (an interval). The count or the wait can be the same every round (fixed) or change (variable). Crossing the two choices gives four schedules.
| Schedule | Rule | Everyday case | Typical pattern |
|---|---|---|---|
| Fixed ratio | Reward after every nth response | A card that earns a free coffee after ten purchases | Bursts of effort, then a pause after each reward |
| Variable ratio | Reward after an unpredictable number of responses, around an average | A slot machine | Steady, fast responding |
| Fixed interval | First response after a set time is rewarded | A weekly quiz on the same day | Responding speeds up as the time nears |
| Variable interval | First response after an unpredictable wait is rewarded | Waiting for a message that arrives at random times | Steady, slow responding |
Under a variable-ratio rule, nothing in the last round tells the player how close the next reward is. The sections below add how that plays out in behaviour and in the brain.
Why behaviour outlasts the reward
A reward that stops is the real test of a schedule. Humphreys ran an early version in 1939 with an eyelid-blink response. One group was rewarded every time. Another group was rewarded on a random mix of trials. The group on the random mix kept responding far longer once rewards ended. This surprised researchers, and it became known as Humphreys' paradox.
The usual explanation is about noticing. When every response used to pay, a sudden stop is easy to spot. When long dry runs were already normal, a stop looks like another dry run.
Why ratio rules beat interval rules
Ratio rules also produce faster responding than interval rules when the overall reward rate is the same. The link between effort and payoff explains it. Under a ratio rule, each extra response brings the next reward closer. Under an interval rule, extra responses before the wait ends earn nothing.
What the brain does with an unpredictable payoff
Wolfram Schultz and colleagues recorded dopamine-producing neurons in monkeys. In one task, the monkey touched a lever after a light came on and received fruit juice. Early on, the neurons fire a short burst just after the juice arrives. After several days of training, the juice no longer triggers the burst. The light now triggers it. The neurons respond to a reward that was not expected, and go quiet for one that was.
This is the usual explanation for why variable rewards hold attention: some of the payoff always stays unexpected. It is an inference from the lab results. The app studies cited here did not measure it.
Why even odds build the strongest anticipation
In 2003, Christopher Fiorillo, Philippe Tobler and Wolfram Schultz varied the chance of a reward from zero to one. They found a second signal beside the burst. It grew gradually as the possible time of the reward got closer, and it tracked uncertainty. Uncertainty is greatest when the chance is one in two.
042 min
Variable reward in products
Product designers met this research through a book. Nir Eyal's Hooked (2014) puts variable reward third in a four-step loop called the Hook Model: trigger, action, variable reward, investment. The investment step is where the user puts in effort that makes the next visit better.
Eyal sorts variable rewards into three kinds.
| Kind | What pays | Example |
|---|---|---|
| Tribe | Approval or connection from other people | Replies and likes on a post |
| Hunt | Money, or information that was searched for | Scrolling a feed to find one good item |
| Self | The feeling of mastery or finishing | Beating a level |
Back to the language app. The manager keeps the total payout the same and changes only its pattern. Over five lessons, the fixed rule pays 10 points each time. The variable rule pays 5, 5, 5, 30 and 5. Both pay 50 points in total, but the learner cannot tell in advance which lesson pays 30. The numbers are made up.
A feed is an interval schedule, not a ratio
People often call email and social feeds variable-ratio, and the label is slightly off. A variable-interval schedule rewards the first response after an unpredictable wait. In an inbox, new mail comes when it comes, so checking more often does not bring more of it. The slot-machine pattern fits designs where each action, such as another swipe or another pack opened, draws a new random result.
A documented case: loot boxes
Loot boxes are items in video games with randomised contents, bought with real money. In 2018, David Zendle and Paul Cairns surveyed 7,422 gamers. The more a person spent on loot boxes, the more severe their problem gambling. The link was stronger than the one for other in-game purchases. The authors could not tell whether loot boxes lead to problem gambling or attract people who already gamble.
A replication with a different sample
In 2019 the same authors repeated the survey with 1,172 gamers who had not chosen to join a loot box study and did not know its aim. They again found a link of similar size between spending and problem gambling.
How people split time between two sources of reward
Richard Herrnstein showed in 1961 that, when two schedules run at once, an animal's response rate to each option matches the share of rewards each one produced. For a product, the lab rule suggests that a feed that pays more often takes a proportionally larger share of a person's attention from a feed that pays less. The lab result is firm. The product reading is a prediction.
052 min
How to add variable reward to a product this week
Start with one action users already repeat, and work through these steps in a spreadsheet or a doc.
- Name the action. For the language app, it is finishing a lesson.
- List what already varies. New posts from other people, new items to find and a learner's own progress all differ each time. Use something that is really different before you add a random prize.
- Keep the average payoff good. Fix the total value a user gets over many actions, then change only the pattern, as in the five-lesson example.
- Make the reward follow the action. A reward tied to something the user did can teach that action.
- Measure return rate by cohort in the second and sixth week, and add one question about whether people feel better off for having used the product.
Reward audit row: action | what pays | fixed or variable | average value to the user
Filling in one row per reward gives you the artifact for the day: a reward audit that shows which rewards are fixed, which vary, and what each is worth.
Start steady, then thin the reward
Trainers who shape a new behaviour usually reward every response first and move to an intermittent rule later. A new user has no history of payoffs yet. Reward the first lessons every time, then make the pattern variable once the habit exists.
062 min
When variable reward fails
Variable reward is a tool for repeat actions, and it breaks in three known ways.
A utility does not need surprise
Jason Hreha, a behavioural scientist, argues that Google won by making its answers more reliable, not less. He also notes that Uber, one of the most habit-forming apps, does not rely on variable rewards. People use these products because they solve a problem. Someone opening a payments app does not want a surprise.
A random reward that is not tied to the action
Skinner also ran an experiment in which hungry pigeons got food about every 15 seconds, whatever they were doing. The pigeons developed rituals, such as turning in circles. Whatever a bird happened to be doing when food arrived was accidentally reinforced. An app that fires random confetti can do the same: it reinforces whatever the user did a moment before.
A paid reward can lower interest
In 1973, Mark Lepper, David Greene and Richard Nisbett promised children who already liked drawing a certificate for doing it. Those children later drew less in free play than children who were promised nothing. This is called the overjustification effect. A badge on an activity people already enjoy can shift their reason for doing it from enjoyment to the badge.
A variable reward becomes a dark pattern when the design works against the user's interest, as the loot box spending shows.
071 min
Where the evidence for variable reward is contested
Three things in this article stand on weaker ground than the schedule tables suggest.
Animals are not users
The core schedule findings come from rats and pigeons in tightly controlled chambers. That limits how directly they carry over to people, who follow rules, make plans and have reasons. The monkey neuron work measures cells, not app habits. No study cited here tests a particular app's reward schedule on its users.
The dopamine ramp is still debated
Niv and colleagues argued in 2005 that the gradual rise Fiorillo's team saw may partly come from averaging across trials. Treat the ramp as a strong finding under discussion, not a settled mechanism.
Direction of cause in the loot box studies
Both Zendle and Cairns surveys found a link, and both say the link does not show which way it runs. People who gamble may simply buy more loot boxes.
What this means for a practitioner: treat variable reward as a design hypothesis. Test it against a fixed-reward group and watch both return rate and how people say they feel about the product.
081 min
Variable reward vs nearby concepts
Several neighbouring ideas get mixed up with variable reward. The table gives the one fact that separates each.
| Concept | What it is | What separates it from variable reward |
|---|---|---|
| Not in the library yetIntermittent reinforcement | ||
| PsychologyDopamine loops | ||
| PsychologyGamification | ||
| MarketingCuriosity gap |
"Variable" also does not mean "unlimited randomness". A variable-ratio schedule has a set average, and only the timing around that average changes.
?3 questions
Questions people ask
Is a streak counter a variable reward?
Does a variable reward have to be a prize?
When should a new user get the first reward?
§10 sources
Sources
Show all 10 sourcesHide sources
Ferster, C. B. and Skinner, B. F. (1957). Schedules of Reinforcement. Appleton-Century-Crofts.
Schultz, W., Dayan, P. and Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275.
Fiorillo, C. D., Tobler, P. N. and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299.
Zendle, D. and Cairns, P. (2018). Video game loot boxes are linked to problem gambling. PLOS ONE.
Zendle, D. and Cairns, P. (2019). Loot boxes are again linked to problem gambling. PLOS ONE.
Simply Psychology, a summary of reinforcement schedules.
Wikipedia, the entry on reinforcement.
Hreha, J., a review of Nir Eyal's Hooked.
Sketchplanations, a summary of variable rewards and the Hook Model.
FineStreak, a summary of reward schedules and habits.





