Variable Reward

Also called variable rewards, variable-ratio reinforcement, intermittent reward. Also in Product, Design

Quick answer

A variable reward is a payoff that follows an action but changes in size or timing each time, so the person cannot predict it. Psychologists first studied it as the variable-ratio schedule, in which a reward arrives after an unpredictable number of responses. Product teams use it because an uncertain payoff makes the next attempt worth trying.

~20 sec read

011 min

Why a fixed reward fades and variable reward does not

Picture a product manager at a language-learning app. Each finished lesson gives the learner 10 points, every time. The numbers in this example are made up.

In the first weeks, learners come back daily. Later, the same learners skip days, and the points counter has stopped changing what anyone does. The manager's first fix is to raise the reward to 20 points. That fails for the reason the first number failed: the learner already knows what a lesson pays.

A reward that can be predicted exactly gives the learner nothing new to look forward to. Animal laboratories found this decades before apps existed, and they also found what to do about it.

021 min

Where variable reward came from

B. F. Skinner and Charles Ferster tested the question with animals in laboratory chambers. They set a rule for when a response, such as pressing a lever, earned food. Then they watched what the animal did under each rule. They called each rule a schedule.

Their 1957 book Schedules of Reinforcement records a distinct pattern of responding for each rule. One rule stood out. Under the variable-ratio schedule, food arrives after an unpredictable number of responses, around a fixed average. Of the simple rules, it made animals respond fastest, and their responding lasted longest after the food stopped.

Slot machines pay out on this kind of schedule. That is why they are the usual real-world example of the rule.

A slot machine standing in a public venue in New Orleans.
Fig. 1A slot machine, the usual real-world case of a variable-ratio schedule.Infrogmation of New Orleans · CC BY 2.0 · Wikimedia Commons

033 min

How variable reward works

The slot machine shows the idea, but the full picture needs the other rules for contrast. A schedule links responses to rewards in two choices. The reward can follow a count of responses (a ratio) or the passing of time (an interval). The count or the wait can be the same every round (fixed) or change (variable). Crossing the two choices gives four schedules.

ScheduleRuleEveryday caseTypical pattern
Fixed ratioReward after every nth responseA card that earns a free coffee after ten purchasesBursts of effort, then a pause after each reward
Variable ratioReward after an unpredictable number of responses, around an averageA slot machineSteady, fast responding
Fixed intervalFirst response after a set time is rewardedA weekly quiz on the same dayResponding speeds up as the time nears
Variable intervalFirst response after an unpredictable wait is rewardedWaiting for a message that arrives at random timesSteady, slow responding

Under a variable-ratio rule, nothing in the last round tells the player how close the next reward is. The sections below add how that plays out in behaviour and in the brain.

Why behaviour outlasts the reward

A reward that stops is the real test of a schedule. Humphreys ran an early version in 1939 with an eyelid-blink response. One group was rewarded every time. Another group was rewarded on a random mix of trials. The group on the random mix kept responding far longer once rewards ended. This surprised researchers, and it became known as Humphreys' paradox.

The usual explanation is about noticing. When every response used to pay, a sudden stop is easy to spot. When long dry runs were already normal, a stop looks like another dry run.

Why ratio rules beat interval rules

Ratio rules also produce faster responding than interval rules when the overall reward rate is the same. The link between effort and payoff explains it. Under a ratio rule, each extra response brings the next reward closer. Under an interval rule, extra responses before the wait ends earn nothing.

What the brain does with an unpredictable payoff

Wolfram Schultz and colleagues recorded dopamine-producing neurons in monkeys. In one task, the monkey touched a lever after a light came on and received fruit juice. Early on, the neurons fire a short burst just after the juice arrives. After several days of training, the juice no longer triggers the burst. The light now triggers it. The neurons respond to a reward that was not expected, and go quiet for one that was.

This is the usual explanation for why variable rewards hold attention: some of the payoff always stays unexpected. It is an inference from the lab results. The app studies cited here did not measure it.

Why even odds build the strongest anticipation

In 2003, Christopher Fiorillo, Philippe Tobler and Wolfram Schultz varied the chance of a reward from zero to one. They found a second signal beside the burst. It grew gradually as the possible time of the reward got closer, and it tracked uncertainty. Uncertainty is greatest when the chance is one in two.

earlylightjuiceafter traininglightjuice
Compare where the star sits in each window. After training, it moves from the juice to the light, and the juice has a faded star.

042 min

Variable reward in products

Product designers met this research through a book. Nir Eyal's Hooked (2014) puts variable reward third in a four-step loop called the Hook Model: trigger, action, variable reward, investment. The investment step is where the user puts in effort that makes the next visit better.

Eyal sorts variable rewards into three kinds.

KindWhat paysExample
TribeApproval or connection from other peopleReplies and likes on a post
HuntMoney, or information that was searched forScrolling a feed to find one good item
SelfThe feeling of mastery or finishingBeating a level

Back to the language app. The manager keeps the total payout the same and changes only its pattern. Over five lessons, the fixed rule pays 10 points each time. The variable rule pays 5, 5, 5, 30 and 5. Both pay 50 points in total, but the learner cannot tell in advance which lesson pays 30. The numbers are made up.

A feed is an interval schedule, not a ratio

People often call email and social feeds variable-ratio, and the label is slightly off. A variable-interval schedule rewards the first response after an unpredictable wait. In an inbox, new mail comes when it comes, so checking more often does not bring more of it. The slot-machine pattern fits designs where each action, such as another swipe or another pack opened, draws a new random result.

A documented case: loot boxes

Loot boxes are items in video games with randomised contents, bought with real money. In 2018, David Zendle and Paul Cairns surveyed 7,422 gamers. The more a person spent on loot boxes, the more severe their problem gambling. The link was stronger than the one for other in-game purchases. The authors could not tell whether loot boxes lead to problem gambling or attract people who already gamble.

A replication with a different sample

In 2019 the same authors repeated the survey with 1,172 gamers who had not chosen to join a loot box study and did not know its aim. They again found a link of similar size between spending and problem gambling.

How people split time between two sources of reward

Richard Herrnstein showed in 1961 that, when two schedules run at once, an animal's response rate to each option matches the share of rewards each one produced. For a product, the lab rule suggests that a feed that pays more often takes a proportionally larger share of a person's attention from a feed that pays less. The lab result is firm. The product reading is a prediction.

five lessonsfixed10variable555305
Both rows add up to the same total. Look at which lesson stands out in the lower row.

052 min

How to add variable reward to a product this week

Start with one action users already repeat, and work through these steps in a spreadsheet or a doc.

  1. Name the action. For the language app, it is finishing a lesson.
  2. List what already varies. New posts from other people, new items to find and a learner's own progress all differ each time. Use something that is really different before you add a random prize.
  3. Keep the average payoff good. Fix the total value a user gets over many actions, then change only the pattern, as in the five-lesson example.
  4. Make the reward follow the action. A reward tied to something the user did can teach that action.
  5. Measure return rate by cohort in the second and sixth week, and add one question about whether people feel better off for having used the product.

Reward audit row: action | what pays | fixed or variable | average value to the user

Filling in one row per reward gives you the artifact for the day: a reward audit that shows which rewards are fixed, which vary, and what each is worth.

Start steady, then thin the reward

Trainers who shape a new behaviour usually reward every response first and move to an intermittent rule later. A new user has no history of payoffs yet. Reward the first lessons every time, then make the pattern variable once the habit exists.

062 min

When variable reward fails

Variable reward is a tool for repeat actions, and it breaks in three known ways.

A utility does not need surprise

Jason Hreha, a behavioural scientist, argues that Google won by making its answers more reliable, not less. He also notes that Uber, one of the most habit-forming apps, does not rely on variable rewards. People use these products because they solve a problem. Someone opening a payments app does not want a surprise.

A random reward that is not tied to the action

Skinner also ran an experiment in which hungry pigeons got food about every 15 seconds, whatever they were doing. The pigeons developed rituals, such as turning in circles. Whatever a bird happened to be doing when food arrived was accidentally reinforced. An app that fires random confetti can do the same: it reinforces whatever the user did a moment before.

A paid reward can lower interest

In 1973, Mark Lepper, David Greene and Richard Nisbett promised children who already liked drawing a certificate for doing it. Those children later drew less in free play than children who were promised nothing. This is called the overjustification effect. A badge on an activity people already enjoy can shift their reason for doing it from enjoyment to the badge.

A variable reward becomes a dark pattern when the design works against the user's interest, as the loot box spending shows.

071 min

Where the evidence for variable reward is contested

Three things in this article stand on weaker ground than the schedule tables suggest.

Animals are not users

The core schedule findings come from rats and pigeons in tightly controlled chambers. That limits how directly they carry over to people, who follow rules, make plans and have reasons. The monkey neuron work measures cells, not app habits. No study cited here tests a particular app's reward schedule on its users.

The dopamine ramp is still debated

Niv and colleagues argued in 2005 that the gradual rise Fiorillo's team saw may partly come from averaging across trials. Treat the ramp as a strong finding under discussion, not a settled mechanism.

Direction of cause in the loot box studies

Both Zendle and Cairns surveys found a link, and both say the link does not show which way it runs. People who gamble may simply buy more loot boxes.

What this means for a practitioner: treat variable reward as a design hypothesis. Test it against a fixed-reward group and watch both return rate and how people say they feel about the product.

081 min

Variable reward vs nearby concepts

Several neighbouring ideas get mixed up with variable reward. The table gives the one fact that separates each.

ConceptWhat it isWhat separates it from variable reward
Not in the library yetIntermittent reinforcementIt includes fixed rules. Variable reward is the part where the unpredictability is built into the ruleWhat it is: Any rule that rewards only some responses
PsychologyDopamine loopsIt describes the brain's response. Variable reward is the payoff pattern a designer setsWhat it is: The cycle of cue, anticipation and payoff in the brain
PsychologyGamificationA points counter is a fixed reward. Variable reward is one mechanic among severalWhat it is: Game mechanics, such as points and badges, added to a non-game product
MarketingCuriosity gapThe reader knows something is missing. In a variable reward, the person does not know if a payoff will comeWhat it is: A known gap in information that a reader wants to close

"Variable" also does not mean "unlimited randomness". A variable-ratio schedule has a set average, and only the timing around that average changes.

?3 questions

Questions people ask

Is a streak counter a variable reward?

Not by itself. A plain streak counter rewards every completion in the same way, which is a steady schedule. It becomes variable only when something unpredictable is added, such as a bonus the user cannot see coming.

Does a variable reward have to be a prize?

No. Any outcome that differs each time can serve, such as new posts in a feed or a reply from another person. What matters is that the person cannot predict the outcome before acting.

When should a new user get the first reward?

In the first session. A new user has no history of payoffs yet, so an early reward shows that acting pays. A steady reward suits this stage, and a variable one can follow once the habit forms.

§10 sources

Sources

Show all 10 sources
  1. Ferster, C. B. and Skinner, B. F. (1957). Schedules of Reinforcement. Appleton-Century-Crofts.

  2. Schultz, W., Dayan, P. and Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275.

  3. Fiorillo, C. D., Tobler, P. N. and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299.

  4. Zendle, D. and Cairns, P. (2018). Video game loot boxes are linked to problem gambling. PLOS ONE.

  5. Zendle, D. and Cairns, P. (2019). Loot boxes are again linked to problem gambling. PLOS ONE.

  6. Simply Psychology, a summary of reinforcement schedules.

  7. Wikipedia, the entry on reinforcement.

  8. Hreha, J., a review of Nir Eyal's Hooked.

  9. Sketchplanations, a summary of variable rewards and the Hook Model.

  10. FineStreak, a summary of reward schedules and habits.

Keep reading

More from Psychology

All of Psychology
All of Psychology

Missing a concept?

Tell us what to write next. We’ll email you when it’s live.

Several? Separate them with commas.

Only to tell you when it’s live.

How we use your email: privacy.