Skip to content
Novandri.

Loyalty Programs: When Points Change Behavior

Points can shift behaviour or just reward people who were already loyal. Read the measurement problem and a worked holdout test.

Rachmadwipa Novandri4 min read

Loyalty programmes are easy to launch and hard to evaluate. Members spend more than non-members, which is reassuring until you ask whether the points caused the spending or whether frequent customers simply joined. The difference between those two explanations decides whether the programme creates value or transfers margin to people who would have stayed anyway.

Direct answer:

Measure a loyalty programme with a comparison group, not a member-versus-non-member average, and focus on a behaviour the programme can plausibly change.

AAPOR's standards govern survey-based claims about member attitudes, and demand-generation principles distinguish preference from capture. The boundary: a points balance is not evidence of changed behaviour, and a redemption is not proof of incremental revenue, as Marketing Attribution: Multi-Touch Models, Incrementality, and Budget Allocation explains for credit rules.

A four-step flow: define the behaviour, set a holdout, compare groups, then count the points cost.
Conceptual: a holdout separates selection from effect. Comparing members with non-members confounds selection with effect.

Three questions

What behaviour should change? More frequent purchase, larger basket, longer retention, or a specific action. A programme with no defined behaviour cannot be evaluated, which the retention article argues for cohorts and stages.

Who would have done it anyway? Members are selected — people who already like the brand join.

What does the reward cost? Points are a liability and a margin cost. The benefit must exceed that cost, which the unit-economics article explains for contribution margin.

A worked holdout test

The following is hypothetical and illustrative.

A retailer launches a points programme to 200,000 customers and holds out a matched group of 50,000 who receive no invitation.

After six months:

  • Invited group: 12% increase in purchase frequency versus baseline.
  • Holdout group: 7% increase over the same period.
  • Estimated incremental effect: about 5 percentage points.

Three checks decide whether it is worth keeping:

  1. Group comparability — were the two groups matched on prior spending and demographics? A non-comparable holdout invalidates the difference, as the geo-experiment guidance explains for comparison regions.
  2. Margin after rewards — does the extra frequency cover the cost of points redeemed? Contribution margin, not revenue, is the measure, which the unit-economics article provides.
  3. Selection over time — did high spenders join at different rates, changing the member mix? The retention article warns that averages hide cohort shifts, which the AAPOR standards reinforce for any survey of members.

Three conclusions follow:

  • The naive member-versus-non-member comparison would overstate the effect, because it ignores the holdout.
  • The 5-point estimate is the decision input, not the 12% raw change.
  • The programme's future depends on margin, not on points issued, which the discount-strategy article explains for incentives generally.

The cost side of points

A programme that changes behaviour can still destroy value if rewards are priced poorly. Three items belong in the cost model:

Redemption liability. Issued points are a future obligation. The unit-economics article explains how to account for a cost that arrives later than the behaviour it rewarded.

Breakage assumptions. Not every point is redeemed, and a programme that budgets on a high breakage rate is exposed when members redeem more than expected, which the risk-assessment article frames as a variable with uncertainty.

Margin, not revenue. A 5-point lift in frequency is worth the contribution margin on those purchases minus the reward cost, which the EBITDA article explains for the difference between accounting profit and what a business keeps.

These items turn a participation metric into a business decision, which the marketing-budget article recommends for all programme spend.

What loyalty programmes cannot do

They cannot create preference. Points can shift timing or basket size; they do not make a brand meaningful, which the demand-generation article argues.

They cannot fix a poor experience. Rewards for a frustrating experience delay churn rather than solve its cause, which the voice-of-customer article covers as a signal to investigate.

They cannot be evaluated by sentiment alone. Member satisfaction surveys are useful but self-selected; the AAPOR standards require a defined population and wording before any claim is made.

The counterargument — that competitors have loyalty programmes, so we need one — is a positioning argument, not evidence of incremental value.

The holdout answers the question better, as the competitive-positioning workshop article recommends for strategic claims.

A decision test

  1. Define the behaviour the programme should change.
  2. Set a holdout before launch, matched on prior behaviour.
  3. Measure the difference between groups, not member averages, as the geo-experiment guidance recommends.
  4. Subtract reward cost using contribution margin, as the unit-economics article explains.
  5. Track cohort mix so selection is not mistaken for effect.
  6. Use AAPOR-style wording for any member survey, with a defined population.
  7. Re-test after a programme change, as the monitoring article recommends.

For the measurement method, see Marketing Attribution; for the behaviour, see Customer Retention Strategies.

References & Further Reading

  • American Association for Public Opinion Research, 2026. AAPOR — The professional association for public opinion and survey research; its standards and resources underpin the survey-design discussion.
  • Wikipedia contributors, 2026. Demand generation - Wikipedia — An overview of demand generation as a marketing discipline distinct from lead generation; secondary reference used with the lead-generation entry.