Instacart — The $5 Rounding Rule Nobody Meant to Test

Instacart rounded every card-authorization hold up to the next $5, so orders a cent apart could get very different holds. Its economists used that quirk as a natural experiment to measure how holds drive card declines.

Key takeaways:

• The quirk. Each order gets a temporary hold above the order total to cover changes, and the buffer was rounded up to the next $5 multiple. A cent above a threshold meant a bigger hold.
• Why not just A/B test? Experimenting with live payment flows is risky, so the team first looked for a causal answer in historical data.
• Regression discontinuity. Customers just below and just above a $5 cutoff are essentially the same. Fitting local regressions on each side isolates the jump in declines caused by the bigger hold.
• Validated by experiment. A follow-up A/B test that reduced the upward rounding confirmed the finding: fewer card declines and a significant increase in order volume.
• Know the limits. RDD estimates a local effect for customers near the cutoff, not a universal law.

Chapters:

00:00 A tiny rounding rule

01:34 Authorization holds explained

03:08 Rounding to the nearest $5

03:53 The pain of a declined card

05:24 Why not A/B test payments?

06:10 An accidental experiment

06:55 Regression discontinuity design

09:12 Fitting each side of the cutoff

10:44 What the data showed

11:30 Confirming with an A/B test

13:01 Our take on novelty

15:19 Thresholds hiding in your product

16:51 RDD vs. other methods

19:10 Strengths and limits

21:27 Recap

Sources:

• Optimizing at the Edge: Using Regression Discontinuity Designs to Power Decision-Making (Levi Boxell, Robert Fletcher and Tilman Drerup, Instacart Tech)

Related episode: E31: Stanford & Marketplace Researchers — Rerandomizing Switchback Experiments

Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify.

Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups.

Made with AI-generated voices; curated by Peter Liu from the sources above.