Everyone's eager to see early results when running tests — but it's important to know when those results can be calculated accurately. You might be surprised to learn that during a 20-day dunning campaign, you still can't calculate a recovery rate accurately after 7 days of testing.
Imagine you've launched a new dunning configuration. Seven days later you have 400 customers recovered, 100 canceled, and 500 still in active campaigns. What's your recovery rate?
Calculate an 80% recovery rate by excluding the 500 still-in-progress campaigns: 400 ÷ (400 + 100). It's tempting because it gives you a number now — but it's an inflated, unreliable measurement that will always overstate performance.
Calculate a 40% recovery rate by including the 500 still-in-progress campaigns in the denominator: 400 ÷ 1,000. It's a conservative early read, but it converges on reality as the campaign completes — a number you can build on.
You need at least 30 days for early-campaign results to average out the natural variance — declined cards, retry windows, customer-side delays — that make 7-day measurements unreliable no matter how you calculate them.
Option A will always mislead. Option B is conservative now and trustworthy later. Pick the methodology that gets you closer to truth, not closer to the answer you want.
Our team can walk through your data and methodology with you.
Or email support@churnbuster.io any time.
