Natural Variance

If you've learned about the 4 outcomes, and the impact of active churn on passive churn, you're ready to explore natural variance.

Natural variance refers to the normal amount that any metric will fluctuate over time, assuming you change nothing about your process.

Smaller brands will see more of it, larger brands will see less of it.

Typically, we'll look at 30-day rolling periods (learn more about this in the next lesson), and compare how different the results are for the last year.

Let's say you're planning on optimizing your passive churn recovery, and a 10% increase in recovery rate would be a huge win. You look at the last completed month and see a 64% recovery rate. So you set the target to be 74%.

The next month, you try a different process that's expensive, but worth it if it delivers!

You achieve 73% recovery. Pretty good, right?

Not necessarily. Had you looked at natural variance, and a broader baseline, you'd know that your recovery rate for 30-day time periods is between 64% and 78%, with most time periods falling between 76% and 78%. Turns out that 64% month was an under-performing outlier, and your new process is actually under-performing your typical range of performance.

Without identifying natural variance, attribution can be a mess, and you're bound to be yanked around by misleading conclusions, and costly mistakes.

Once you've identified your natural variance, there are a couple ways to reduce natural variance:

1. Run a longer test, with longer time periods: Looking at 60-day periods will show less natural variance than looking at 30-day, 14-day, etc time periods.
2. Use segmentation: Natural variance is largely caused by different cohorts of customers running through the process over time. And much of that variance is caused by 2nd, 3rd, etc renewals, when customers are newer and more impacted by discounts or promotions you're running. With segmentation, we can exclude those customers and focus on the performance of long-term customers with less natural variance.

Next Up: Conducting Rolling Analysis

Once your data is formatted in daily cohorts, with the 4 outcomes broken out, and you've learned about active/passive churn, and natural variance, you're ready to conduct rolling analysis.

The biggest mistake companies make with passive churn analysis is comparing arbitrary date ranges against each other, not accounting for natural variance.

Rolling analysis identifies the natural variance, and adds necessary context to your analysis, leading to proper attribution.

Learn more about rolling analysis.