The first auto-cut system most teams build is naive. It says: if EPV is below break-even, pause the placement. It saves money on day one. Sometime in week three, it kills a scaling winner that was about to come into its own. The team realizes what happened, turns the system off, and goes back to manual review.
The mistake isn’t the threshold. It’s treating the placement as if it has been observed enough to judge.
What the naive version misses
A placement with eighty visits and a four-cent EPV is not “underperforming.” It is under-sampled. At low visit counts, the variance of EPV is large enough that your point estimate is meaningless. Half the placements that look like losers at eighty visits would have been winners by visit three hundred. The naive system can’t tell the difference. It cuts them all.
The teams that get auto-cut right have made a different bet. They wait. They warm up. They model variance. They don’t cut on the point estimate.
What “wait” means in practice changes by vertical, by source, by offer. We don’t write the numbers down because the numbers are part of the work. The principle is portable. The settings are not.
What separates the systems that work
A few things, none of them about the threshold itself.
The system knows when it doesn’t know yet. It refuses to act on placements it hasn’t observed long enough. Most naive systems skip this step and pay for it.
The system measures confidence, not just performance. A placement that looks bad with high confidence gets cut. A placement that looks bad with low confidence gets watched. The distinction matters more than the cut threshold.
The system writes down what it did and why. When you discover you’ve cut something you shouldn’t have, you can rebuild the policy. When you discover you’ve let something run that shouldn’t have, same. The audit is what makes the system tunable.
These three properties take work to build. They also take work to operate — the team has to actually look at the audit log, actually tune the warmup windows when verticals change, actually push back when finance asks for tighter thresholds. The system is a partner, not a replacement.
What “good” looks like once it’s running
A well-tuned auto-cut layer produces a small number of cuts per week — usually somewhere between five and twenty percent of what a naive system would have cut. The placements that get cut are the ones that should have been cut. The placements that survive are the ones that should have survived.
Over ninety days of production, the metric we watch is: how many scaling winners did the system kill prematurely? On a system that’s been tuned right, the answer is zero. Not “low.” Zero.
The goal of auto-cut isn’t to cut more. It’s to cut accurately, fast. The first metric kills your team’s confidence. The second one earns it.
— AffiliateTech Engineering