A test backlog that ranks itself.
Impact, ease, and evidence combined
Analytics, surveys, behavioral data
No more loudest-voice testing
The problem.
Experiment ideas came from everywhere, web analytics, the post-purchase survey, heatmaps, and plain opinion, and got prioritized by whoever made the case most forcefully. There was no consistent way to say which test deserved to run first, so the roadmap followed conviction instead of expected value.
The approach.
I built a custom prioritization framework that scores every proposed experiment on impact and effort: whether the change sits above the fold at the most common resolution, the nature of the change (adding beats changing beats removing), the target page's share of traffic, and, critically, whether the idea is backed by analytics data, by survey and qualitative evidence, or by behavioral data like heatmaps and recordings, plus its closeness to the decision point and the design and dev effort to ship it. Every input rolls up into a single priority score.
The outcome.
The test backlog got a ranking the whole team could trust, driven by evidence weight and expected impact, not the loudest voice in the room. Experiments that were close to the money, well-evidenced, and cheap to ship rose to the top automatically, and the survey work fed straight into the qualitative-justification score.
If your experiment roadmap is set by whoever argues hardest, I'll give you a scoring model that ranks the backlog on impact and evidence. One slot open for Q3 2026.