Page Speed OptimisationAboutContact
PILLAR
A/B testing is the discipline of comparing two variants of a webpage or interaction against a control to determine which produces a higher conversion rate at statistical significance. GoGoChimp operates at the 99 Rule — 99% statistical confidence with a 14-day minimum run length and 1,000 conversions per variant — versus the industry-standard 95% threshold which produces winners that reverse 40–60% of the time at 90 days.
DEFINITION
A/B testing is the practice of running two or more versions of a webpage, email, or feature in parallel to a randomly split audience and picking the version that produces more conversions. It is the closest thing to an experimental method the web has. The difference between amateur A/B testing (winner reverses 40-60% of the time at 90 days) and disciplined A/B testing (reverses 3-8%) is confidence level, sample-size discipline, and downstream-metric validation.
A/B testing is also called split testing, controlled experiments, split-URL testing, or conversion testing. Multivariate testing (MVT) is a related but separate discipline testing element-level combinations. Frameworks: the 99 Rule (99% confidence standard), the Evidence Stack (four-layer hypothesis prioritisation), ICE / PIE scoring. Leading practitioner voices: Ronny Kohavi (former Bing, wrote the peeking-problem paper), Peep Laja (CXL Institute), Andrew Anderson (Recipe for CRO), Optimizely and Statsig engineering teams.
FIRST-PARTY RECEIPTS
Every framework, benchmark, and playbook you will read on this page is grounded in 13 years of running A/B tests inside operator-led CRO programmes.
99%
Statistical confidence GoGoChimp requires (vs 95% industry standard) — the 99 Rule
3-8%
Winner reversal rate at 90 days when following the 99 Rule and Evidence Stack
40-60%
Winner reversal rate at 90 days for amateur A/B testing (peeking, under-powered, no downstream check)
30+
Experiments per quarter on Scale-tier engagements (vs 2-5 amateur, 8-15 standard)
4-7 → 28-34%
Self-serve AI ceiling vs operator-led lift on the same tooling (the 4-to-34 Gap)
13 yrs
A/B testing operator experience, Glasgow-based, founded 2013
THE CLUSTER
The full stack: from foundations through statistical rigour to tool selection.
FOUNDATIONS
The full grounding: statistical basis, history from R.A. Fisher to Kohavi, and why message match is the #1 amateur miss.
TOOLS
VWO, Optimizely, Statsig, GrowthBook, PostHog ranked by use case. Free tiers, mid-market, enterprise.
COMPARISON
The two dominant testing platforms head-to-head: pricing, statistical engines, personalisation, integrations, support.
FRAMEWORK
Why ICE scoring under-performs at scale. The four-layer Evidence Stack that produces 30+ tests per quarter without ship-fatigue.
DEEP CONCEPT
Why iterative A/B testing traps you on hill-tops the visitors never asked to climb. When to burn the page down and start again.
TACTIC
The CTA copy, colour, size, position, and contrast tests that move conversion, with the ones that never do.
TEARDOWN
Seven real landing pages torn down for CRO fundamentals. The tests each one should run first, and why.
TRAINING
Where else to learn CRO if CXL isn't the fit. Institute alternatives, community options, and the free-tier learning path.
LANDING
Message match, hero, social proof, CTA framework. The pre-test baseline every A/B test should start from.
TOOLS
Heatmaps generate hypotheses, they do not validate them. The tool stack that feeds your A/B testing pipeline.
FAQ
A/B testing is the practice of running two or more versions of a webpage, email, or product feature in parallel to a randomly split audience, then picking the version that produces more conversions. It is the closest thing to controlled experimentation the web has. Statistical rigour matters: the difference between amateur A/B testing (winner reverses 40-60% of the time at 90 days) and disciplined A/B testing (reverses 3-8%) is confidence level, sample-size discipline, and downstream-metric validation.
GoGoChimp requires 99%, the 99 Rule. Industry standard is 95%. The difference matters more than it sounds: 95% confidence means 1-in-20 winners you ship will reverse when they hit production traffic. At 99% it is 1-in-100. On a 30-tests-per-quarter Scale programme, that is the difference between shipping 1.5 fake winners per quarter versus 0.3, which compounds into wasted engineering time and misdirected roadmap.
Until you hit the pre-calculated sample size at your target confidence level. Not until it looks significant. Peeking at a test daily and stopping the moment it crosses 95% roughly doubles your false-positive rate (Optimizely 2015 peeking-problem study). Practical rule: minimum 14 days to cover a full business cycle, and minimum sample size hit. If your traffic is under 10,000 sessions per month, run tests for 4-6 weeks, not 2.
Depends on baseline conversion rate, minimum detectable effect, and confidence level. Rough anchors at 99% confidence and 80% power: 5% baseline detecting a 10% relative lift needs ~30,000 sessions per variant; 2% baseline detecting 15% lift needs ~40,000 per variant. Use a sample-size calculator (Evan Miller, Statsig) with your actual numbers. Under-powered tests are the single most common cause of false-positive winners in mid-market SaaS + ecommerce.
A/B tests compare complete versions (Variant A vs Variant B). Multivariate tests (MVT) test combinations of individual elements (headline + button colour + hero image, 8 permutations). MVT needs 5-10x the traffic of a simple A/B for the same statistical power, and is only worth running when you already know the direction of each variable independently. For most mid-market sites, sequential A/B tests beat MVT on speed and reliability.
2-5 for amateur programmes. 8-15 for standard programmes. 30+ on Scale-tier operator-led AI CRO programmes. The velocity gap is not tool cost, it is hypothesis quality and prioritisation discipline. Most teams get stuck at 8-15 because they run out of well-formed hypotheses. The Evidence Stack framework (four-layer hypothesis prioritisation) is how we sustain 30+ per quarter without ship-fatigue.
Statsig for engineering-led SaaS (feature flags plus experimentation, generous free tier). VWO for marketing-led (visual editor, mid-market pricing). Optimizely for enterprise (compliance, personalisation, dedicated support). GrowthBook for open-source teams. PostHog for product analytics plus experimentation combined. Full teardown in our best A/B testing tools 2026 guide and VWO vs Optimizely 2026 head-to-head.
Four common causes: (1) under-powered tests declared significant on false-positive noise, (2) novelty effect (variant wins for 2 weeks then fades as users acclimatise), (3) segment interaction (variant wins on desktop, loses on mobile, blended average was significant), (4) upstream-traffic-mix change post-launch (paid channel added new segments the test never saw). The Evidence Stack catches all four before ship, which is why 99 Rule tests reverse at 3-8% vs 40-60% for amateur.
GET STARTED
Book a 15-minute A/B testing audit. We will show you your current false-positive risk, which of your last three winners is likely to reverse at 90 days, and how the 99 Rule cuts reversal from 40-60% down to 3-8%.
Book my free CRO audit →
A 60-second AI scan shows which page-speed issues are leaking conversions on your homepage, and the £/month each one is costing your revenue.
✓ Built on Build Grow Scale's 347-store CRO research
✓ Avg 28-34% lift (expert-led AI CRO benchmark)
✓ Free, no signup