A/B testing

A/B testing compares two versions of a page or element by randomly splitting visitors between them at the same time and measuring which version produces more conversions. Version A is usually the current page (the control) and version B is the change you want to test. Because both run simultaneously with randomly assigned visitors, outside factors such as seasonality or a new ad campaign affect both equally.

A good test starts with a hypothesis based on evidence: what you will change, why you think it will help and which metric you will judge it by. Test meaningful changes, such as a new headline, a different offer or a shorter form. Tiny tweaks usually produce differences too small to detect without very large amounts of traffic.

The most common mistakes are stopping a test as soon as one version looks ahead, running it for less than a full business cycle, changing other things during the test and testing on too little traffic. Decide the sample size and duration before you start, run the test for at least one full week so weekday and weekend behaviour are included, and only then check whether the result is statistically significant. If your traffic is low, make bolder changes and accept that some questions are better answered by research and customer feedback than by testing.

How to improve it

• Write a hypothesis for every test: the change, the reason and the metric.
• Decide your sample size and test duration before launching, and do not stop early.
• Run tests for at least one full week to cover differences between weekdays and weekends.
• Check results with the free A/B test significance calculator (open tool).
• On low-traffic pages, test larger changes such as the offer or headline rather than colours.

How PageScore checks it

PageScore does not run A/B tests. Its ranked list of fixes gives you a prioritised source of test ideas, and you can scan each variation's URL to compare how they score.