A/B Testing on a Small Site: Why You Probably Don't Have the Traffic Yet

A/B testing gets requested by founders who've read about how Amazon or Booking.com tests everything, without the traffic volume that makes those tests statistically meaningful. Running a test on 200 monthly visitors and declaring a winner after a week is closer to reading tea leaves than doing science, even with a testing tool that produces a confident-looking percentage.

The traffic math nobody mentions

Detecting a real, meaningful difference between two versions of a page requires a specific, calculable sample size, and for most small or early-stage sites, reaching that sample size would take months — long enough that other things (seasonality, marketing changes, the product itself) will have shifted enough to invalidate the comparison anyway.

What actually works instead, at low traffic

• Qualitative feedback — watching a handful of real users attempt a task tells you more at low volume than a quantitative test with too little data to trust.
• Sequential changes with clear before-and-after tracking, rather than simultaneous variants competing for a small pool of visitors.
• Focusing changes on the parts of the funnel proven most influential elsewhere — checkout friction, page speed, clear calls to action — rather than testing minor color and copy variations that need enormous sample sizes to detect.

When A/B testing genuinely becomes worth it

Once traffic is high enough to reach significance in a reasonable timeframe — usually a threshold much higher than founders assume — it becomes a genuinely valuable tool, the same kind of evidence-driven decision-making behind deciding what actually gets built.

A/B testing without enough traffic doesn't give you an answer. It gives you a random number that happens to look like one.

Until you're there, the more reliable path is watching real users struggle with real tasks, and fixing what you see — cheaper, faster, and honestly more useful than a test with too little data behind it.