A/B Testing & CRO
This section covers the statistics behind conversion testing: what statistical significance, p-values, and confidence actually mean, how to size and read an A/B test, the peeking problem, frequentist vs Bayesian methods, and statistical vs practical significance. Each topic is paired with a free A/B significance calculator.
Guides in this section
- What Is Statistical Significance in A/B Testing?
- What Is a P-Value?
- How to Interpret A/B Test Results
- What Is Statistical Confidence (95% Confidence)?
- How Big Should My A/B Test Sample Be?
- Frequentist vs Bayesian A/B Testing
- What Is the Peeking Problem in A/B Testing?
- One-Tailed vs Two-Tailed Tests
- A/B Test Significance Calculator: How to Use It
- Statistical vs Practical Significance
A/B Test Significance CalculatorPaste two conversion rates and get a clear ship-it or keep-testing verdict.
Open the tool →
Why it matters
A/B testing only earns its keep when the statistics are trustworthy. A lift that looks impressive on a dashboard can vanish once you account for sample size, random variation, and how many times you checked the result. Reading a test correctly is what separates a real conversion-rate win from a coin flip you happened to call right. The guides above give you the mental models (significance, confidence, p-values, sample size, and the peeking problem) so you can decide with confidence whether a change is worth shipping, and avoid declaring victory on noise.
Last updated: 14 June 2026