Back to Glossary
Glossary
Probability to Win
The likelihood that a variant has a higher true conversion rate than the other variant.
What is Probability to Win?
Probability to Win, often written as P(B > A), is the core metric in Bayesian A/B testing. It tells you the probability that Variant B has a higher true conversion rate than Variant A.
How to Interpret It
| P(B wins) | Interpretation |
|---|---|
| ≥ 95% | Confident B is better — safe to ship |
| 70-95% | B is likely better — keep running for more confidence |
| 30-70% | Uncertain — no clear winner yet |
| 5-30% | A is likely better — keep running |
| ≤ 5% | Confident A is better — keep original |
Why 95%?
The 95% threshold is a convention that balances confidence with practicality. At 95%, there's only a 5% chance you're making the wrong decision.
For high-stakes decisions, you might want 99%. For quick iterations, 90% might be acceptable.