Back to Glossary
Glossary
Expected Loss
The expected cost of choosing the wrong variant, measured in conversion rate points.
What is Expected Loss?
Expected loss (also called "risk") quantifies how much you could lose by picking a variant. It's measured in conversion rate points.
For example, an expected loss of 0.1% means if you pick this variant and it's actually worse, you'd expect to lose about 0.1 percentage points of conversion rate.
How It Works
Expected loss considers two things:
- The probability you're making the wrong choice
- How bad that wrong choice would be
Even if there's a small chance B is worse, if that "worse" is very bad, the expected loss would be high.
Risk Levels
| Expected Loss | Risk Level |
|---|---|
| < 0.1% | Low — safe to decide |
| 0.1% - 0.5% | Medium — consider waiting |
| > 0.5% | High — keep running |
Why It Matters
Expected loss helps you make risk-aware decisions. A 90% probability to win might seem good, but if the expected loss is high, you might want to wait for more data.