Back to Glossary
Glossary
Confidence Level
The threshold used to determine statistical significance, typically 95%.
What is Confidence Level?
The confidence level (often 95%) is the threshold used to determine statistical significance. It's related to the significance level (α) by: Confidence = 1 - α.
At 95% confidence (α = 0.05), you're accepting a 5% chance of a false positive — declaring a winner when there isn't one.
Common Confidence Levels
| Confidence | α (Significance) | Use Case |
|---|---|---|
| 90% | 0.10 | Quick iterations, low stakes |
| 95% | 0.05 | Industry standard |
| 99% | 0.01 | High stakes, critical decisions |
Trade-offs
- Higher confidence: Fewer false positives, but need more data
- Lower confidence: Faster decisions, but more risk of being wrong