What does “Confidence Interval” mean?
A statistical range that likely contains the true value of a measured model metric.
How Confidence Interval Works
A model scores 87 percent accuracy on a test set of 500 examples. A 95 percent confidence interval of roughly 84 to 90 percent shows that, with a test set this size, the true accuracy could plausibly be a few points either way.
Practical Example
A model scores 87 percent accuracy on a test set of 500 examples. A 95 percent confidence interval of roughly 84 to 90 percent shows that, with a test set this size, the true accuracy could plausibly be a few points either way.
Why It Matters
A single score can overstate precision. Intervals show how much a result might change with different test data, which helps when comparing models.
Common Questions About Confidence Interval
Q: What does 95 percent mean?
If you repeated the measurement many times, about 95 percent of such intervals would contain the true value.
Q: How do I make the interval narrower?
Use a larger test set.
Q: If two intervals overlap, are the models the same?
Overlap suggests the difference may not be reliable, though formal tests on the paired results are more precise.