What does “Decision Boundary” mean?
The surface in feature space that separates classes predicted by a classifier.
How Decision Boundary Works
Decision Boundary comparing Class A side (Points on this side get label A) versus Class B side (Points on this side get label B).
Practical Example
Imagine plotting emails by two features, such as number of links and number of exclamation marks. A classifier draws a line or curve through that plot; emails on one side are called spam and on the other not spam. Points close to the line are the uncertain ones.
Why It Matters
Looking at the boundary shows where a model is confident and where it is guessing. Complex boundaries can fit training data well but fail on new examples.
Common Questions About Decision Boundary
Q: Is a decision boundary always a straight line?
No. Linear models give straight lines or flat planes, while neural networks and trees can create curved or blocky boundaries.
Q: Can we see boundaries with many features?
Not directly, since they live in many dimensions. People plot them in two dimensions using a few features or a projection.
Q: What does an overly wiggly boundary mean?
It may indicate overfitting, where the model has bent around noise in the training data.