What does “Label Smoothing” mean?
A regularization method that softens hard labels to improve generalization.
How Label Smoothing Works
Label Smoothing comparing Hard labels (cat = 1.0, dog = 0.0, bird = 0.0) with Smoothed labels (cat = 0.9, dog = 0.05, bird = 0.05).
Practical Example
Instead of telling a classifier the right class has probability 1 and every other class 0, training uses 0.9 and spreads 0.1 among the others. This stops the model from becoming extremely overconfident.
Why It Matters
Smoothing is a simple regularizer that can improve generalization and calibration. It is a small change with often useful effects.
Common Questions About Label Smoothing
Q: Does it always improve accuracy?
Not always, but it often helps slightly, especially in large classifiers.
Q: Does it change what the model predicts?
It changes training targets, so the final probabilities are softer and the model is less extreme about its confidence.
Q: Are there downsides?
It can reduce the usefulness of the model's outputs for distillation in some cases, because softened probabilities carry less information.