What does “Loss Function” mean?
A mathematical objective that quantifies prediction error during training.
How Loss Function Works
Model predicts
Loss function scores the error
Optimizer lowers the loss
Weights update
back to the start.
Practical Example
In a house-price model, the loss might be the average squared difference between predicted and actual prices. Training tries to make that single number as small as possible.
Why It Matters
The loss defines what the model is rewarded for. A loss that does not match your real goal can lead to a model that scores well but is not useful.
Common Questions About Loss Function
Q: Is loss the same as accuracy?
No. Loss is a smooth training signal; accuracy is a human-friendly measure. They often move together but not always.
Q: What are common loss functions?
Mean squared error for numbers and cross-entropy for classes and next-token prediction.
Q: Can I design my own?
Yes, and teams often do when the standard ones do not reflect their goals.