What does “Evaluation Set” mean?
A held-out dataset used to measure model quality after training.
How Evaluation Set Works
Training data: model learns from this
Validation data: tune settings
Evaluation (test) data: final exam, used once.
Practical Example
A team sets aside 10% of its labeled data before training and never lets the model learn from it. After training is finished, they measure accuracy on that held-out set to get an honest estimate of performance on new data.
Why It Matters
Testing on data the model has already seen overstates how good it is. A clean evaluation set tells you whether it will work in the real world.
Common Questions About Evaluation Set
Q: Is an evaluation set the same as a test set?
Usually yes. Some teams use the terms differently, with evaluation meaning any held-out data, including validation.
Q: How big should it be?
Large enough that scores are stable and cover the cases you care about. Rare cases may need deliberate extra coverage.
Q: What ruins an evaluation set?
Leakage: if examples from it appear in training, scores become inflated.