What does “Self-Supervised Learning” mean?
Learning representations from unlabeled data by predicting masked or transformed parts.
How Self-Supervised Learning Works
Unlabeled text: 'The cat sat on the ___'
Hide part of the input
Model predicts the hidden part
Check against the original, learn.
Practical Example
A language model is shown sentences with the next word hidden and learns by guessing it. The text itself supplies the answers, so nobody has to label billions of examples.
Why It Matters
Self-supervision makes use of vast unlabeled data, which is the basis of modern language and many vision models.
Common Questions About Self-Supervised Learning
Q: Self-supervised vs unsupervised?
Both avoid human labels. Self-supervised creates its own prediction task from the data, such as filling in missing parts.
Q: Which models use it?
Large language models and many image and audio models are pre-trained this way.
Q: Does it still need labels?
Not for pretraining, though later stages may use labeled examples.