What does “Transformer” mean?
A neural architecture that uses attention to model relationships across sequences in parallel.
How Transformer Works
Input tokens to embeddings + positions
Self-attention: tokens look at each other
Feed-forward layers
Repeat in many stacked blocks
Output predictions.
Practical Example
A transformer reads a whole sentence at once. For each word it weighs how relevant every other word is, so in 'The bank by the river' it can use 'river' to interpret 'bank'.
Why It Matters
The transformer, introduced in 2017, is the basis of most modern language models and many image and audio models.
Common Questions About Transformer
Q: Which paper introduced it?
'Attention Is All You Need', published by researchers at Google in 2017.
Q: Why did it replace recurrent networks?
It processes tokens in parallel, which trains faster on modern hardware, and handles long-range relationships well.
Q: Is a transformer only for text?
No. Variants process images, audio, video, and protein sequences.