AI ART Discover · Learn · Create
T AI Glossary Term • Term 210 of 224
Plain English Definition

What does “Transformer” mean?

Core Definition

A neural architecture that uses attention to model relationships across sequences in parallel.

Operational Mechanism

How Transformer Works

1

Input tokens to embeddings + positions

2

Self-attention: tokens look at each other

3

Feed-forward layers

4

Repeat in many stacked blocks

5

Output predictions.

Real-World Scenario

Practical Example

A transformer reads a whole sentence at once. For each word it weighs how relevant every other word is, so in 'The bank by the river' it can use 'river' to interpret 'bank'.

Core Significance

Why It Matters

The transformer, introduced in 2017, is the basis of most modern language models and many image and audio models.

Expert Q&A

Common Questions About Transformer

Q: Which paper introduced it?

'Attention Is All You Need', published by researchers at Google in 2017.

Q: Why did it replace recurrent networks?

It processes tokens in parallel, which trains faster on modern hardware, and handles long-range relationships well.

Q: Is a transformer only for text?

No. Variants process images, audio, video, and protein sequences.