AI ART Discover · Learn · Create
AI Art > Glossary > Positional Encoding
Browse Full AI Glossary
P AI Glossary Term • Term 155 of 224
Plain English Definition

What does “Positional Encoding” mean?

Core Definition

Information added to token embeddings so transformers can distinguish sequence order.

Operational Mechanism

How Positional Encoding Works

1

Token embeddings: no order info

2

Add position information

3

Model can tell 'dog bites man' from 'man bites dog'.

Real-World Scenario

Practical Example

Without positions, a transformer would treat 'dog bites man' and 'man bites dog' as the same bag of words. Adding position information to each token tells it which came first.

Core Significance

Why It Matters

Word order carries meaning, and attention by itself ignores it. Different encoding methods affect how well models handle long texts.

Expert Q&A

Common Questions About Positional Encoding

Q: Original method?

The 2017 transformer used fixed sine and cosine patterns added to embeddings.

Q: Modern alternatives?

Learned positions and relative methods such as RoPE and ALiBi.

Q: Why does it affect context length?

Models can struggle with positions longer than those seen in training.