AI ART Discover · Learn · Create
I AI Glossary Term • Term 104 of 224
Plain English Definition

What does “Inference” mean?

Core Definition

The runtime phase where a trained model generates predictions or outputs.

Operational Mechanism

How Inference Works

1

Trained model, weights frozen

2

New input arrives

3

One forward pass or token-by-token generation

4

Prediction or reply.

Real-World Scenario

Practical Example

When you send a message to a chatbot, the provider runs inference: the already-trained model reads your message and generates a reply. No learning happens during that step.

Core Significance

Why It Matters

Inference is the part users pay for and wait on. Its speed and cost affect product pricing, latency, and energy use.

Expert Q&A

Common Questions About Inference

Q: How is inference different from training?

Training adjusts the weights using data. Inference uses fixed weights to produce outputs.

Q: Why can inference be expensive?

Large models need substantial memory and compute for each request, and popular services handle huge numbers of requests.

Q: Can inference run on a phone?

Smaller or compressed models can, which is the aim of on-device AI.