AI ART Discover · Learn · Create
M AI Glossary Term • Term 128 of 224
Plain English Definition

What does “Mid-training” mean?

Core Definition

An intermediate training phase between pretraining and post-training, often used for capability or domain adjustments.

Operational Mechanism

How Mid-training Works

1

Pretraining: broad data

2

Mid-training: targeted data such as code, math, long documents

3

Post-training: instructions and preferences

4

Released model.

Real-World Scenario

Practical Example

After broad pretraining, a lab continues training on a higher-quality mix with more code and math, and on longer documents to extend the context length, before the final instruction-tuning stage.

Core Significance

Why It Matters

It lets developers strengthen particular skills without repeating the full pretraining. The term is used loosely, and boundaries between stages vary by lab.

Expert Q&A

Common Questions About Mid-training

Q: Is it a standard term?

It is informal and used differently across labs.

Q: Why add this stage?

To improve specific capabilities or extend context length at lower cost than starting over.

Q: How is it different from fine-tuning?

It is usually larger scale and still focused on general capabilities, while fine-tuning typically targets a specific task or behavior.