AI ART Discover · Learn · Create
Q AI Glossary Term • Term 166 of 224
Plain English Definition

What does “Quantization” mean?

Core Definition

Converting model weights to lower precision formats such as 8-bit or 4-bit.

Operational Mechanism

How Quantization Works

1

Weights stored as 16-bit numbers

2

Convert to 8-bit or 4-bit

3

Smaller file, less memory, faster.

Real-World Scenario

Practical Example

A model file that is about 14 GB at 16-bit precision can shrink to roughly a quarter of that at 4-bit, so it can run on a laptop. Outputs may be slightly different from the original.

Core Significance

Why It Matters

Quantization makes it practical to run models on cheaper or local hardware. Testing quality on your own tasks is still important.

Expert Q&A

Common Questions About Quantization

Q: What do 4-bit and 8-bit mean?

How many bits are used to store each number. Fewer bits mean less memory and some loss of precision.

Q: Is it reversible?

Not exactly. Original precision cannot be perfectly recovered from the quantized version.

Q: Does it work for training too?

Mostly used for inference, though methods like QLoRA combine it with fine-tuning.

Conceptual Cross-References

Related AI Terminology