What does “LoRA (Low-Rank Adaptation)” mean?
A parameter-efficient fine-tuning method that adds low-rank adapter matrices.
How LoRA (Low-Rank Adaptation) Works
Frozen pre-trained weights, unchanged
Small trainable adapter matrices, only these learn
Combined at run time: adapted model.
Practical Example
Instead of updating all billions of weights in a language model, a team trains small adapter matrices for a customer-service style. The adapter file is small, and the same base model can swap different adapters for different clients.
Why It Matters
LoRA makes fine-tuning possible on modest hardware and makes adapters easy to store and share.
Common Questions About LoRA (Low-Rank Adaptation)
Q: What does low-rank mean?
The weight changes are represented by two thin matrices multiplied together, which needs far fewer numbers than a full update.
Q: Does LoRA match full fine-tuning?
Often close on many tasks, though full fine-tuning may do better on some.
Q: Can adapters be combined?
Sometimes, with care. They can be merged into the base weights or swapped depending on the task.