LoRA (Low-Rank Adaptation) is an efficient fine-tuning technique that avoids modifying all of a model's parameters. Instead of retraining the original billions of weights, it freezes the base model and adds small low-rank matrices that are trained separately. The result is a lightweight adaptation file (often just a few megabytes) that is applied to the model without altering it.
QLoRA takes this idea further by combining it with quantization: the base model is loaded in reduced precision (usually 4-bit), which drastically decreases memory consumption. This allows for fine-tuning large models on a single consumer GPU.
Its importance is practical and economic:
- They reduce hardware costs and training time.
- They allow for maintaining several specialized adapters for the same base model.
- They facilitate sharing and distributing adaptations without moving the entire model.
A nuance: these techniques are ideal for specializing a model in specific tasks or styles, but they do not usually replace full training when seeking to incorporate deeply new knowledge.