Instruction tuning, fine-tuning, and alignment
Fine-tuning such large models is potentially very expensive. In classical fine-tuning, the idea is to fit the weights of a model for a task or a new domain. Even if it is a slight update of the weights for a few steps, for a model of more than 100 billion parameters, this means having large hardware infrastructure and significant costs. So, we need a method that allows us to have efficient and low-cost fine-tuning and preferentially keeping the model weights frozen.
The intrinsic rank hypothesis suggests that we can capture significant changes that occur in a neural network using a lower-dimensional representation. In the case of fine-tuning, the model weights after fine-tuning can be defined in this way:
∆W represents the update of the weights during fine-tuning. For the intrinsic rank hypothesis, not all of these elements of ∆W are important, and instead, we can represent...
