Explaining LLM Model Weights to 10 year old(LLaMA and Vicuna)

LLaMA and Vicuna are like two smart robots that can talk and write like humans. They are made of something called a “neural network,” which sounds fancy but it just means they have a lot of tiny parts that work together to make choices. These choices help the robots figure out and create sentences based on the words they see..

Now, these tiny parts are called “weights.” You can think of weights like the power of connections between different parts of the robot’s brain. The more power, the more impact it has on the final choice. For example, imagine you and your friends are choosing where to go for some yummy ice cream. Each friend has a different idea, and you weigh their ideas based on how much you like them. The liking you have for your friends is like the weights in the robot’s brain.

Since LLaMA and Vicuna are transformer-based language models, they are built to read and understand text in a way that is similar to how we humans do. They pay attention to the order of words, grammar, and context to create meaningful sentences. For example, if you give LLaMA a sentence like “The cat is on the ___,” it will try to fill in the blank by picking a word that makes sense, like “mat” or “roof.” Or maybe “hat,” if it’s feeling silly.

We can use something called “delta weights” to turn LLaMA into Vicuna. Delta weights are like small changes we make to the connections in LLaMA’s brain to create a new robot, Vicuna, that works slightly differently. It’s like changing your liking for your friends to make better choices about where to go for lunch. By using these delta weights, we can create a new robot that might be better at talking and writing than LLaMA. Or maybe worse, if we mess up.