Learning without training: The implicit dynamics of in-context learning

0:00Welcome to the Deep Dive. Today, we're going to try and get our heads around maybe the most striking feature of modern large language models. This thing called in-context learning or ICL. You've probably seen it. You give an LLM maybe two or three examples right there in the prompt, like showing it how you want something formatted or maybe a few examples of summarizing a legal doc. And then, bam, it just picks it up. It adopts that new pattern perfectly. It's ICL. It's this ability the model has after it's been trained, after its weights are supposed to be frozen solid. Yeah, set in stone. Exactly.

0:35Yeah. It suddenly learns new patterns, new behaviors just from those examples in the context window. And the really weird part, there's no explicit weight update, no retraining. Right. It all happens at inference time, which for ages has felt, well, frankly, a bit like magic. It really has. I mean, we've all been using it, this incredible capability that just sort of emerged. But how it works, that's been a bit abstract, hasn't it? So our mission today is to try and move past that mystery. We're going to unpack some really interesting recent research that actually proposes a concrete mechanism, you know, something mathematically solid for ICL.

1:15And we're going to argue that this apparent magic, well, it's actually more like a very clever, very structured, implicit training session happening under the hood. Implicit training. So we're going to explore how the context those examples you feed it is somehow secretly modifying the LLM's internal machinery, even though it's supposed to be fixed. Hmm. Let's start with that core conflict, right? Because normally in machine learning, learning means changing things. Absolutely. Learning is optimization. You adjust weights, you save them. It's a dynamic process. But ICL seems to just break that rule because when you're just using the model, those weights are static.

1:52They shouldn't be changing. And that paradox, that's what really fueled the whole debate early on. People were asking, you know, is this even real learning? Right. Or is it just remembering? Exactly. Maybe the model isn't learning anything truly new. Maybe it's just retrieving stuff it already learned during pre-training. Like, um, the prompt just helps it figure out which existing skill to use. Bayesian conditioning, some called it. Makes sense. And for a bit, that retrieval idea seemed plausible. There were experiments, especially with smaller models, where even if you put random labels in the prompt examples...

2:23Like nonsense answers. Yeah, total nonsense. The models still did okay on the task, which suggested they were relying more on the structure, the patterns they already knew, rather than actually learning the new mapping from the examples. But then the models got bigger. Much bigger. And that's when things shifted. Counter-evidence started popping up. It showed that, okay, maybe small models just retrieve. But the really large models, they seem to genuinely learn, even from those random labels. They can generalize from nonsense. In some cases, yeah. They learn tasks they definitely hadn't seen in pre-training, which points towards, you know, actual emergent learning happening right there in the context window.

3:00Okay, so that forces the issue. If it is learning, but there's no explicit weight update. then there must be an implicit one somewhere, somehow. So the hunt was on. Where inside the transformer architecture could this hidden optimization be happening? So if we accept it's implicit, we have to look at the basic building block, right? The transformer layer. The self-attention plus the MLP part. Exactly. Your standard transformer block, self-attention stacked with a multi-layer perceptron, the MLP. So this new research, it generalizes that whole unit. They call it the contextual block. Contextual block.

3:36Okay. And the core idea, the real theoretical leap, focuses on that self-attention layer or actually any layer that processes the context sequence. So it's not just about attention. Well, the argument is that this layer, the contextual layer, its job isn't just figuring out how tokens relate to each other. In the context of ICL, its main role is to let the transformer implicitly modify the weights of the MLP layer. Ah, okay. So the attention layer reads the instructions in the prompt. The examples. And then it sort of writes a temporary patch for the MLP that follows it. That's a really good way to put it.

4:09A temporary software patch. The contextual layer takes the context, the examples, and transforms them mathematically into a change for the MLP's weights. Let's call this change the delta weight. Delta weight. Got it. And this is where it gets really neat mathematically. They actually provide an explicit formula. It shows exactly how the context translates into this delta weight. Wow. Okay. But the most surprising bit, this delta weight matrix, the modification, it turns out to be specifically rank one. Rank one. Okay, I remember enough linear algebra to know that rank is important, but unpack that for us.

4:45What does rank one mean here, practically? Why is that significant? Yeah, it's super important. It basically tells you about the efficiency and the nature of the change. A typical weight matrix in these models is huge, right? Billions of parameters, potentially. A high rank update would be like completely changing the whole structure. A major renovation. Exactly. But a rank one update, that's the absolute simplest, most minimal way to induce a structural change in a matrix. Think of it like instead of rebuilding the whole engine, you're just adjusting one very specific, very critical screw. OK, so it's a highly targeted, minimal change.

5:21Minimal, targeted, and computationally very efficient to represent and presumably to compute implicitly. So if the change is that minimal and targeted, does that immediately suggest something about how we should write prompts if we know we're triggering just a rank one update? I think it absolutely does. It sort of shifts the goalposts for prompt engineering, doesn't it? Instead of just thinking about giving relevant examples, maybe we should be thinking about crafting examples that trigger the cleanest, most effective rank one update for the task. Moving from descriptive prompting to maybe mathematically prescriptive prompting?

5:57Potentially, yeah. And another interesting point, this theory seems pretty general. It holds up even if you swap out the self-attention layer for other kinds of contextual layers, like maybe from older RNN architecture. Really? Yeah. It suggests the core mechanism isn't just about attention. It's something more fundamental about how these layered networks can translate input context into changes in their own weight structure. That's fascinating. And what about skip connections? Pretty much all modern transformers have those. How do they fit into this delta weight picture? Good question. They're everywhere, right?

6:30The researchers looked at that too. They found that when you have skip connections, the context update this delta weight process. It doesn't just hit the first layer weight matrix of the MLP. It also affects the bias term in the last layer of the MLP. So the prompt isn't just making one tiny tweak. It's causing this coordinated change across a couple of different parameters at once. It makes the implicit fine-tuning even more functionally effective. Wow. Okay, so that's the static picture. Context becomes a rank one delta weight. But problems aren't static, are they? You read them token by token.

7:02Exactly. And that brings us to the dynamics. Since the context is a sequence, the model processes it step by step, token by token. And this creates what the researchers call an implicit learning dynamics. Okay, implicit dynamics. How does the sequence part change the delta weight idea? Does it build up? Yeah, essentially, it turns that static calculation into an iterative process. And here's the really cool finding. This sequential process, these iterative updates to the implicit weights, they can be shown mathematically to be equivalent to a series of stochastic gradient updates. Whoa, wait, so processing the prompt is like running gradient descent.

7:41It looks remarkably like it. The examples in your prompt are basically acting like mini-batch data points, and the model seems to be performing online gradient descent, optimizing some kind of internal loss function defined by the context itself. That is wild. It's like a tiny training loop running inside the inference pass. But hang on, if it's doing gradient descent, even implicitly, does that mean a bad example in the prompt could mess things up? Like cause catastrophic forgetting, but just for that one inference run? That's a really sharp question. And the low rank nature, the rank one aspect is probably key here.

8:15Because the change is so minimal, so constrained rank one, it's unlikely to completely trash the model's existing knowledge. It's more like creating a very specific temporary specialization rather than doing a full destructive update. It minimizes the risk. Okay, that makes sense. And if it is like gradient descent, it should converge, right? The updates should get smaller as it processes more context. Exactly. Optimization needs to converge. And they tested this. They actually tracked the size, the magnitude of these implicit gradient updates as the model processed more and more tokens from the context.

8:49And they found that the updates did get smaller, they decreased, and crucially, they essentially vanished as the model got closer to processing the full context. That pattern, the vanishing gradient, is like the mathematical fingerprint of a converging gradient descent process. Wow. Okay, that's compelling theoretical evidence. But theory is one thing. Yeah. How did they actually prove this experimentally? You mentioned the side-by-side test. Right, the verification. They set up a controlled experiment. They trained a transformer specifically on the task of learning simple linear functions, just from examples given in the context.

9:22Linear functions? Yeah. Like y equals mx plus b. Why something so simple? Why not, I don't know, learning Shakespearean sonnets in context? Because they needed mathematical clarity. Linear functions are simple, the ground truth is perfectly known, it gives you a clean environment to test the mechanism without other complexities muddying the waters. Right, isolate the variable. Precisely. If they tried poetry, comparing the output to the theoretical delta weight formula would be almost impossible. With linear functions, they could check if the math lined up perfectly. Okay, so what was the crucial test?

9:56The core test was this. They compared two things. First, the normal output of the model when given the prompt with example standard ICL. Second, the output of the model without the prompt, but where they manually changed the MLP weights using their exact delta weight formula derived from that prompt. Ah, so does the formula perfectly predict the effect of the prompt? That was the question. The ultimate stress test, as you said. And the result? It was pretty definitive. They looked at the validation loss curves, how well the model performed the linear function task for both methods, the curve for standard ICL and the curve for the explicitly modified weights using the delta W formula.

10:34They were in remarkable agreement, as the paper puts it. Wow. It tracked almost perfectly throughout the learning process. That's strong quantitative proof. The formula really does capture how the prompt information gets baked into the MLP weights. Okay, it's huge. And they went one step further. They also compared the performance of this implicit delta weight update against actual explicit fine-tuning, you know, running standard gradient descent on the MLP using the same prompt examples as training data. How did that compare? Interestingly, the final results weren't numerically identical, which you might expect since one is a constrained rank 1 update and the other is standard SGD.

11:11But crucially, both methods clearly minimize the task loss in very similar ways. It reinforces the idea that ICL isn't just pattern matching. It's a genuine, albeit constrained, form of optimization. Okay, let's try and summarize this then. It feels like we've made a big leap. We've gone from ICL being this kind of mysterious emergent thing to having a pretty rigorous theory. This contextual block model shows how the prompt context gets mathematically translated into a specific minimal rank one change to the MLP weights inside the transformer. Mm-hmm. A delta weight. And this whole process acts like an implicit, low-rank fine-tuning session, driven by something that looks a lot like online stochastic gradient descent.

11:56It's not magic. It's math. It's a huge conceptual step forward, definitely. But, you know, we should also be clear about the limits here. This analysis is mostly worked out for a single transformer block. Right. Not the whole stack of dozens of layers interacting. Exactly. And also importantly, the math right now really only fully explains the effect on the very first token the model generates after the prompt. Ah, not the whole sequence it generates in a conversation. Not yet. So extending this to deep models and multi-step generation is still future work. Okay, noted. But even with those boundaries, this feels like a major piece of the puzzle, right?

12:33Demystifying how these incredible abilities just pop up at inference time, we're starting to get a handle on the mechanism. Absolutely. And it leads to a really provocative thought, maybe something for you to mull over. If we now understand mathematically that the prompt context is literally performing an implicit weight update. A specific rank one update to the MLP. Yeah. Then maybe we should stop thinking of prompts purely as instructions or examples. Maybe we should start thinking of them as well as parameters themselves. Parameters. How do you mean? Well, if the prompt causes a specific weight change, can we engineer the prompt not just based on intuition, but based on the math?

13:09Can we design prompts specifically to trigger the optimal low-rank delta weight for a very specific task? Huh. Calculating the mathematically perfect prompt. Exactly. Imagine deliberately crafting a prompt sequence designed to induce the most effective possible rank one fine-tuning for, say, medical diagnosis or legal contract review, making the LLM hyper-specialized just through the context. The era of mathematically optimized prompting. That definitely changes how we think about interacting with these models, and maybe even how we design them in the first place. Something to chew on. Thanks for joining us on this deep dive.