


17 Jun 2025 · 12m · 6 chapters

“Extrapolation by association” for length generalization in transformers—how co-training on a related auxiliary task with longer inputs lets a model extrapolate to longer lengths on a main task it only saw at short lengths.
No specific podcast guests are named; the episode is presented as a discussion of a research paper.
Not applicable (no guests identified).
(1) Co-training main+auxiliary tasks enables length extrapolation; training tasks separately fails. (2) Transfer requires task relatedness; unrelated co-training yields no benefit. (3) Pre-training can create reusable “scaffolding” that improves later length generalization. (4) Mechanism: shared internal computation—reused attention heads—and positional encoding (RoPE) correlates with transfer.
reverse add with auxiliaries (no carry, carry only, reverse subtract, NA3-co-t multiply); string copy/capitalized tasks with MQAR/capitalized reverse; maze DFS trace with shortest path; controls like reverse add with copy-first-op and string copy with reverse string.
Written by AI. May contain mistakes.
Exploring how AI models generalize knowledge to tackle unseen challenges.
“And the core question they're asking is, well, it's pretty neat.”
Discussing the methodology of co-training main and auxiliary tasks.
“The paper has some great visualizations.”
Reviewing the striking outcomes of co-training on length generalization.
“So the million-dollar question, did it work?”
Analyzing the role of pre-training in enabling length generalization.
“Its accuracy on reverse add without fine-tuning was basically zero.”
Examining how positional encoding and attention heads contribute to transfer.
“Okay, this is starting to sound a bit like magic.”
Discussing the broader implications of length generalization for AI development.
“Even at lengths, it's never seen for that main task before.”

Get the gist, explore an idea or ask about something said.
On Best AI papers explained, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00Okay, let's dig into this. How do these incredibly advanced AI models, you know, the ones driving things like ChatGPGPT, actually learn to tackle problems they've never encountered. especially when those problems are way longer or more complex than their training examples. Today, we're doing a deep dive into some fascinating new research that reveals a pretty surprising way these transformer models generalize. They call it extrapolation by association. Think of it like a shortcut to knowledge. Learning a skill for one task can sort of instantly unlock it for a related but totally new challenge.
0:33Our main source here is a paper called Extrapolation by Association, length generalization transfer in transformers. It's got researchers from UW-Madison, CMU, University of Michigan, and Microsoft research behind it. And the core question they're asking is, well, it's pretty neat. Can a model trained on a long version of one task transfer that ability, that length generalization, to a different but related task it only ever saw short examples of? Yeah, and it's fascinating because it challenges some assumptions. You know, when we talk about generalization in AI, we mean applying learned skills to new stuff.
1:04But the real acid test is out of distribution or O generalization. Can it handle inputs that look really different from training data, like much, much longer? Right. So you train it to add, say, two-digit numbers. Can it then suddenly handle 10-digit numbers without specific training on those longer ones? That's the big leap, isn't it? Precisely. And that's length generalization specifically. Can it extrapolate beyond the length it was trained on? This paper introduces a, well, a new mechanism for this. extrapolation by association. The hypothesis basically is that related skills can be shared or leveraged.
1:40They propose trading a main task on short inputs and an auxiliary task, a helper task on longer inputs, and the crucial bit, trading them together. Yeah. Jointly. Okay, so the idea is teach the model one complex task at a long length, and it might just figure out how to do a related task, one it only learned on short examples, at that same long length, even without seeing it before. That's exactly the core idea. The main task sort of inherits the generalization range, the length capability from the auxiliary task it was trained alongside. The paper has some great visualizations. Figure one, for instance, showing how training them separately just doesn't cut it.
2:15The model fails. Right. But train them together and boom, you get this transfer. Okay. That sounds promising. So how did they actually test this? You can't just, you know, hope it happens with web text. Right, exactly. They needed control. So they use what they call synthetic algorithmic tasks, basically simplified controlled problems. This lets them really see what the model is learning and how. Makes sense. Keep it clean. Precisely. They pick tasks representing similar kinds of algorithms. First up, arithmetic tasks. Their main one was reverse add. You add two numbers, but the digits are given in reverse order.
2:47So 56 plus 78 becomes like 65 plus 87 equals. It's a standard benchmark for length generalization, actually. Then they had auxiliary tasks like no carry, which is just digit-wise, addition, mod 10, simple stuff. Or carry only, which just identifies where the carries happen in addition. Also reverse subtract and even a multiplication task, knee 3 TT multiply, which multiplies by three using chain of thought steps. And for all of these, length was straightforward, just the maximum number of digits and the numbers being added or subtracted. Gotcha. Clear definition for length there. What about other types of problems?
3:20Did they look beyond just numbers? Well, yeah. They moved on to string manipulation tasks, working with sequences of characters, things like string copy, just duplicating the input, multi-query associative recall or MQAR, that's about finding the next character given a piece of the string. Also reverse string, capitalized string, and capitalized rest, which does both. And here, length was, again, pretty simple, just the number of characters in the input string. Okay, arithmetic, strings. What else? You mentioned three categories. The third one was maze solving tasks, which, as you said, sound a bit more involved.
3:54They use DFS trace, which is simulating a depth first search through a maze and shortest path, finding the best route. The mazes were given as lists of connected nodes, basically, like node 1 connects to 5, node 5 connects to 8, and so on. And length here was the total number of nodes in the maze graph. They did something clever, kept the grid size fixed, say 8x8, but varied the number of active nodes within it, like 16, 32, or 64 nodes. So longer length met a more complex maze structure on the same grid. Okay, so a really diverse set of controlled tasks. And the setup was always train a main task, like reverse add, only on short lengths, maybe up to 16 digits.
4:30And then co-train it with an auxiliary task, like no carry, but train that one on longer lengths, maybe up to 32 digits. Exactly that setup. It's designed perfectly to isolate this transfer effect they hypothesized. So the million-dollar question, did it work? Did they actually see this extrapolation by association? They absolutely did. The results were pretty striking, actually, for the arithmetic tasks, like reverse ad. When trained alone, it hit a wall. Couldn't generalize past its training length, just as you'd expect. But when they co-trained it with those auxiliaries, no carry, carry only, reverse subtract, nae-3-co-t multiply, it suddenly could extrapolate.
5:07It generalized to much longer inputs, often reaching the full lengths the auxiliary task was trained on. Wow. Okay, so it really seems like learning the structure of the longer auxiliary task helped it figure out the main task at longer lengths. That's the strong implication. It wasn't just seeing longer sequences. It was seeing longer sequences in the context of a related computation. And they saw this across the board. In the string tasks, too, string copy and capitalize reverse when trained alone fail to generalize lengthwise. but co-train them with related tasks like MQAR or capitalized and revosed respectively, and boom, substantial length extrapolation again.
5:45Even for the mazes, those seem structurally quite different. Even for the mazes. Co-training DFS trace with shortest path, or the other way around, significantly boosted length generalization for both tasks. It suggests this principle is quite fundamental. That's really compelling. But you mentioned related tasks. What if the tasks aren't related? Does just seeing any long mazes help with long addition? Seems unlikely. Excellent. point. And that's exactly what their control experiments tested. They deliberately paired unrelated tasks. Like they tried co-training reverse add with a task called copy first op, which literally just copies the first number and ignores the second.
6:21Totally unrelated algorithmically. Or they paired string copy with reverse string. Again, different operations. And in these cases, no length generalization transfer, none at all. Okay, that's crucial then. It has to be task relatedness. It's not just about the model seeing long sequences. It's about seeing long sequences processed in a way that's somehow analogous or shares components with the main task. Precisely. There needs to be some shared underlying algorithmic structure or logic for the model to leverage. This is fascinating for models trained from scratch on these specific tasks. But, you know, most of the AI we interact with are these huge pre-trained models.
6:58Does this kind of association happen naturally during pre-training? That's one of the coolest parts of the study, I think. They looked into this, And the answer seems to be yes. Natural language pre-training itself seems to act like a massive, implicit set of auxiliary tasks. They used a model called SMOLM360M pre-trained on a mix of text and code. Importantly, they checked first the model hadn't actually learned these specific synthetic tasks during pre-training. Its accuracy on reverse add without fine-tuning was basically zero. So the pre-training didn't explicitly teach it reverse edition, but it taught it something else, something useful.
7:35Exactly. It seems to have taught it general sequence processing abilities or maybe reusable computational patterns. Because when they fine-tuned SMO-LM on tasks like reverse add or shortest path, they saw a really clear trend. Models fine-tuned from later pre-training checkpoints, ones that had seen more pre-training data, showed significantly better length generalization on these synthetic tasks compared to models fine-tuned from earlier checkpoints. Whoa. So just the process of learning language patterns somehow builds this. Scaffolding, you called it. That helps it extrapolate on totally unrelated tasks like arithmetic or mazes later on.
8:09That's what the evidence strongly suggests. Pre-training seems to equip these models with fundamental, reusable, computational building blocks that facilitate extrapolation, even for tasks that look nothing like natural language on the surface. Okay, this is starting to sound a bit like magic. How does this actually work inside the model? What's going on under the hood? Did they figure out the mechanism? They did dig into that using ablation studies, basically, tweaking parts of the model or analysis to see what matters. One key factor seems to be the positional encoding. They compared models using rotary positional embeddings, RhoPE, which encodes relative positions against models with no positional encoding, no P.
8:48and consistently the models with Rho-P were much better at enabling this length generalization transfer between tasks. Huh. That's interesting because sometimes Nopi is considered good for general length stuff, right? But here, for the transfer, Rho-P seems important. Exactly. It suggests that understanding the relative positions, which Rho-P is good at, might be crucial for identifying those shared patterns or structures between the related tasks that enables the transfer to longer lengths. Okay, so positional encoding matters. What about the actual brain cells, so to speak, the circuits? Is it using the same parts of the model for both tasks?
9:24That seems to be the other major piece of the puzzle. The transfer strongly correlates with the sharing of internal computational mechanisms, specifically reusing the same attention heads. They measured this using a couple of clever metrics looking at differences in attention patterns and in how important different attention heads were for each task, the mean ablation map difference. Lower differences, meaning more shared computation, more reuse of the same heads for similar functions, strongly correlated with better length generalization transfer. Ah, okay. So if the model learns a specific trick, maybe how to handle carries in addition, or how to track paths in a maze using certain attention heads for the long auxiliary task, it can then reuse that exact same trick using those exact same attention heads.
10:08when it encounters the related main task. Even at lengths, it's never seen for that main task before. That's precisely the picture that emerges. It develops a sort of generalized skill module or algorithm tied to specific attention mechanisms for processing sequences of a certain structure or length. And if two tasks are related enough, they can both tap into that same pre-developed module. It's a very efficient way to generalize. Okay, let's recap. We've seen this really interesting phenomenon, extrapolation by association or length generalization transfer. Models can get better at handling long sequences for one task if they're co-trained with a related task that uses longer sequences.
10:47It works across different domains, math, strings, mazes, and might even happen implicitly during large-scale pre-training. And the key seemed to be task-relatedness and the model literally reusing the same internal machinery like attention heads for both jobs. Yeah, and thinking bigger picture, this isn't just some arcane detail about AI internals. It gives us real insight into how these incredibly complex models actually learn and generalize. It suggests knowledge isn't rigidly siloed. These fundamental patterns learned in one area can be, well, elegantly reapplied elsewhere. And that could lead to smarter ways to train AI, right?
11:22Maybe needing less specific data for every single variation of a task, we can encourage this kind of transfer. Absolutely. Which leads to a final thought to chew on. If AI can spontaneously To continuously generalize length by associating related tasks, what other kinds of generalization might be happening through similar associative mechanisms? Could it extrapolate complexity or abstraction or maybe even creativity by linking related concepts learned in different contexts? Imagine an AI learning from, say, complex legal document analysis and suddenly getting better at interpreting intricate philosophical arguments because it found some underlying associative link.
11:59How do we even begin to harness that intentionally? What does that mean for the future of truly intelligent systems?
This academic paper explores length generalization transfer in Transformer language models, investigating their ability to extrapolate knowledge from shorter inputs to longer, unseen ones. The authors demonstrate that training a model on a related "auxiliary task" with longer inputs can significantly improve the generalization of a "main task" trained only on shorter examples, across diverse domains like arithmetic, string manipulation, and maze navigation. This transfer effect is also observed in pretrained language models, suggesting they develop reusable computational frameworks. Furthermore, the research provides mechanistic evidence that this transfer correlates with the shared use of attention heads between related tasks, indicating a compositional reuse of inductive structure.
keepSave to notecopy_alldocsAdd noteaudio_magic_eraserAudio OverviewflowchartMind Map