


25 Jun 2025 · 19m · 10 chapters

A discussion of a position paper arguing that “deep learning phenomena” (edge cases like double descent, grokking, and lottery ticket hypothesis) should be studied for their ability to refine broad, general explanatory theories—not for narrow, ad hoc explanations with little predictive value. It critiques “narrow hypotheses” and promotes sociotechnical pragmatism (research value tied to downstream utility).
Alan Jafarez and Mihaela Van Der Char (authors of the referenced position paper). No other guest backgrounds are provided in the transcript.
Edge cases are uncommon in real large-scale deployments but are valuable as testbeds; many proposed explanations are “true but useless” (example: Figure 4 claims double descent/grokking can be explained by tracking the number of prime parameters).
Double descent vanishes under common regularization/optimization (early stopping, SGD); grokking appears mainly in small algorithmic datasets and can be metric-dependent; lottery tickets require post hoc pruning/rewinding and are sensitive to hyperparameters, but their ideas influenced sparsity/pruning, quantization, and PEFT.
Written by AI. May contain mistakes.
Exploration of unexpected behaviors in deep learning models and their implications.
“Yeah, and to get really perceptive about the kind of phenomena we're talking about, the paper focuses on what they call edge cases.”
Discussion of edge cases that challenge our understanding of neural networks.
“What exactly are these deep learning phenomena that have generated so much buzz?”
An overview of the double descent phenomenon and its implications for model performance.
“Now, you're likely familiar with the classic U-shaped curve in machine learning, right?”
Examining the grokking phenomenon and its impact on generalization in neural networks.
“That reevaluation itself is incredibly valuable.”
Introduction to the lottery ticket hypothesis and its influence on network efficiency.
“Grokking really highlights the need to maybe reevaluate our internal performance measures.”
Criticism of narrow ad hoc hypotheses in explaining deep learning phenomena.
“Okay, so this brings us really neatly to the central argument of the paper, right?”
Advocating for broad theories to better understand deep learning phenomena.
“That is exactly the valuable approach the paper advocates for because that provides actionable insight.”
Explore the importance of edge cases in deep learning research and their benefits.
“And it sounds like they still see these edge cases as incredibly valuable, despite their, let's say, practical irrelevance for direct application.”
Learn about three main recommendations for researchers in deep learning.
“Okay, so with all that potential, what are the actionable steps?”
Understand the broader implications of deep learning research on scientific inquiry.
“We've really journeyed into a critical debate happening in deep learning right now.”

Get the gist, explore an idea or ask about something said.
On Best AI papers explained, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00Welcome to the Deep Dive. Today we're plunging into a really fascinating corner of AI research. It's a place where deep learning models sometimes behave in ways that just, well, they don't seem to make sense. These unexpected actions, they're called deep learning phenomena, and they've definitely got the research community buzzing. Our mission for this deep dive is basically to understand what these surprising observations are, and importantly, why some of the research around them might not be quite as valuable as it seems. And then crucially, what a more productive path forward looks like for the field.
0:30We're unpacking a really thought-provoking new position paper. It's by Alan Jafarez and Mihaela Van Der Char, and it challenges the very way researchers are currently studying these phenomena. Think of this as your shortcut to grasping a really important ongoing debate right at the heart of deep learning. Yeah, and to get really perceptive about the kind of phenomena we're talking about, the paper focuses on what they call edge cases. These are those surprising behaviors you see in neural networks that kind of defy our current intuition. But, and this is a key distinction, they don't commonly pop up in real-world, large-scale, deep learning applications.
1:04And the paper's core argument isn't that studying these things is useless. Not at all, actually. It argues their true value lies in helping us refine the really broad explanatory theories of deep learning, the fundamental principles, if you like, rather than just seeking these narrow, isolated explanations for each individual quirk. It's about using them to test and improve our foundational understanding. Okay, right. Let's unpack this a bit more then. What exactly are these deep learning phenomena that have generated so much buzz? We're talking about things like double dissent, grokking, and the lottery ticket hypothesis.
1:39What makes them so captivating? Oh, the interest is huge. It's hard to overstate it, really. The original papers on just those three have collectively racked up over 7 ,200 citations since just 2019. Yeah, and major conferences like NeuroPS and ICML, they've even dedicated entire workshops to them. Researchers are drawn to them because they're, well, they're genuinely counterintuitive. They challenge our established understanding of how neural networks actually learn. And the paper specifically defines these edge cases as, like you said, unusual behavior. Behavior you can isolate and analyze, but that doesn't prominently feature in practical day-to-day deep learning work, which forces us to ask, if they're not a direct practical problem, what is the value in studying them?
2:24Okay, let's start with double descent. Now, you're likely familiar with the classic U-shaped curve in machine learning, right? Where if you increase model capacity, the test error first drops, then it rises because of overfitting. It's pretty fundamental stuff. But double descent seems to throw a wrench in that, doesn't it? It absolutely does. Yeah. Coined by Belkin and others back in 2019, double descent describes this unexpected second drop in test loss. This happens as a model's parameter count continues to grow, just keeps getting bigger, and it completely defies that traditional understanding of generalization.
2:57So imagine that U-shaped curve error goes down, then up. But then, and this is the weird part, it dips again as the model gets even larger. Huh. That sounds like it would, I don't know, send a shiver down the spine of anyone building a huge language model. Is this something that keeps LLM developers up at night, or does it offer a different kind of insight? Well, surprisingly, it's practically irrelevant in most real-world scenarios. Oh, okay. Yeah, this phenomenon typically occurs when you measure model complexity only by parameter count, which isn't really how we usually manage complexity in practice.
3:30When you apply standard regularization techniques, things like early stopping or the common optimization method, stochastic grading descent. Right. Which, let's be honest, they almost always are applied. Double descent usually just vanishes. It disappears. You generally won't see it showing up in scaling analyses of the big models like LLMs or vision transformers. So, okay, if it's not a practical problem, what's the real takeaway from double descent? What did we learn? Well, I think the big thing is it fundamentally shattered a core assumption, an assumption we've held in machine learning for decades, really, that more model complexity always eventually leads to worse performance because of overfitting, that classic U-shape.
4:11Right. It forced this critical reevaluation of foundational principles, like the classic bias-variance tradeoff, that fundamental push and pull between a model being too simple to capture patterns and being too complex, basically just memorizing the noise in the data. That reevaluation itself is incredibly valuable. It suggests that sometimes, kind of paradoxically, outing more complexity can actually improve generalization in ways we just didn't expect. Okay, fascinating. And then there's grokking. The name alone suggests something really surprising is going on there. It does, yeah. Grokking.
4:44Introduced by Power et al. Yeah. In 2022. It describes a scenario where a neural network achieves, like, perfect training accuracy quite early on. Okay. But its generalization performance, so its test accuracy, only sharply rises much, much later in training. Yeah. Which seems to go against the intuition that, you know, test curves pretty closely follow training curves. Yeah, definitely. And it suggests that maybe simply stopping early might be premature in some cases. It's like the network suddenly just gets it, it grocks it, and unlocks generalization way far into its learning journey, even though it could already perfectly handle the training data hours ago.
5:22That is fascinating. But wait, if a model is already perfectly accurate on its training data, why would we need to wait for generalization to kick in later? And I guess, is this something that would genuinely impact someone training a huge model with massive data sets? Yeah, it's another one that's largely practically irrelevant for those large-scale real-world applications. Grokking is typically observed in very small, often algorithmic data sets, tiny ones really. And as the data sets get bigger, the effect tends to diminish. Attempts to find standard grokking in realistic scenarios have, well, mostly failed.
5:56Now, you can synthetically induce it, kind of force it to happen. Ah, okay. But studies that don't use those artificial setups usually just report standard generalization, not grokking. And actually, the striking visual effect is often just down to the choice of metric. If you look at the loss plots instead of accuracy, they're much less dramatic. So, yeah, concerns have been raised about whether findings from such simplified domains actually generalize to realistic setups at all. But it's not just a quirky anomaly, right? There must be a deeper lesson here, too. Oh, absolutely. It's not just a curiosity.
6:30Grokking really highlights the need to maybe reevaluate our internal performance measures. It pushed us to think more deeply about how learning actually progresses under the hood, suggesting that validation performance can plateau while deeper, maybe more meaningful learning is still continuing. And this led to a more holistic evaluation of learning dynamics. It even inspired some methodological advancements like a more stable alternative to the softmax function, which is super common for outputting probabilities in neural nets. Plus, this core idea that important quantities can improve after validation performance seems to have flatlined.
7:06That has actually spurred discoveries in other areas, like adversarial robustness and out-of-domain performance. So, ribbles outwards. Okay, our final phenomenon, the lottery ticket hypothesis. This one sounds like a gamble, and for a moment, it really seemed like the winning ticket for efficiency, didn't it? It absolutely felt like a potential game changer at the time. Yeah. Yeah. Introduced by Frankel and Karbin in 2019. The lottery ticket hypothesis basically suggests that inside any dense, randomly initialized neural network, there's a smaller subnetwork lurking inside. That's the lottery ticket.
7:40And if you could just find it and train only this subnetwork in isolation, it could achieve the exact same accuracy as the original, much larger network and the same number of training iterations. So the initial idea was, wow, this could revolutionize how we train networks, make them much more efficient right from the start. That sounds incredibly powerful, like transformative. So why isn't everyone in the industry using these lottery tickets to train their production models today? What's the catch? Well, the huge challenge lies in identifying these winning tickets before you start training. The original demonstration, the way they showed it worked, involved this post hoc process.
8:19You had to train the full dense network first, then prune it down, then get this, rewind the weights of the small network back to their initial state. Wow. And then retrain just that pruned subnetwork. That's just, it's not practical for real world applications. Too complex, too slow. And this whole approach is also highly sensitive to hyperparameters, the specific settings you use. And despite, honestly, significant efforts to develop a practical a priori discovery method, meaning finding the ticket before all that training, progress has been pretty limited. There are also some questions about whether the efficiency gains would truly materialize on modern hardware anyway.
8:55Actually, Jonathan Frankel himself, the original proposer, he's actually stated that he doesn't think people should focus on lottery tickets as like a primary research area anymore because of these challenges. Okay, so the direct application isn't really the point anymore. But what deeper understanding did exploring this idea unlock for the field? What did we gain? Well, the core ideas, even if not the method itself, have profoundly influenced our understanding of things like sparsity, pruning, and just general network efficiency. It really shaped our understanding of how sparsity, having lots of zero weights, relates to training dynamics.
9:33And that informed both post-training sparsification techniques, where you prune after training, and also during training techniques. The basic intuition that only a small subset of weights is actually crucial for performance. That's been integrated into other efficiency methods too, like quantization, which simplifies model weights. Right. And parameter-efficient fine-tuning, or PEFT, which lets us adapt these huge models with far fewer trainable parameters. Yeah, yeah, PEFT is everywhere now. Exactly. So it really fundamentally changed how we think about the underlying structure and efficiency of neural networks.
10:07Even if the lottery ticket method itself wasn't the direct path to deployment, it sparked a lot of related progress. Okay, so this brings us really neatly to the central argument of the paper, right? The problem isn't these fascinating phenomena themselves, double descent, grokking, lottery tickets, but maybe how we try to explain them. It feels like we're sometimes building these very elaborate, specific explanations for things that maybe don't have broad implications. That's exactly it. The paper defines and criticizes what it calls narrow ad hoc hypotheses. These are basically custom isolated explanations that are kind of overfit to the specific phenomenon they're trying to explain.
10:48And because they're so specific, they don't provide generalizable insights. They offer these sort of shallow resolutions that don't really broaden our understanding of deep learning as a whole. And the paper provides this fantastic, really vivid example to illustrate the point. It's in their figure four. They show how both double descent and grogging can be explained mathematically. by simply tracking the number of prime numbers in a network's parameters during training. Wait, the number of prime numbers? Seriously? Seriously. And while it's technically true in those specific contrived experiments, the count of primes does track the effect.
11:23It's obviously an absurd theory. It has zero broader utility. Right. It just perfectly illustrates how an explanation can be true in a very narrow sense, but utterly useless. It gives you no predictive power, no actionable insight for the field whatsoever. That's a great, almost comical example of the pitfall. It seems almost too easy to fall for these true but useless explanations then. How do researchers avoid getting sidetracked? And what's the alternative? How should we be approaching these phenomena to get genuine value? The alternative, according to the paper, is to focus on developing broad explanatory theories.
11:57These are explanations that actually refine our understanding of general aspects of deep learning. The big picture stuff. They use these phenomena, these edge cases, as test beds, opportunities to challenge or update our fundamental principles. Think back to the double descent example again. It's incongruous, the fact that it didn't fit with the classical bias-variance tradeoff, that fundamental principle we discussed. That disconnect prompted a reevaluation of that broader theory itself. And refining the bias-variance tradeoff, a broad explanatory theory, based on observations like double descent, That is exactly the valuable approach the paper advocates for because that provides actionable insight.
12:37It tells us, hey, if the standard tradeoff doesn't always hold under certain conditions, then maybe investing more in model complexity is actually worthwhile sometimes in ways we hadn't fully appreciated before. That makes a lot of sense. And this raises an important question. Why is prioritizing this kind of utility so important in deep learning research right now, especially with AIs growing impact everywhere? where it feels like there's maybe more at stake than just theoretical elegance these days. Yeah, absolutely. The paper aligns with what it calls sociotechnical pragmatism. It's basically the idea that research value is really rooted in its downstream impact, which encompasses both technical progress and societal consideration.
13:16It's about asking, is this knowledge useful and how? Especially when we think about the widespread influence deep learning now has. It pushes researchers to think beyond just, you know, theoretical elegance or solving a neat puzzle and consider the real world implications, whether that's building more robust systems or ensuring ethical deployment or something else. And the deep learning community, by and large, already values utility, right? Concepts like performance and generalization appear in nearly every research paper. The scientific method helps us refine theories, sure, but pragmatism helps guide which theories are actually worth pursuing based on their expected utility.
13:56Got it. So this isn't just an abstract philosophical argument. The paper provides some concrete recommendations for researchers on how to actually shift this focus. And it sounds like they still see these edge cases as incredibly valuable, despite their, let's say, practical irrelevance for direct application. Oh, they absolutely do. They make a strong case for why deep learning phenomena are actually a compelling research area in modern ML. even as edge cases. For one thing, they offer computational accessibility. Ah, right. You can study them with pretty modest resources, unlike, you know, training billion-parameter models, which opens up research to way more people in the community.
14:32Yeah. They also often have a low knowledge barrier. Because they're frequently studied in these minimalist, simplified settings, they're approachable even if you just have standard textbook knowledge. That's a good point. And furthermore, these phenomena encourage scientific inquiry over just competition. They shift the focus away from that relentless, state-of-the-art soda chasing towards more collaborative knowledge creation, which maybe reduces the pressure for only positive results and helps improve reproducibility. And finally, they represent this powerful intersection of theory and empiricism.
15:07Their simple side settings often make them amenable to mathematical theory, which fosters this great cross-pollination between empirical work and theoretical approaches. Okay, so with all that potential, what are the actionable steps? What should researchers do if they want to contribute meaningfully in this space, following the paper's advice? The paper details three main categories of recommendations. It's nicely laid out. First, identification and cataloging. This means really thoroughly exploring the phenomenon, understanding its precise settings, all the influencing factors, and deconstructing it into its basic features.
15:41Okay, like really mapping it out. Exactly. It emphasizes centralizing these efforts, open sourcing high quality code implementations, and maybe even changing publication norms to actually value these systematic descriptive studies more. It's about building a robust, shared foundation of understanding first. Makes sense. What's second? Second is prioritizing utility. Researchers should explicitly discuss the purpose behind studying a phenomenon. What's the goal? And how does it relate to broader aims like improving generalization or robustness or interpretability? Right. Connect it to the bigger picture.
16:14Precisely. It's about prioritizing ideas that have potentially generalizable implications and actively comparing and contrasting existing theories, not just proposing new isolated ones. The goal is really to maximize the relevance and impact on the broader field rather than just, you know, solving isolated puzzles for their own sake. Okay. And the third category? And third, simply following scientific principles. Good research practice. This means applying all the best practices we know, doing hypothesis-driven research, reporting negative results. Which is still hard to get published sometimes.
16:50Absolutely, but trucial. Ensuring falsifiability, meaning a theory, can actually be proven wrong. Improving reproducibility, maybe using pre-registration, doing replication studies. It encourages preferring theories that are general, predictive, and falsifiable. and evaluating a phenomenon's scope, where does it apply? Where does it, rather than just showing it exists under some highly contrived circumstances and fostering collaboration through shared repositories is also a key part of this. Building knowledge together. This has been a truly fascinating deep dive. We've really journeyed into a critical debate happening in deep learning right now.
17:24We've seen how phenomena like double descent, grokking, and the lottery ticket hypothesis, While maybe not always directly relevant for building the next big app, they are actually invaluable testbeds for our fundamental understanding of AI. Yeah, and to summarize the core takeaway again, the paper argues really strongly against just solving puzzles in isolation. Instead, it advocates for using these unique, sometimes weird observations to refine our broad explanatory theories. grounding our research in this idea of sociotechnical pragmatism, where knowledge is ultimately valued by its downstream utility, its usefulness.
18:03Right. So the discussion around these edge cases ultimately helps us ask better questions. It helps us build a stronger, more robust scientific foundation for the entire field of deep learning. It's really about moving from just knowing what happens in these weird situations to truly understanding why it matters and what larger lessons we can actually extract from them. Exactly. And maybe a final thought to leave you with. this whole perspective, critically assessing research value, distinguishing between narrow, ad hoc explanations, and broad, useful theories, that's not just for deep learning phenomena, is it?
18:35If you connect this to the bigger picture, how might this framework, this way of thinking, apply to other areas of science? Or maybe even to your own problem-solving approaches, whenever you encounter unexpected or counterintuitive results in whatever you do? How do you decide what's a useful insight versus just a curious anomaly?
This academic paper argues that not all explanations for deep learning phenomena hold equal value, particularly those observed in "edge cases" like double descent, grokking, and the lottery ticket hypothesis. The authors contend that focusing on narrow, ad hoc explanations for isolated phenomena is often inefficient and lacks practical utility in real-world applications. Instead, they advocate for a more pragmatic and scientific approach, urging researchers to leverage these phenomena as test beds for refining broad, generalizable explanatory theories of deep learning principles. The paper also provides actionable recommendations for improving research practices to maximize the broader impact and utility derived from studying these intriguing observations.
keepSave to notecopy_alldocsAdd noteaudio_magic_eraserAudio OverviewflowchartMind Map