


27 Oct 2026 · 22m · 10 chapters

Predictable performance in AI “model routing” and cascades—whether external performance predictors/checker AIs are necessary, and when cascades fail economically.
No guest names or bios mentioned; the episode is presented as a two-host discussion.
A model’s internal self-confidence (token probability of its chosen output) is a strong baseline for error prediction. If an external predictor can beat that self-confidence, the base model is suboptimal; with fine-tuning that reduces suboptimality, the predictor’s advantage shrinks or disappears. Similarly, learned routers only help when the small model is weak; fine-tuning makes simple self-confidence threshold routing sufficient. Cascades face a “strong model error trap”: even escalated large models can fail on inherently ambiguous/unanswerable questions, erasing compute savings.
Benchmarks Winogrand, Common Sense QA, GSM-8K, MMLU-PRO; models include Llama 3.1, Qwen 3.5, Mistral, Gemma 3, and “Ulmo.” Reported correlation R=0.98 on Common Sense QA; 55/59 predictor cases and 53/59 router cases where fine-tuning eliminates advantage.
Written by AI. May contain mistakes.
Exploring the role of multiple AIs in performance prediction and its implications.
“Yeah, developers are building these massive, incredibly complex workflows.”
Understanding the self-confidence metric and its significance for AI accuracy.
“Yeah, we really have to start with the baseline of the original model.”
Investigating how external predictors interact with base AI models and their necessity.
“And the researchers used this exact self-confidence metric as their baseline.”
Discussing the theoretical limits of performance improvement through external predictors.
“But, you know, it also introduces a massive theoretical question.”
Reviewing extensive testing of AI models and the findings that question the need for external predictors.
“The team wanted to see if this math actually held up in the meat grinder of real state-of-the-art language models.”
Learn about the strategy behind using cascading AI models for cost efficiency.
“Well, let's lay out the strategy first, because cascading sounds incredibly logical on paper.”
Discover the experiment comparing complex routers to simple confidence thresholds.
“But the researchers set up a showdown to test this.”
Explore the concept of the strong model error trap and its implications.
“In 53, out of 59 model pairs, fine-tuning the weak model massively reduced the learned router's advantage over the simple self-confidence threshold.”
Understand the ramifications of optimizing AI models on predictability and performance.
“And then you paid the massive premium API cost to run the 120 billion parameter model.”
Consider the potential dangers of creating perfectly optimal AI models.
“But, you know, I want to leave you with one final thought to mull over, and it builds on the very mathematical bounds we've been unpacking today.”

Get the gist, explore an idea or ask about something said.
On Best AI papers explained, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00Imagine you just hired an incredibly brilliant, highly paid executive assistant assistant. Okay, sounds great so far. Right. They're lightning fast, they know practically everything, and they just handle all your daily tasks. But, and here's the catch, just to be safe, you decide to hire a second, slightly less experienced assistant. Wait, really? Why? Exactly. And this second assistant's entire job, like their only purpose, is to stand over the shoulder of the first assistant, watch the work, and predict when they're about to make a mistake. I mean, that sounds like a colossal waste of payroll.
0:34Totally. In the corporate world, you'd probably call that toxic micromanagement. Or, you know, a fast track to a hostile work environment. But the wild thing is, in the cutting edge world of artificial intelligence, this dynamic is actually one of the most dominant strategies in the industry right now. Yeah, developers are building these massive, incredibly complex workflows. Like one AI generates the answers, and a completely separate AI evaluates whether those answers are actually going to be right or wrong. Right, and it's everywhere. It really is. We call it performance prediction or model routing.
1:07And we are spending absolute fortunes on compute power and API calls to essentially build this digital bureaucracy of algorithms that do literally nothing but double check each other. Which brings us to the big question of today's deep dive. Yes. What if that second AI, that super expensive digital supervisor, is completely unnecessary? Today we're unpacking some serious data from the Max Planck Institute for Intelligent Systems. Yeah, it's some fascinating research. We are looking deep into the mechanics of large language models to figure out if this multi-layered checking system is actually just a massive illusion.
1:41And the research we're looking at today is incredibly rigorous. I mean, it forces us to rethink a very fundamental assumption. Like, we usually assume that if a separate AI can predict when our main AI is going to fail, well, we've built a robust, safe system. Right. That feels logical. But the data reveals this mathematical reality that just turns that completely upside down. So our mission for this deep dive is to cut through all the hype of these multi-AI systems. We want to help you understand one mind-bending rule, which is why an AI's mistakes are only predictable to an outside observer if the AI itself is fundamentally flawed.
2:18Or, to use the industry term, if it's suboptimal. Suboptimal, exactly. But before we can decide if firing our hypothetical checker AI is a good idea, I feel like we have to look at what the original AI already knows about its own accuracy. right? Yeah, we really have to start with the baseline of the original model. And in language models, that baseline is a metric called self-confidence. Okay, let me stop you right there because self-confidence sounds, I don't know, incredibly human. It does, yeah. It sounds like my AI is giving itself a little pep talk in the mirror before it answers my prompt.
2:50But we're talking about cold hard math here, right? Oh, absolutely. So think of it like a student taking a multiple choice test. The student fills in bubble C, but right next to it, the teacher makes them write down a percentage of exactly how sure they are. Right, like their certainty level. Exactly. So if the student writes C but notes they're only like 10 % confident, the teacher already knows they're probably guessing. They don't need a second student to point out the guess. Yeah, that's actually a really good analogy for the human experience. But let's look at how it works mechanically under the hood.
3:22Because when a language model generates an answer, it isn't just picking words out of a hat. Right. It actually calculates the mathematical probability distribution across all the possible tokens it could generate next. Wait, hold on. Tokens? Like individual words? Well, sort of. Think of tokens as the basic building blocks of language for an AI. Sometimes it's a whole word, but sometimes it's just a chunk of a word, like a prefix or a suffix. Oh, okay. Got it. So when the model looks at your question, it assigns a mathematical probability to thousands of different tokens that could potentially follow.
3:59Its self-confidence is literally the percentage probability it assigns to the specific sequence of tokens it ultimately chooses to output. So if the model outputs, say, Paris is the capital of France? It might internally attach a 99.9 % probability to that specific output being correct. Okay, so the model has its own internal probability score. It inherently knows how much it's guessing. Exactly. And the researchers used this exact self-confidence metric as their baseline. The core question they asked was, can a separate external performance predictor AI actually beat the original model's own self-reported confidence?
4:41Like, can that second assistant consistently spot an error that the first assistant was completely blind to? Right. And if the external predictor can beat it, that sounds like a massive win for the tech industry, right? I mean, it justifies all the money we spend on these multi-AI setups. You would think so. But here is the massive plot twist for you guys. The data shows that if an external predictor successfully beats the base AI's self-confidence, you actually do not need to keep that external predictor around as a separate step in your workflow. Yeah, it all comes down to this mathematical mechanism called exact transfer.
5:16Exact transfer. Right. The researchers proved mathematically that if an external predictor is better at evaluating your model's correctness than the model itself is, you don't need a separate checking step. Because why? Because you can just take the predictor's mathematical estimates, feed them back into the original model, and instantly recalibrate it. Wait, hold up. How does a math equation just absorb a checker? Are you saying they, like, merge the code? No, not merging the code, but adjusting the scoring system itself. Let's use a physical analogy. Okay. Imagine a golfer who consistently slices the ball to the right.
5:51That's our base model. And they hire a coach, the predictor, who watches them swing and says, hey, you're gripping the club 10 degrees too far to the left. OK, I follow that. Well, the golfer doesn't need to hire the coach to stand there for every single swing for the rest of their life just shouting 10 degrees right before the club hits the ball. Right. That would be absurd. Exactly. The golfer simply adjusts their own grip 10 degrees to the right. They recalibrate. Oh, wow. And in AI, this is measured using something called log loss, which is just a mathematical way of scoring how heavily a model is penalized for being confident about a wrong answer.
6:25So they tweak the penalty. Right. Through recalibration, the advantage of the predictor transfers exactly into the original AI's own loss function. You adjust the internal probabilities and the model absorbs the feedback. It becomes smarter on its own. So the golfer just incorporates the 10 degree fix into their swing and suddenly the coach is out of a job. Pretty much. I mean, if someone can spot your errors better than you can, you just mathematically absorb their judgment and improve your own baseline. That totally blows up the idea that we need a permanent, separate AI just to grade the homework.
6:57It really does. But, you know, it also introduces a massive theoretical question. Which is? If we can instantly absorb a predictor's knowledge to improve our base AI, is there a limit to that? Like, can we just keep attaching better and better coaches and infinitely improve our golfer? Oh, right. Like, can I just keep stacking predictors and absorbing them until my AI basically knows the secrets of the universe? Sadly, no. The researchers established a very strict theoretical ceiling here. They looked at a metric called excess predictability. Excess predictability. Yeah, that's simply the measurement of how much the external predictor can beat the base model's self-confidence.
7:37And they formulated a bound equation proving this excess predictability is strictly capped by two specific variables. Okay, lame on me. The base model's suboptimality plus its capacity gain. Okay, you just dropped some heavy jargon on me there. Suboptimality and capacity gain. Let me take a stab at capacity gain first. Go for it. Does that just mean throwing more computing power at it? Like adding more memory or processors? You're definitely in the right ballpark, but it's specifically about parameters. basically the size of the neural network's brain. Ah, gotcha. So if you have an 8 billion parameter-based model and you attach a 4 billion parameter predictor to it, you are effectively running a 12 billion parameter system.
8:20Right. So the capacity gain is just the brute force improvement you get simply because you're using a bigger, more resource-intensive system to answer the question. Ah, so it's not really a clever architecture at all. It's literally just paying for a bigger brain. Exactly. And what about suboptimality? Well, suboptimality is the gap between how your model currently performs and the absolute best it could theoretically perform at its current size. So if you have an 8 billion parameter model, there's a physical mathematical limit to how smart it can get. If your model is poorly trained, it's highly suboptimal.
8:53It's nowhere near its ceiling. Okay. Let me just make sure I'm wrapping my head around this. If I have a model that is perfectly trained, meaning the developers optimized it flawlessly and it has zero suboptimality, An external predictor cannot beat its self-confidence. Exactly. Unless I just brute force it with a massive capacity gain. That is the exact mathematical reality. Predictable performance isn't a sign that you've built a clever, innovative error checking system. It's actually a giant, glaring red flag. Wow. Yeah, if an external system can easily predict when your AI is going to fail, it heavily highlights the flaws and suboptimality in your original base model.
9:33It just means your original model was undertrained or poorly optimized for the task. I mean, that is a complete paradigm shift for anyone building tech right now. We look at these predictable, multilayered systems and think, wow, our error-checking architecture is so robust. Yeah. When we really should be looking at it and saying, wow, our base model is terribly optimized. Right. It completely forces developers to confront where they are spending their resources. But, you know, a theoretical bound on a whiteboard is one thing. Sure. The team wanted to see if this math actually held up in the meat grinder of real state-of-the-art language models.
10:09Oh, I know they ran a ton of experiments for this. What did the scale of that testing actually look like? It was massive. They tested 16 different models, and we're talking about heavyweights that developers are using right now. Like what? Models like Llama 3.1, Quen 3.5, Mistral, Gemma 3, and Ulmo. Wow. So literally the foundational models of the current tech boom. Exactly. And they ran them through four incredibly brutal benchmarks. They used Winogrand and Common Sense QA for logic and reasoning, GSM-8K, which tests complex math word problems, and MMLU-PRO, which is this massive multitask language understanding test.
10:49That's a serious conlet. It is. And for each of these models on each task, they trained a separate dedicated 4 billion parameter predictor just to evaluate it. So they essentially built a custom coach for every single model and every single test. Yes. And the initial findings were just staggering. They found a massive positive correlation between how suboptimal a base model was and how easily that 4 billion parameter predictor could beat its self-confidence. How massive are we talking? Well, on the Common Sense QA benchmark, the Pearson correlation was an R of 0.98. Okay. I have not taken a statistics class in over a decade.
11:27If I'm staring at a chart, what does an R of 0.98 actually look like? It looks like a practically perfect straight line moving up and to the right. Oh, wow. Yeah. It means there's an almost lockstep relationship. The worse the base model was, the more suboptimal it was, the easier it was for the external predictor to beat at self-confidence. So the whiteboard math held up perfectly in the real world. Perfectly. Okay. But proving a correlation is basically just observing a problem. Did they actually try to fix the models to see what would happen? They did, and this is where the experiment gets really fascinating.
12:01The researchers went in and fine-tuned these models. They gave them task-specific training to improve their performance, actively shrinking that suboptimality gap. Okay, let's drill into the how for a second. How does fine-tuning a model actually change this dynamic? Like, are we just feeding it an encyclopedia so it knows more facts? It's deeper than just memorizing facts. Fine-tuning adjusts the internal weights of the neural network. Okay. It teaches the model to align its internal probability distribution with reality. So instead of being 90 % confident in a wild hallucination, fine-tuning teaches the model that for certain types of ambiguous questions, it should only be, say, 10 % confident.
12:43Oh, I see. It learns the shape of its own ignorance. Ah. So it literally becomes more self-aware of when it's guessing. Exactly. And when they improved these base models through fine-tuning, they trained brand new, highly optimized predictors to evaluate them. And what happened? The results were totally decisive. In 55 out of 59 model task pairs tested, when the base model was fine-tuned and improved, the external predictor's advantage either shrank dramatically or vanished entirely. Because the base model's self-confidence caught up. Yes. As the model got smarter, its ability to say, I don't know, perfectly matched the external checker.
13:20Exactly. So if you're a developer, a product manager, or, you know, anyone paying the server bills for an AI workflow, this data sends a very clear, urgent message. Don't waste your money. Right. Do not waste your budget and expensive compute power building separate, complex AI assessors to double-check your outputs. Spend those resources fine-tuning your base model. Because as the core model improves, its self-confidence becomes so well-calibrated that paying for an external predictor becomes mathematically redundant. You got it. Which brings us to the real-world application that is absolutely everywhere right now.
13:55Cascades. Cascades. Or model routing. If fine-tuning kills the need for a predictor, what does this mean for companies trying to save money with Cascades? Well, let's lay out the strategy first, because cascading sounds incredibly logical on paper. Yeah, I'll set the scene for you guys. If you're building a massive enterprise system, you want to save money on API calls. Obviously. Naturally. So for 90 % of your user queries, you use a really cheap, fast, small AI model. But some questions are just too hard for the small AI. So you build a router, a predictive algorithm, whose job is to look at a hard question and say, nope, escalate this.
14:34Right. And it sends that hard question up the chain to a massive, expensive, state-of-the-art AI, like a giant$120 billion parameter model. You save money on the easy stuff and bring out the expensive heavy artillery only when needed. Yeah, and the theory assumes that a complex machine-learned router will intelligently identify exactly which nuances and edge cases the cheap AI will fail on and gracefully escalate them. Makes sense. But the researchers set up a showdown to test this. They pitted a highly complex learned router against the absolute simplest rule you could possibly write. Let me guess.
15:10The simple rule is just look at the cheap AI's self-confidence. Yep. If the cheap AI's internal probability drops below a certain threshold, just escalate it. No fancy routing algorithms, just basic self-doubt. Yes. And the data from this showdown completely mirrors what we just saw with the predictors. The complex learned router only provided significant efficiency gains when the cheap base model was highly suboptimal. Okay, so the worse the small model was, the better the expensive router looked. Exactly. But the moment the researchers fine-tuned the cheap model and made it better at its job...
15:43The learned router became useless. In 53, out of 59 model pairs, fine-tuning the weak model massively reduced the learned router's advantage over the simple self-confidence threshold. That's crazy. So, are these expensive learned routers just a total scam? Like, if I'm building an enterprise system today, should I just delete my routing layer and stick to self-confidence thresholds? It strongly suggests you should. But, you know, it also raises a deeper question about where those escalations are actually going. Because the researchers uncovered a phenomenon they call the strong model error trap.
16:18The strong model error trap. Why does everything in AI sound like a sci-fi movie type? I know, right? What exactly is the trap? It's basically the fatal flaw in the entire cascading philosophy. The premise of a cascade is that when the small model fails, you escalate to the giant 120 billion parameter strong model and it magically saves the day. Right, because we inherently assume size equals infallibility. Exactly. But there is absolutely no guarantee the strong model gets it right either. Wait, why wouldn't the big model get it right? It has 120 billion parameters. It's read the whole internet.
16:53Because language and logic are fundamentally messy. Fair point. On incredibly rigorous tests like that MMLU Pro benchmark, questions often contain inherent ambiguity, or they rely on poorly phrased premises or unstated human assumptions. Oh, I see. Yeah, a 120 billion parameter model can't magically solve an impossible paradox any better than an 8 billion parameter model can. Ah, so if the question is so incredibly hard or vague that the small model fails, it might just be an inherently unanswerable prompt to begin with. Exactly. The researchers found that even when the router worked perfectly, like even when it successfully caught the small model's mistake and escalated it, the massive strong model often failed the exact same question.
17:37Oh, man. And when the strong model fails, it completely wipes out any efficiency advantage the router provided. Because you paid the compute cost to run the router. Yep. And then you paid the massive premium API cost to run the 120 billion parameter model. And you still handed the user the wrong answer. That is brutal. It really is a devastating blow to the economics of cascading. I mean, if the strong model is going to fail anyway on the hard questions, and if fine-tuning your small model makes its own self-confidence just as accurate as a complex router... Then the massive architectural overhead of these multi-tier systems becomes incredibly difficult to justify.
18:13Precisely. Wow. We've covered some incredibly dense ground today. So let's summarize the journey we've just taken through this data. Sounds good. We started by looking at AI self-confidence, that internal probability distribution in a model assigns to its tokens. And it turns out to be this incredibly powerful, vastly overlooked baseline. Right. And the overarching takeaway from all the experiments, the benchmarks, and the bound equations is unambiguous. Predictability equals suboptimality. Exactly. If an outside system can predict your AI's errors better than the AI itself, it simply means your AI is poorly optimized for its size.
18:53So the better you make your base model, specifically through task-specific fine-tuning, the less you need external predictors, checker AIs, or complex routing gymnastics. You just absorb the predictability right into the model itself. Our hypothetical second assistant becomes entirely obsolete because the first assistant actually learned how to do the job right. It is a really compelling argument for simplicity in software architecture. Absolutely. But, you know, I want to leave you with one final thought to mull over, and it builds on the very mathematical bounds we've been unpacking today. Okay.
19:25It's a slightly chilling implication for the future of artificial intelligence. We've established that excess predictability shrinks as a model becomes more optimal. Right. As the golfer perfects their swing and hits their theoretical ceiling, the coach has absolutely nothing left to predict. Right. So imagine a future where the industry succeeds. Imagine a time when we train language models that are perfectly optimal for their given parameter size. Okay. Imagining it. They have absolutely zero suboptimality. According to the mathematical bound proven in this lab data, a perfectly optimal AI will have zero excess predictability.
20:02Meaning no external system could ever beat its self-confidence. Right. Now, what happens when that highly advanced, perfectly optimal AI inevitably encounters a truly novel, impossible situation and makes a catastrophic error? Oh. Because it's optimal, that error will be flawlessly aligned with its own internal self-confidence metrics. It won't flag a low probability. It will be entirely terrifyingly confident in its wrong answer. Wow. And because it has zero access predictability. No external safety checker, no supervisory algorithm, and no override router will be able to see the mistake coming either.
20:42That is wild. It forces us to ask a very uncomfortable question about how we design AI safety architectures. By rushing to optimize our base models and eliminate all suboptimality, are we accidentally mathematically optimizing our way into a future of totally unpredictable, uncatchable failures? That is an incredibly heavy thought to sit with, especially for anyone building these systems. You hired the perfect assistant. They do everything flawlessly for years. And then one day they confidently burn the building down. And absolutely no algorithm on Earth could have predicted it. Exactly. Well, thank you so much for taking this deep dive with us today.
21:20Whether you're actively managing compute budgets for the next great AI product, or you're just trying to understand the invisible algorithms shaping our world, remember that sometimes the most complex tech architectures are just expensive bandages for a base model that needs a little more time in the training gym. Keep questioning the hype, keep looking at the data, and we will catch you on the next deep dive.
Recent research explores performance prediction and model routing in large language models by studying when and why an AI's errors are predictable. The authors demonstrate that any performance predictor surpassing a model's self-confidence can directly improve the model's answer loss through recalibration. Furthermore, a predictor's advantage is strictly bounded by the model's suboptimality combined with the additional parameter capacity of the predictor. Empirical experiments across multiple benchmarks reveal that less optimal models yield greater prediction and routing advantages, which naturally diminish after task-specific finetuning. Finally, these theoretical limits extend to two-model cascades, showing that stronger routing advantages emerge in suboptimal models before they absorb the benefits of further training.