Sonnet 4.5 & the AI Plateau Myth — Sholto Douglas (Anthropic)

29:54And coding is a huge market full of people who are really, really, really keen early adopters who love trying and switching things, who are really excited to play with new tools. There's massive, massive demand. There's dramatically more demand for software in the world than there is good software. We've seen that in every previous iteration of compilers and general web abstractions and so forth. There's just a booming demand for software. And so, I mean, basically, the models are better at coding earlier than anything else because coding is a uniquely tractable problem in some respects for the techniques that we have.

30:40The data exists in many ways. you can containerize and run things in parallel, you can run unit tests, and so you can verify. Something that you know when it works and know when it doesn't work. Self-driving is uniquely hard, right? You need the car to work first time, kind of. Whereas coding, the models can fail 100 times. As long as it succeeds once, then that's fine. So there's this tractability, there's this replayability that doesn't exist in other fields that touch the real world in some ways. You don't want a lawyer arguing your case that is an AI, right? because if what if it gets the case wrong sorry so as techniques develop coding is uniquely tractable it is and you can see that right like already people are dramatically I myself am dramatically like higher productivity when I'm using the AI tools to write code and I have a friend who like manages nine clawed codes which is just like a crazy number I don't know how he does that but I can only handle two.

31:43So it's maybe like a skill issue on my behalf. All right. So Sonnet 4.5 is presented as the best coding agent in the world. So maybe unpack that for us, including performance on the SWE Bench benchmark. What are the numbers? What are the facts? And then we'll go into how that works. So SWE Bench is the current benchmark for how we measure coding progress. in the outside world, which all the companies use to evaluate against each other, it's an imperfect benchmark in many ways, right? It is like 50 % one particular web framework and this kind of thing. But what it does do is it takes real-world scenarios of work that people have done, just submitting a pull request, so a change to a code base.

32:33That's stuff that's on GitHub. Stuff that's on GitHub. Right. And it checks whether or not the model is able to do that same pull request and pass the same tests. And this ends up being a pretty decent proxy for a couple hours of work from a software engineer. These changes aren't incredibly complicated, but they're a reasonable complexity, a couple hours of work. We moved recently from roughly 72 to roughly 78 in SweetBench, which is a pretty substantial step up. I think it's worth pointing out that as recently as a year ago, I think we were under 20 % or something like that as a field. So there's been dramatic progress on the ability of models to do this unit of work that a software engineer does.

33:20I think that Sweebench is imperfect in a lot of ways. And it's probably pretty close to what we call saturated. One interesting thing to look at, like AI benchmarks, is you see these, they lose their utility past the point because they no longer disambiguate the differences between different models of high capability.

33:44But the models, one, are SOTA. They're the best in the world on SweetBench. It's a decent proxy. We're also, I think, more excited by the fact that a lot of our customers and partners are really excited by the model. So one example of this is the Cognition folks in Devon found the model so useful. They had to rebuild their architecture around it. Yeah, yeah. They had a great blog post on this. A great blog post on this, right? I think that's the real measure of whether or not a model is good, is whether or not it enables people to do things that they couldn't do before. And really coding as a whole has been transformed in this way over the last year.

34:20So let's roll back a year and we look at 3.5 Sonnet, which is the first really strong agentic coding model. The first model that you could ask to do something in front of you and it was able to interact with your code base in your computer and do it. In many ways, this model is what caused the PMF for Cursor. Cursor took off like a rocket with 3.5 Sonnet because they were in the right place and they were able to capitalize on that model as offering a coding experience that didn't previously exist. And then actually Cognition and Windsurf went for an even more ambitious target. So basically there's like a spectrum of agency here where either you can ask it to do 30 seconds of work or a couple of minutes of work.

35:08Windsurf was made as a company in part by betting more aggressively on the agentic abilities of 3.5 Sonnet. Then roll into this year. Which just as a quick aside is one of the key lessons for anybody in the startup world in 2025, which is bet on what the models will be able to do in six months from now. Right, exactly. Bet on the exponential. So what I think something that a lot of coding startups are now asking themselves is what can they now do with models that are capable of independently pursuing goals for substantially longer than previous goals? You know, before you had to supervise the models every 30 seconds.

35:49Over time, over the next couple of months, you're probably going to end up in a situation where you only need to supervise the models every, you know, 10 minutes, 20 minutes or so. That's a pretty dramatic change depending on the complexity of the task. even we have a couple of examples I think it was mentioned in the blog post where we asked it to build something that looks roughly like a chat app you know something like Slack or you know and it was the model just worked for 30 hours like it was just spinning there on a computer for 30 hours and came out with a really good working Slack like you know Teams like app it's pretty incredible yeah that is nowhere near built into any of like the existing products maybe maybe cognition is cognition, I think, has always bet on a longer running, more independent, like, you know, agentic SWE, and maybe this is the moment that, like, really hits PMF for them, for example.

36:40Yeah. Let's unpack the 30-hour aspect, which is fascinating. So, first of all, to just ground it for people, so this is a computer use... Just coding, I think. Just coding. So what does the agent do for 30 hours? Is it clicking on stuff? Yeah, it is there. It's reading files and it's writing code and running tests. So in exactly the same way that a human would.

37:13Basically, you can think of the model as running in a loop where it can constantly decide what to do. People often mention something called tool use. And tool use is the ability to, well, I mean, it's in the name, But in this case, it can use things like tools, like read file, write file, et cetera, or run code in the terminal. And it is sitting there in a terminal on a computer in a loop, just constantly looking at the current code, deciding, oh, well, it can't quite do this yet, so I'm going to work on that next. It's often making plans, particularly to run for 30 hours. One of the things that we're pretty happy with about the recent launches, we finally taught the models to use what's called memory.

37:55And we've built that into the agentic harness. So it's able to create a markdown file of to-dos and things that it thinks are important to do, check them off and work on them and check whether they've been completed. There's almost this like self-verification loop. One of the things that people were worried about with language models over like I think a year ago or so was that they would fall off track. Like they wouldn't be able to self-correct. and that this would basically ruin their utility. I think one of the things that may be remarkable about the current generation of agents is that they can self-correct.

38:30In fact, they're astonishingly good at self-correcting. And this, like, emergent ability has been pretty helpful. Yep, so much to unpack on this. So this, I think I heard you speak about two axes in the past, one being raw intelligence and the other one being how long an agent can operate. So is the fundamental breakthrough in very simple terms that if you can do it longer, you basically have a very smart AI that can just work longer? Yeah, exactly. If you can maintain long-term coherency, then the model is able to do things that it couldn't possibly have done. If I asked you to just, in a single stream of thought, write a Slack or Microsoft Teams working version of one of those.

39:20You wouldn't be able to do it, right? You have to sit there and take notes, do this closed loop feedback system. So long-term coherency is really important. And it's something that we think is just really critical for this. I think a good way of measuring this is to look at the meter evals. They're probably my favorite eval at the moment. And what this eval is, is they've taken a bunch of tasks, which humans do, particularly in the machine learning or programming context. And they've annotated how long it takes a human to achieve strong performance of those tasks. And then they ask AI models to do them.

39:59And what they found is that there's this really strong relationship between progress and the time horizon over which the AI is able to complete tasks. And so I think it's like every couple of months, the time horizon that the AIs are capable of doing is doubling or something crazy. Maybe every six months, the time horizon doubles, which is just, it's like, it's utterly insane. Yeah. Now, again, like all benchmarks, this one is imperfect, right? It only measures pretty simple tasks. It only measures, I think, 50 % success rate or something like this at the task, not like 99 % success rate. But it's a good directional measure.

40:48And it certainly resonates with my own experiences of, you know, as I've been using the recent models, I start to feel, if I just set everything up right, I feel like I could leave this overnight. And I could just churn away and it would probably have something pretty useful for me in the morning. Yeah. What are some examples of tasks that you can do with 30 hours that you could not do with shorter runs? Yeah. I think in this case, the Slack-like thing is a pretty good example, where it's a significant piece of software. It's really like an end-to-end working piece of software, which often takes a bit of time, like not an MVP demo.

41:27Other things I think are interesting is your machine learning experiments and stuff like this are pretty interesting. you want something that's able to propose an experiment and write a bit of code, run some initial tests, come back later, et cetera. Really, it opens up the world pretty dramatically. Basically, working software rather than demos, I think, is the key thing. Right. Fascinating. Now, I'm not saying that the models will spin you up a full working software right now, right? It's not going to spin you up a Slack competitor this weekend. Yeah, although the cloud AI demo that you guys produced was pretty impressive.

42:04Yes, right? I think we're seeing... Which, for people who haven't seen it, like shows the progression of the models and how replicating the website went from basically... Impossible. Yes, caricaturing, but like doing wireframes to now doing a fully functional website built autonomously by the AI. Yeah, and it got some pretty complex features. Like it got artifacts. Artifacts is a feature where the model is able to write code and then the results of that code are displayed in the web browser. And in this case, the model replicated Claude.ai with artifacts, with everything else. I can't quite remember how long that one took, maybe a couple hours to do.

42:49But basically regard this as the first halting steps of this. It kind of works. Sometimes it won't work. Sometimes it will. over the next six months, over the next year, expect dramatic progress here. And look at where we are now versus where we were a year ago. And the difference is I expect the same jump, basically. Let's double click on the breakthrough part of this. So, Sunnet 4.1, I think, was able to run up to seven hours. In this case, it's 30 hours, which I realize is not across all tasks, but that's the upper limit. You alluded to some of this memory evolution, this context, this ability to self-correct.

43:34Maybe explain in greater detail the advances that enable that jump to 30 hours. I mean, I think that the biggest things here are, or the question that we often ask ourselves is what is preventing the models from like working for a long go, basically? Or when do you need to intervene? And I quite like the model of interventions in a Tesla sense. as an example. Because right now you need to intervene quite frequently. But it's usually on questions of taste rather than it is questions of raw programming ability. It's not like the model is unable to, when it's decided to do the right thing, to do it.

44:12But sometimes the models take shortcuts and sometimes the models forget the overall structure of what they're doing and they sort of lose themselves in the context. They're doing a locally sensible change, but it doesn't actually make sense in the global context of what they're trying to achieve. And so I think a lot of the improvements, both that we've made and that are still to go, are on this taste and context, basically. It's on making the model better able to decide smart things about the overall structure of the program that it's going to do.

44:48And not take shortcuts and write sensible and good code. What about memory? Memory is also very important because, you know, the models do eventually run out of context. And so being able to manage memory over time and sort of like, I suppose even learning from experiences is something which would probably help this a lot. You don't want the model to be constantly rediscovering facts about how a particular system or code base works. And now, this is actually one of those areas where, like, the question of taste or, like, the bit of lesson comes up. Because you can imagine us going and launching a massive effort to teach the models coding taste.

45:31And that could be one way that you solve taste. You have heaps of human software engineers decide, well, no, this is good or this is bad or whatever. Where does taste come from in software engineering or what do we regard as taste? It's typically that it's able to like, it easily sets you up to make changes later on or so on and so forth. Or it's easy for maybe multiple agents to communicate with each other and collaborate. Like, often, good abstractions are something that, you know, you or I could work together on a code base and not conflict with each other, right? And so, there is this question of how much do you focus on teaching the model coding taste via getting software engineers to decide what is good or bad, you know, or should you be creating, like, a society of models that all, like, have to together code a giant monolithic code base and, like, you know, if they're arguing that it's bad.

46:20You can sort of imagine the spectrum of potential strategies and picking the right one there is like a difficult thing. Going back to the jump in performance from last year's models or even this year's models or even actually Sonnet 4.1 to like 4.5, again, to the point about the pace of progress accelerating. What were some of the breakthroughs? That I can't really talk about. Yeah, I mean, I think it's important to recognize that it's not one individual breakthrough, really. It is the continuous application of lots of different things across the entire stack for many people. And it's mostly just a function of compute in many ways.

47:09There are obviously individual breakthroughs, but fundamentally progress has been pretty smooth. Like on the meter eval, if you look at progress over the last two years, you can plot it with a straight line, right? And so similar to Moore's law of the past and this kind of thing, even Moore's law is made up of lots of individual improvements. It's not any one critical breakthrough. It's more the accumulation of a huge amount of work in an environment where there's a sort of exogenous force of compute pushing progress forward. Okay, so maybe let's talk about progress at a more abstract level, but grounded in 2025.

47:49So a big part of the discussion seems to have been the evolution from a focus on pre-training to RL, which we touched upon a couple of times. Talk about the impact of RL and why is RL such a big part of the conversation today? For those listening, a good way to understand at a high level of pre-training in RL, pre-training is like skim reading every textbook in existence. And RL is like doing the work to problems and getting feedback on whether you were wrong or right. And there are actually a lot of things that you can only learn via RL. And a good example of this is the skill to say, I don't know, in response to a question.

48:29because in pre-training, remember, you're modeling the, you know, you're trying to predict what text is going to come next in the, you know, all of these, you know, textbooks, the entire internet in the world. And so the only reason you would say, I don't know, you know, as a pre-trained model is if you think the character that you're modeling in the text would say, I don't know. Like if it's a likely completion, right? Not whether you, in fact, don't know, but whether you think that the sort of player that you've pulled from this cast of characters that you could model would say I don't know.

49:07Whereas in reinforcement learning, you could, in theory, set up a battery of tests where there are things the model knows and things the model doesn't know. And you could reward it for correctly answering things it should know and penalize it for falsely answering when it doesn't know. And what it will then learn to do is it will learn to look up information inside itself and assess its own confidence in whether it knows that information. So saying I don't know or solving hallucinations intrinsically requires reinforcement learning in many ways. So that's one example. There's a whole bunch of things you can't otherwise know.

49:48So I think also an important change in this sort of like era of reasoning models and RL on language models is at the end of last year, RL on language models finally started to work. And I think OpenAI deserves a lot of credit for releasing the first like serious RL plus LLMs release with O1. And I think this really kicked off a pretty substantial change because it opened up a new axis of scaling, right? There was pre-training scaling, and now there's test time compute and RL scaling. I think this is something which all of the research labs were investigating already. One of the reasons that DeepSeq was able to follow so fast was that they'd actually already released papers in the direction of doing RL on language models before, for example.

50:37and so this was it was already an idea in the air but opening I deserves a credit for that crystallizing it releasing it and detailing the first public existence of those scaling laws and maybe to continue making this super educational how do test time compute and RL overlap yes one way of thinking about this is test time compute is doing a lot of reasoning and then RL is the feedback signal on whether or not that reasoning was right or wrong. And so test time compute is a way of answering questions that are hard for you to answer. Let's say I ask you a question that you just know off the cuff of your, like, you know, off the back of your hand, basically.

51:24It's not like from a field that you really know, whatever heuristic that you've already done. You've already baked that in, like your muscle memory, so to speak. but for something which requires you to really think and really learn. Like when you're first doing math, if I ask you a basic times table right now, you can say that off like this. But if you're a kid, you have to like do out the math and all this kind of stuff. You need to like do the reasoning chain to learn it. And then you get feedback on whether it's right or wrong. So test time compute lets you do harder problems than you can currently do off the cuff.

51:58And RL then allows you to sort of distill that back into the model. It's almost like a ladder. You can like constantly do slightly harder problems because you're learning strategies to do harder and harder and harder problems. Reinforcement learning is not a new concept. So we were talking about Richard Sutton, who's been doing work in the field for decades and others as well. And then there was, you know, AlphaGo, like the whole, that old line of very successful, impressive RL-based successes. So why is it that in 2025, there seems to be a breakthrough to apply those to LLMs? Yeah. In some ways, it's quite funny.

52:38A lot of the... Okay, yeah. How do I say it? Let's take the DeepSeek paper, for example. In the DeepSeek paper, they detail one, an approach that works, and two, a lot of approaches that don't work. Actually, some of the approaches that didn't work were the approaches that led to AlphaGo's success. one of the craziest things about RL on language models in the RL from verified rewards regime is it's almost the simplest possible thing. It's almost too simple to work. And this is, again, comes back to that question of taste where really, I think a lot of people thought this was just too simple to work.

53:19And so they tried more complex methods that ultimately ended up being harder to get to work. And there may still be juice in those methods. But it was actually really important to nail the simple thing first. And so I think people were like almost too ambitious with the RL strategies that they tried initially. I think there's also a minimum bar in LLM quality that is required. Like you need the model to be able to solve meaningfully difficult coding and math problems before you can get that feedback loop of, well, you solve these ones right and you solve these ones wrong. right um and i think also one of like maybe the unintuitive things is those reasoning chains of tokens uh people for a long time thought that you'd need to do something clever to give the model long-term coherency you have to remember that two years ago 8000 tokens was in long context for a language model you know two and a half years ago 8000 tokens was long and now models are using 8 ,000 or 30 ,000 tokens to reason about something, right?

54:32So there was this real phase shift in, oh, language models are smart enough, underlying priors, that they can solve sensibly difficult questions. They're actually reasonably coherent at longer context than we thought they would be coherent. And that this ability to reason in long chains of tokens can emerge naturally with the right feedback signal. And this is a little bit counterintuitive, I think. Most people wouldn't have expected off the bat that the ability to reason would emerge naturally. There was a lot of thought that you'd have to structure it. You'd have to provide strategies for it to do reasoning.

55:12You'd have to build all these things, you know, prompted and hinted and this kind of stuff. And actually it turns out, well, no, you give it math questions, tell it whether it got them right or wrong, and the model will learn. But it comes down to a bit of lesson in scale and search is just allow the model to search, have enough compute to run the experiments, and the model actually ends up figuring out a really effective and sensible strategy. And that's what's happening now, right? The big labs are basically giving a lot more compute to RL. Yeah. Minimum base model quality, minimum amount of compute for RL, sort of trust in the ability for long-term coherency, doing the simple thing that works.

55:50they still sound obvious but they're actually like a little bit counterintuitive sometimes. You mentioned the word AGI earlier so is your personal sentiment that the combination of ever more powerful LLMs plus RL gets us there with a side obvious question of what there actually means and what AGI means today Yeah, there's a few definitions that one could use. I think a useful one is better than most humans at most computer-facing tasks. Because I think that's a really important moment for the world, where we go, okay, intellectual labor is addressable via this set of algorithms, and that totally changes the world.

56:38I think there are other definitions that are stronger that you could use. One of those is... Stronger. That was pretty strong. Yeah, sorry. I mean harder to meet, maybe. Yes, yes, yeah. Because you could have this and it could still not learn as effectively as humans, right? We learn and generalize from very few examples. We have incredibly high what's called sample efficiency, whereas AI models need hundreds of thousands of times more experience, hundreds of thousands of lifetimes, basically, to learn the things that we learn.

57:14And they can, over those thousands of lifetimes, they do learn the skills that we do to an incredibly high degree of accuracy. I think one of the important change over the last year has been that RL has finally meant that we have sort of this algorithm that allows us to take a feedback loop and turn it into a model that is at least as good as the best humans at a given thing in a narrow domain. And you're seeing that with mathematics and you're seeing that with competition code, which are the two domains most amenable to this, where rapidly the models are becoming incredibly competent competition mathematicians and competition coders, right?

57:57There's nothing intrinsically different about competition code and math. It's just that they're really amenable to RL. And any other domain, but importantly, they demonstrate there's no intellectual ceiling on the models, right? They're capable of doing really tough reasoning, given the right feedback loop. so we think that that same approach generalizes to basically all other domains of human intellectual endeavor where given the right feedback loop these models will get good enough that they are at least as good as the best humans at a given thing and then once you have something that is at least as good as the best humans at a thing you can just run it you know a thousand in parallel or a hundred times faster you have something that's actually even just with that condition substantially smarter than any given human.

58:48And this is completely throwing aside whether or not it's possible to make something that is smarter than a human. It seems entirely plausible, right? Like the brain is ultimately a biological computer. It seems possible to make a better one. But the implications of this are pretty staggering, right? Which is that in the next two or three years, given the right feedback loops, given the right compute, given the right elbow grease, and this kind of stuff. We think that we as the AI industry are all on track to create something that is at least as capable as most humans on most computer-facing tasks, possibly as good as, you know, many of our best scientists at their, you know, fields.

59:30This is really wild. It'll be sharp and spiky. You know, this will, like, there'll be examples of things it can't do and this kind of stuff, but the world will change. What do you make of the, you know, the counter thesis of the, again, Rich Sutton or Yann LeChan that seem to be saying that like a different approach is needed or RL only. Where do you make of that debate? Yeah. I think that it's true that our models don't learn anywhere near as efficiently as humans do, right?

1:00:04They take, you know, a thousand lifetimes to learn. but this is I think fine because they can live those thousand lifetimes whether in simulations or doing a job at a thousand firms and so on and so forth I think that the maybe I would disentangle there's two arguments one is like architecturally that transformers are like insufficient I don't think that's true I think we haven't yet really found anything that transformers haven't been able to model provided sufficient data and sufficient compute I think RL as an objective is a pretty powerful one Rich Sutton is actually a big fan of RL as an objective he just thinks we're actually encoding too many priors in with pre-training and this kind of thing it's not an adequate representation of the world I think so far the evidence indicates that our current methods haven't yet found a problem domain that is not tractable with sufficient effort and yeah so things that would make me eat my words is like if there was some domain that we put a lot of effort into that just didn't move like the goal the goalpost like so the like benchmarks just didn't move as and we just couldn't make any progress for a year then I would be like okay yeah there's some fundamental limitation here but instead what I just constantly see is every time we make a benchmark that measures something we care about, progress is incredibly rapid along that.

1:01:39Yeah. I think this is like worth crying from the rooftops a little bit of like, guys, anything that we can measure seems to be improving really rapidly. Where does that get us in two or three years? I can't say for certain. Yeah. But I think it's worth building into like, you know, respect to worldviews that there's a pretty serious chance that we get. something that is AGI. So you think people don't realize, you know, it's always interesting, right? Because, you know, reading stuff like online in the last, you know, three, four months, it's like this theme of the, you know, we've reached a plateau, but basically saying the opposite, right?

1:02:21We are in an exponential curve and many people don't realize that it's the case. Exactly. And I mean, people have said that we're hitting a plateau every month for the last three years um and if you look at what we've come over the last three years it's incredible uh i think that one other thing that makes me think god we're not anywhere close to a plateau is i look at how these models are produced um and every part of it could be improved so much like it is a primitive pipeline held together by duct tape and the best efforts and elbow grease and late nights and like god it's actually i remember um uh i don't know if this is a good analogy or whatever but i remember i went sailing with a couple of friends a few months ago and the boat was so well designed it was just like clearly the product of you know like millennia or like you know centuries of like accumulated human design and effort and i was like wow like this is this is what it feels like to be in a uh sort of the the accumulation of a lot of human effort right it's Actually, it's pretty hard to beat today's best sailboat designs.

1:03:27But when I look at an LLM training pipeline, it is two and a half years of best effort, last minute, desperate effort. And there's just so much room to grow on every part of it. So first of all, Sonnet 4.5, which is described as the best coding model in the world, also seems to be performing across a lot of different other domains like economics, research, and finance. So it's already just to verbalize that. One of the things I was really excited by, actually, was that there was that GDP eval that OpenAI released. And I'm not sure, Sonnet 4.5 was only just released, so it's not on there. But 4.1 Opus was the leading model there.

1:04:14And I think that's a really interesting and really good eval because it demonstrates such a breadth of tasks across all parts of the economy. It's an eval that's across the various sectors of the economy, right? So manufacturing, and basically they took a bunch of experts to describe what success looks like. And now the models are going to be able to be measured not across just coding or some limited tasks, but across everything. Is that a fair way of describing it? I've wanted someone to do this for a long time. Take the Bureau of Labor Statistics. I think the most important input into policy would be take the Bureau of Labor Statistics, take all the jobs there, break them down to tasks and see whether the AI models are able to do that and measure progress over time, right?

1:04:53And this is obviously going to be in perfect measure. We'll probably reach like better than human on the GDP eval and it won't change anything economically because it'll be all the connective tissue and all the like, you know, the context and actually like the task won't be representative. But again, we'll then find better ways to measure these difficulties and we'll keep pushing benchmarks. I've wanted someone to do this for a long time. I'm really glad they did it. I'm really glad that our models were general and generally strong and sort of straight up top across all the areas. And I think policymakers should really look at this and extend it and really make an effort in investing and figuring out whether we are on track for what I've been claiming we're on track for.

1:05:36We can measure this. Yes. And we should be. Yes. So, yeah, just to close on that last theme. so awesome, all very exciting. What do we all do? How do we prepare for this world that seems to be around the corner? Yeah. I think the most actionable piece of advice is keep planning for a world where you as an individual have more leverage, right? Right now, I can use two coding agents to do twice the work that I could have done before. If coding agents progress in the way I've been saying in a year or two, you'll be able to manage a team, basically, that works 24-7 for you doing work. I think we should expect in the digital domain for individuals to get dramatically more leverage over the next couple of years.

1:06:29I think then many incredibly important problems, you're going to track, our world is so imperfect in so many ways. people still live in dramatic poverty health and medicine is unsolved housing is completely unsolved the world could be a million times better in so many different ways and what I hope is that people take initially models giving us leverage over the digital world and then hopefully models giving us leverage over the physical one through robotics to dramatically improve it Is that happening? Robotics is another thing that seems to be one of the key themes But on the other hand, to use actually the word hand, it seems like people are just still struggling to make hands move the way.

1:07:15So, like, the physics of it seems to be the limiting factor. Yeah, there's this thing called Moravec's paradox, right, which is that things which we find really easy, like manipulation, picking up objects, are really hard for AI. But maybe things which we find are hard, like reasoning through mathematical problems, are easy. I actually think Moravec's paradox is a little bit fake. and I think this is mostly a question of data availability and like, you know, RL signal and stuff. And I think one interesting, one reason to look at this, if you look at like robotic locomotion, so, you know, the ability of robots to walk around and balance and stuff and look at the videos of the unitary robots, this difference now versus two years ago is crazy.

1:07:56These things are incredibly agile. Like there's this video, I think someone kicking one over and it literally does a matrix kind of like get back up thing. It's crazy. This is because locomotion is a really easy RL signal. And right now you can pretty much, like locomotion is kind of solved, to be honest, with basic RL. Manipulation is a bit harder. But there's a few things that make me think that robotics is going to work. For starters, I've seen incredible progress from the robotics labs over this year. Really, they've gotten to the point where they can do pretty interesting, you know, basic physical tasks.

1:08:31Two is the existence of a large generator verifier gap, which is that one of the things that makes improving our models hard is we constantly need to find people who can beat the models at the things we want to improve them on. But with robotics, we're making really smart general models. So you can actually have these as teachers or judges for whether or not the robot is doing the right thing. If I say stack the red block on top of the blue block, we could then ask the language model, did it stack the blocks appropriately? If so, give it a reward. If not, don't. So you can use the generator verifier gap to give models feedback.

1:09:10And finally, for a long time in robotics, people thought they would have to solve long-term coherency in planning. And that's also something that language models have made easier. They can break things down into multiple steps. So all the robotics labs are focused really hard on making great motor policies, and they're making incredible progress. It's mostly just a data and feedback loop question. All right, it's been fascinating. I can think of another 40 questions that I would want to ask you right now, but you've been incredibly generous with your time. Thank you so much. This was terrific.

1:09:40Really appreciate it. It was a real pleasure. Thank you very much. Hi, it's Matt Turk again. Thanks for listening to this episode of the Mad Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.