Scaffolding is coping not scaling, and other lessons from Codex | OpenAI’s Thibault Sottiaux

0:05My guest today is Thibault Sotio from OpenAI's Codex team. And Tebow has been working shoulder to shoulder with research and engineering in San Francisco to solve one of the hardest problems in tech, true agentic autonomy. Tebow, it's really great to have you here on Dev Interrupted. Hey, awesome to be here. Really excited to talk about all of this. Great. So today we're going to look at the bitter lesson of AI development. There's so much we can dig into about what you're building and about why clever tricks and domain expertise can sometimes fall short and why scalable primitives are winning.

0:38We're going to explore the Codex team and how it balances that research with exacting the requirements for production and making a happy developer tool that works for everybody. But I want to start at Codex, just in your own words, like learning about OpenAI's flagship coding agent. Can you tell us a bit about what it is and why you describe it as an agent first instead of a product? Yeah, so we think about the two parts. First and foremost, we're building a SOTA agent that is able to act and perform incredible amounts of work on the coding front and help software engineers in this world. And this agent is fairly general.

1:18You can put it to work in many places, and that's also where the products come in. So it's like figuring out what the best way is to leverage an interface with this agent that is going to be ever increasingly more capable. And there is something really interesting when you shift your mindset to building an agent first and then figuring out where to put it to work. It's like you find like a remarkable amount of places where this agent, you know, comes in handy and, you know, can actually do economically valuable work. And we're also thinking about, you know, what does this mean beyond just coding, right?

1:48Even for software engineers, it's not just about code generation. It's about solving many other parts of the day-to-day that are actually the bottlenecks. And so building that general agent is what we're after. Yeah. And we're going to talk more about what those other bottlenecks are. We talk about that a lot on Dev Interrupted, especially in the last year, about how agentic development has really made bare all of those human and communication problems that are affecting teams really deeply. Something that you said there about thinking about it differently from being a product, it actually makes me think of almost like a product and a platform analog.

2:25With a platform, you can put a lot of products, a lot of things in an ecosystem together. An agent, it seems to be emerging as something that operates the same way you can build an agent, like you said, and then figure out what are the use cases, how do we step this forward or backward into the products that make sense for people. Yeah, that's right. Right. So if you think about, you know, building an autonomous, like ever more capable entity and keep your mind sort of flexible about, you know, what the best form factor is, what the best product around it is. And, you know, perhaps we put it to work inside our own products, but we also partner with other companies to put it to work in their products.

3:00It opens the door for like a lot of like, you know, great ideas that you don't necessarily have. You know, we don't have them ahead of time. It's like, you know, we can focus on like building the agent and then, you know, figuring out like where to put it to work later. And what is it like to work in an environment where you're building a coding agent and you're sitting on top of a frontier model? Like you have the vertical integration of that entire process, which is a unique leg up. What is that like? That's right. It's very interesting because we're able to take some of the best ideas from engineering and have them influence research and then do the reverse as well.

3:36where research influences the entire engineering roadmap for how we build out the agent. And one of the things that you can do when you're vertically integrated, you can decide where you actually fix problems. So you don't have to fix everything in your harness. Some of the things we decide to fix downstream by training new models. And we know that by training the model, we will have a jump in the capability that we need three months down the line, six months down the line. And it allows us to do these trade-offs that you can't do without a vertical integration. There's also this thing called, you know, the no free lunch theorem, which is basically like if you're trying to adapt and be intelligent in any possible distribution.

4:17Well, this is going to be strictly less optimal that if you were to actually build something for a very specific distribution. And so by coupling like the harness and the model and like that's what we named the agent, we're able to get a lift in capabilities. And that's, you know, obviously very interesting, like, you know, for something as important as coding. Yeah, it's fascinating that you mentioned too that those two things, they feed into each other, the research and the engineering. It's a big loop. And inside of it, there's lots of little loops. I think that's what we've all been discovering in the last year, especially people who have a lot of success working with agents, is finding those loops, those workflows that work perfectly, the inputs and outputs, feeding into each other and making the system better.

4:58So that is a unique advantage and kind of like a vertical integration front. And I guess in this world where there's so much greenfield, you have the model, you have the coding agent, you can build it and take it any direction, take this almost like agentic first approach and then build towards what individual people and organizations need. But how do you, as the person at the center of that, prevent complexity from trapping your own architecture and keeping it simple? That way it works for everybody. Yeah, so we try to stay grounded in things that have stood the test of time and that we know have been true in the past, where simplicity just often leads to better scaling over time and better overall performance of the system that you're building over time.

5:45And what we're really looking at is the ability of the agent, so it is like the system of the harness and the model to continue to increase in capability over time as our models continue to increase in capabilities. So it would be really unfortunate if we introduced bias in the harness or in our solutions, which means that when we have a capability jump in the model, we're actually not able to express that because we somehow constrained it in a way that prevents it from expressing its true capability. And that's what we would refer to as a capability overhang. And by keeping things simple, you reduce the amount of things that can actually go wrong.

6:30And it's all about choosing the primitives that are proven to scale with model capabilities. And that's actually also closely related to a research problem. We can do the research on that of, hey, this harness scales with different model sizes and different model capabilities. We look at on the smallest model and then a mid model and then a frontier model. How do we get the scaling that we expect? So you can extend the scaling laws on the entire system. Right. And so in order to apply the scaling laws, you have to have simplicity, ultimately, that those things are leveraging, that they're driving.

7:14Otherwise, things are going to clash. They're not going to work. And like you said, you can't take advantage of immediate gains. I think that's really the biggest win of the system is that when you're so lightweight, it's so easy to pick up and put it onto the next new thing, which is what's so important when you're coupled closely with a model. And so that's something that's really fascinating. I think the simplicity, though, you know, it can't be understated. It's earned, right? It's like earned through research and engineering. Like you had to climb a summit of complexity, I'm sure. And at one point, things were very complex.

7:48And then you have to come down from that summit, like you said, to find the primitives, the things that emerged as more clear, right? And so that's like the journey that you're kind of describing. Yeah, it's an ever-continuing search for the right primitives. And once you discover those primitives, they seem delightfully simple. They often seem delightfully simple. But the search for those primitives, it's a complex matter. And you might not find those primitives immediately. And so you have complexity in your system, and you look at that complexity, and you go, we think there's quite a bit of bias here that we introduce and we constrain the model in a way where it will hamper its productivity at some point.

8:33But this is also this delicate balance between the harness and to me it's called a harness because you're scaffolding it in a way where you want to remove the scaffold over time because the model is able to stand on its own. And it's all about figuring out when is the right time to remove like pieces of the scaffold and to build in a way where you can do that. And the advantage of like coupling the two together is like, you know, we have to care about like our model series and our model series only. And so every time, you know, we improve things, we can remove bits of the scaffold without, you know, being afraid of like breaking something that is not under control.

9:10That's a really smart insight that I want to like click on again because, you know, you're calling out that ultimately the harness, that scaffold that you build, it should lean towards that simplicity because like you said, But it's supposed to be kind of there until the agent can stand fully on its own and do what that harness is kind of getting it to do. Some people lean, of course, the opposite approach. And they treat that harness more like a jetpack. And it doesn't go the way of simplicity. It goes of how can I put so many tools and so many things in this box? And so I think it's like two different types of ways that people come at them.

9:44And they obviously achieve different levels of success. It doesn't scale as well as having a lightweight harness like what you're calling out. And I think that's kind of part of what y 'all have earned from your years of research. And, you know, there's also Richard Sutton's Bitter Lesson, which calls out that clever tricks and domain expertise, they don't scale compared to these primitives, right? And when we talk about these primitives, they're compute and power and time. And that's what you're on the mission to find is how can we take best advantage of those so everybody can get it. But, you know, along that journey, you mentioned that you open sourced like the repo, you open sourced the process by which people can build these agents.

10:24What was that like? Like what led you all to that decision? So around the time that we decided to open source a repo, I think there was a lot of mysticism around agents and how they worked and what you needed to build on top of a model in order to have it be able to act safely into the world and perform economical, valuable tasks for you. And so we wanted to sort of show how, you know, delightfully simple it can actually be and show some of the primitives that are important to get right. And, you know, we also wanted to show that if you do these things, you know, you can get incredible performance out of our models.

11:04And so, you know, it plays this like dual role. Additionally, on top of that, sort of like had this idea that, you know, if you solve for code generation, open source is going to change. and we want to deeply understand how open source is going to change by being open source ourselves, at least for part of our technology. And then one additional idea and like reason for why, you know, I thought this might be interesting and like others agreed was that we were going to see a lot of tinkering in the space of agents and a lot of interesting products being built on top of it. And having like an open source core, you know, allows people out there in the world to like, you know, get inspired and tinker and build things that we might have never imagined.

11:49And so it allows to tap into this creativity of the open source community and to let people innovate and invent new things. And I think altogether, it just made so much sense for the agent to be open source. And yeah, it's something that I've been very proud of. Yeah, and as part of building that community, making a place where people can come and be part of the development, Have there been things about that that surprised you or stood out? You're like, this was an earned thing from the community. This is a proven success of why open sourcing this was the way. Yeah, one of the things is initially we got things somewhat wrong.

12:27We accepted too many contributions and we sort of lost control over the repo. We course corrected later. At some point, we migrated to Rust and we had decided that we had conviction on some of the primitives that we needed to build. And, you know, we wanted to build those primitives, right? And, you know, we also had conviction that we were going to see millions, if not billions of agents running concurrently at some point. And we wanted to write this like in an efficient language. So like the Rust migration was a bit of a difficult moment with the community because we had been accepting a lot of PRs through like our TypeScript open source repo at the time.

13:06And then, you know, we just sort of like rewrote things to Rust. we did build like very good partnerships with like awesome contributors over time to the rust core and so that that relationship to the community has evolved over time one of the things is just how great it is to have it all there out in the open it's like we do get a lot of reports and people like sort of like point at you know this is where it is like you know here's like how you should fix it or like you know here is like an attempt at fixing it and then we also get a lot of inspiration from all the forks. I think there's like, I don't know the number, but like over a thousand forks of the repo.

13:42Some of them are actually quite popular. And, you know, they just come up with like new ideas and like we work with the authors of like the forks like to port some of the changes back into, you know, the Codex open source repo. And it's pretty cool like to be able to learn those things from how people change and alter your code. And you can also then get a glimpse of how the agent can be adapted to other things. You're going to get a really good front row seat at how these early adopter developers are picking up these systems and molding them to their specific domain problems. So it's a really great partnership and definitely something I think people should check out because that's kind of rare and it's part of that relationship that even goes back as deep as the model, right?

14:24That's kind of how deeply penetrating this type of technology can be. Every week I hear from a company that, you know, builds their company around like the Codex open source agent where we're like, oh, you know, it's just like it works so well. It works actually like on non-coding things. And we adopted it like, you know, we adapted it like in this and this way. And like now it's able to do like this other economically viable task, such as editing spreadsheets, for example. And or like we embedded it into a browser. And then I get these demos and like it's always very cool to see that. somehow we're contributing to those things as well.

15:01Yeah, totally. I'm curious to know, you talked about earning all of this simplicity and getting back to the primitives. And we're at the top of a new year. It was spent all last year experimenting with these new tools. What is something that you picked up in 2025 that maybe you're not going to pick up again in 2026? How is your workflow and the way that you think about agents getting simpler this year? It's one of the big issues and challenges that we had last year There was very long-running sessions where the agent goes through something called a compaction, where we allow the agent to perform work beyond the context window that is actually available to the model.

15:42And a lot of the complaints were, you know, this is not working well. And this was really part of the scaffolding at the time, where it was just kind of like a heuristic of how you needed to summarize the work done so far, and then reset the context window so that the work could be allowed to continue when given a fresh. And the model was losing context on quite a bit of the work that was performed before. It's really hard to get this heuristic, right? You can try and prompt models. You can try and scaffold your way through it. For a lot of agents out there, this is a very significant part of the complexity of the harness.

16:21And so we decided to solve this at the model level. And, you know, we trained on this like end to end. And now this is something that we receive like almost no complaints on anymore. And, you know, the model is able to, like the agent is able to like work across like 20 windows, 20 context windows, like, you know, for very, very long time horizons without losing track of like, you know, what it was doing before. So that's like something that we just, we removed a whole bunch of complexity just by solving it, you know, in a way that we're uniquely positioned to solve it. And that's the great thing about like, the kind of trifecta of the simplicities of those simple primitives is when one moves and gets a major advantage, when you come back to another one, you realize that you can move it way further because of all these gains are so closely coupled.

17:06Are there any other ways that you're approaching Codex itself for like this year that stand out for that, like will be different, you think, from how you approached it next year? Like what's like the top of the year vision for how we make this like the most popular code agent in the world? Yeah, so there's a couple of things that we're really pushing on. We think last year, single agents became reliable enough that they can perform work. Like this year, we're going to see like reliable multi-agent networks performing significantly more work. What does that mean for the user when you have, you know, maybe an order of magnitude or like two orders of magnitude more, you know, economical, valuable work being produced in, you know, the same time span?

17:51It means that you're able to spend a lot more tokens in the same time, but it also means perhaps you need to review a ton more code or how do you do that exactly. So this is something that we're looking to solve. We're working towards making things significantly faster as well. We feel like we're at the frontier with our models on the intelligence front. We're not at the frontier yet on how fast it can be. We expect these models to get significantly faster this year. And it's almost like a sweet spot that you want to reach of like a level of intelligence at a sufficient speed that, you know, it feels delightful to use in the product.

18:31So that's one thing that we're working towards as well. And the last one I would say is like, you know, it's really this super collaborative personality. Our models, like in our agent, like Codex is known to be a little bit terse, a little bit stubborn sometimes, you know, just really this blunt, pragmatic engineer. that is not the right persona for everyone. Personally, I like to be a little bit more validated in my ideas, not to be told that I'm right when I'm not right, but I want the model to acknowledge that I'm there behind the laptop as well and trying to do something and to be there along the way with me.

19:07And so solving for that is important as well. I know what you mean. I think that's something we've experienced with all of the front foundation models is they all have a different level of politeness, terseness. And like you said, like sometimes too much politeness is just a waste of everyone's time. It's not accurate either. So, but there's like a happy medium there. Like you said, you can't, like what personality works for everybody? No personality. Who's liked by everybody? No one. It's like, so how can you build an agent that's built as liked by everybody? You can't, but you have to make it easy to adapt to what people do like.

19:39That's right. So it's really about making it personal to you and, you know, working in the ways that, you know, you like working. So if you're like highly creative and you like brainstorming and, you know, you don't want to get like pedantic like nits on like the quality of your code is like that should be possible. If you're working on like a super critical code base and, you know, you want every single thing that can go wrong to be flagged to you, then, you know, you should also be able to get that. And that's something that our models are incredibly good at. Codex has been involved in finding some of the most impressive exploits last year, some of the React exploits, which we can talk about.

20:23But for that, you don't need a super friendly personality. You just need Codex to go and figure out what a potential exploit is or fix a very gnarly bug for you and then come back to the solution. And you want to be very confident that it's right. And for others, it's like, you know, you just really want this bubbly, like, you know, super collaborative thing, you know, and it needs to be personal.

20:49AI has changed how we build software, but faster code doesn't always mean faster delivery. That's why Linear B is launching the Essentials Plan, your toolkit for measuring and improving AI productivity. You get the new Linear B MCP server, AI Insights dashboard, and developer surveys, all designed to reveal how AI tools impact delivery speed, code quality, and developer experience. Stop guessing which AI tools work and start leading with data. Visit LinearB.io to learn more and unlock the next chapter of AI productivity. so speaking of like being able to customize and couple things together like you're in an environment where you have the distinct privilege of being like we've talked about connected to the foundation model all the way up to the implementation of the agent and all the research in between so the full stack but for other teams that are building with agentic tools and are in our and are making these kinds of agentic systems like how should they think about like coupling like really tightly coupling to a foundation model versus being more portable in what they build.

21:56And I'm curious, like, how that affects the simplicity or the primitives that we've been talking about. Yes, I think the primitives should roughly be similar, although they might differ in shape a little bit. And I think one of the challenges is, like, if you're building for many foundation models and are completely agnostic to the model provider is you have to find a common ground of all these models. And it's quite inevitable that if you do not adjust at least a little bit, you are going to see the penalty and the overall performance that you can get. And so what we're seeing is we're working with some of the players in the ecosystem and we're advising them on how to make it work very well.

22:45And this is also why our code is open source. is like, it's just all out there. It's an example of how you can get the best performance out of the GPT models. And, you know, we work with them in order to adjust things. So I do think that you need to adjust it to like a certain degree in order to benefit from the models. At some point, if you want to do that for, you know, the thousands of models out there, it becomes like quite prohibitive. And I expect like, you know, major players to like, you know, only do that for like a handful of models. So we've been talking about the agent, But I want us to take a step back and talk about the developer, the engineer, the person sitting at the computer using the agents and what their new normal looks like.

23:27And that's what we've all been exploring in the last year of how this type of agentic coding is impacting developer teams and individual engineers. Different people are getting at this at different speeds, right? And with it, the folks who are experiencing that hyperproductivity, it really blows away all of the code bottlenecks and new bottlenecks emerge. And they're often around like planning and integration, actually taking the time to like sit down and chat with somebody. So, you know, there's a new skill set involved with what makes a developer successful. In a way, each developer almost becomes like their own mini team that they can customize in whatever way works for them specifically.

24:06And they become the representative, almost like the decision manager of this team. You know, what has that been like for your engineering team? How does their day-to-day work transform in that kind of new world? Does everyone just spend less time in an IDE and just lots and lots of time planning and reviewing? Yeah, so interestingly, it actually has brought people closer together. And, you know, we have like more FaceTime. We have more, you know, creative like ideation and planning together because everyone is so accelerated that once you decide on the thing, you can just like almost immediately do it.

24:48And agreeing together and getting organized together, you know, as like a small team, even like of 20 people, it's like it matters a lot because you will be able to achieve like in a week what you traditionally were able to achieve, like, you know, maybe in a month with the team of like, you know, the same size. And so it's kind of like interesting to see that it does bring engineers, to talk more and align more, maybe upfront than before. We're also thinking about, obviously, hey, how can you help and accelerate the planning phase and building consensus and routing things into user feedback and the realities of finding PMF and these things.

25:30One of the bottlenecks that is downstream of that is code review as well. And this is something that we proactively thought about. Last year, we built a custom code review model, which is SOTA. I believe it's still SOTA today. And we deployed it internally across all of OpenAI. And this has been, to my surprise, one of the largest successes within OpenAI for Codex, where it's now pretty much enabled for everyone by default. A lot of teams just require, it's mandatory to have Codex review the PRs because it catches so many bugs. and it's inevitable that if you're generating so much more code, you're also going to produce some amount of code that you then want to catch.

26:18And figuring out what those bottlenecks are starting to become and solving for them one at a time, I think that's the big challenge of 2026. Yeah, I think so too. And those bottlenecks, those are things that are popping up. Those problems, like groundhogs popping out of the ground, right? for these teams to solve because they've already been in motion with each other. They're already working. But there's also this whole other wave of people right now who are finishing up their CS degrees, who have been experimenting with these tools while they've been studying, who are getting their first jobs as engineers, and they're not coming in with all of that baggage.