OpenAI COO Brad Lightcap: GPT-5's Capabilities, Why It Matters, and Where AI Goes Next

0:00GPT-5 is here and OpenAI COO Brad Lightcap is with us to break down the new models capabilities what it means for the AI business, and what's next for this promising technology. Brad, it's so great to see you. Thank you for joining us on an emergency episode of Big Technology Podcast. My pleasure. Thanks for having me. All right. So briefly, I just want you to talk a little bit about what GPT-5 is. So maybe within like 60 seconds or so, can you talk about what it is and how it improves on previous OpenAI models? Yeah. So GPT-5 is our next generation flagship model. It does something really interesting, which is it actually combines into one model the ability to dynamically choose whether to think hard about a problem and reason about it to give you an answer or not.

0:45And so you'll remember previously you had to go deal with the model picker in ChatGPT, everyone's favorite thing. You had to select a model that you wanted to use for a given task. And then you'd run the process of asking a question, getting an answer. Sometimes you choose a thinking model. Sometimes you wouldn't. And that was, I think, a confusing experience for users. GPT-5 abstracts all of that. So it makes that decision for you. And it's actually a smarter model. So you're gonna get a better answer in all cases, regardless of whether you're using the thinking mode or not. And it's vastly improved on things like writing, coding, health.

1:24It's much more accurate, it's much faster. And so all around, we think a better experience. And now for those of us who've been following the hype, I think we probably imagine you would lead with this is an explosive increase in intelligence versus there's a switcher on the model that will go to reasoning or non-reasoning when it makes the most sense. So can you explain like what's the disconnect there and why lead with the usability versus the intelligence increase? Yeah, because intelligence really is a function of how much time the model is going to be thinking. And so depending on how much you want to allocate thinking time to a problem, you're going to get a better answer.

2:03Typically, the longer it thinks, the better an answer it can give you. So when we test the model on certain benchmarks and evals and we allow it to think, it will dramatically outperform any of our existing models by far. Even though if you don't allow any thinking time, you still get a typically better answer than you would for one of our non-thinking models like GPT-4-1. So it is a dramatic improvement in intelligence. It should be, I think, a better quality model across pretty much all dimensions. But that reasoning time and being able to use the reasoning time dynamically to think, we think actually is the important part.

2:36It makes it for a much better user experience. I'm going to parse your words a little bit. You said that it is a dramatic improvement over previous models. Sam, in a press call, said that GPT-5 is a pretty significant step over 4.0. Simon Wilson, who's been using your model for a little bit, says it doesn't feel like a dramatic leap ahead from other LLMs, but it exudes competence. It rarely messes up and frequently impresses me. I'm just setting this up because I'm curious whether we could say or whether you would say that this model is an exponential increase in capabilities or an incremental increase in capabilities.

3:19You know, it's hard to measure it that way. I think we're now kind of into this regime of having to measure intelligence across a lot of different dimensions, which isn't a way to dodge the question so much as it is to explain why GPT-5 is such a special model. And so obviously it's better at the core things that you'd expect it to be better at. It scores better on things like Sweebench. It scores better on all the kind of academic evals that we put it through. This one in particular, we actually made a real emphasis to have it score better on certain health benchmarks. So it's better at medical reasoning and other health related things.

3:52But there's a lot of things that go into what makes a model good now because you have a lot of dimensions to play with, depending on kind of how that model is trained and how it can think about problems. So if it's faster, for example, we think that's actually indicative of it being better. If it can give you a better answer per unit of time thinking, we think that's an improvement that is an important vector to measure also. if it can do things like structured thinking, problem solving, tool use. All these things are things we actually measure, and they're kind of invisible to users. You know, if you're just using ChatGPT, you don't necessarily appreciate each of these things happening under the hood.

4:29But all those things are better for GPT-5 than they were for our previous models. Right, and the reason why I'm asking is because I think a lot of people have pointed to the leaps from GPT, original GPT to GPT-2, GPT-2 to GPT-3, GPT-3 to GPT-4. and one of the things people have seen is just a general increase in capabilities across the board. There were no caveats of like, and maybe there's a reason for those caveats, but there were no caveats of, you know, there's intelligence increases in this place, in that place. It was, we trained a bigger model. I'm pretty sure this is what it was and it's better across the board.

5:04So have things changed? They've changed, yeah, from a technical perspective. I think when you go from GPT-2 to GPT-3, 3 to 4, these were really just exploits of what was and is the scaling paradigm of training larger, pre-training bigger and bigger models, training larger models. It's kind of one vector of training, and you get a better model as a result. And that continues to hold true. But we now have this kind of other category of training, which is post-training and being able to use test time compute in more interesting ways than we used to as almost kind of a second stage of training. And so we think that that actually gives us a little bit of a boost, a force multiplier on our ability to push the model toward new intelligence levels and also be able to train into it a lot of the things that you want an intelligent model to be able to do.

5:55So using tools, for example, is something that re-think is really important for overall intelligence. GPT-2 and 3 couldn't really do that as well. GPT-4 could do it in a more nascent way. And now GPT-5, you get that baked in with the benefit of these kind of multi-step and longer horizon reasoning processes. So yeah, we want to abstract that from users. Obviously, we don't think that you as a chat GPT user should have to stop and think about that. And in some sense, I think the model picker being a point of frustration for people was an expression of the fact that people don't necessarily want to have to make those decisions every time they talk to an AI model.

6:31They kind of want the model to make those decisions for them. And so that's why we think GPT-5 is a big step. And going back to that increasing pre-training, increasing the scale of pre-training, delivering predictable improvements in model performance. Yes, now post-training is in the picture. It's making models better in really impressive ways. But are you of the belief, and is OpenAI of the belief now, that there are diminishing returns from pre-training, given that we're now talking about different forms of training these models? Not at all. Our scaling laws still hold. Empirically, there's no reason to believe that there's any kind of diminishing return on pre-training.

7:13And on post-training, we're really just starting to scratch the surface of that new paradigm. You know, the O series of models, which were kind of the previous reasoning models, were really just the beginning of us starting to explore what's possible in that post-training regime. And I think that's going to be kind of the dominant theme here for the next year or two is continuing to scale in that dimension and continuing to see the gains that you get there simply because they're so significant. And so now we're pushing on two axes for how to improve models, and we think that's going to tighten and condense the rate of innovation.

7:49Zopan A, I believe that the vast majority of improvements from here are going to be coming from scaling or from algorithms? I think it'll be a combination. It's always a combination, right? It's always algorithms, scale, compute, and data, right? And so we push on all three. And they all play a really important role, I think, in how we look at the future. And then the hard part, obviously, is having them come together. So being able to train larger models requires typically that you want to train on more data, obviously with more compute. And so that's a delicate balance between those things because just scaling up doesn't necessarily mean, you know, in all cases that you're going to get kind of the same, you know, corresponding rate of improvement.

8:35You have to be able to bring those other pieces also. So it's not like we push one button or the other. we actually make a really conscientious effort to try and kind of pull all of those together. Okay. And you're not calling it AGI. And I have to say, I've lost a bet on this show because I was listening to Sam on the Theo Von show. He says, he said, GPT-5 is smarter than us in almost every way. And I said, all right, well, that sounds like what you would imagine AGI would be. And then, you know, GPT-5 comes out yesterday or as the release happens, Sam says, I kind of hate the term AGI because everyone at this point uses it to mean a slightly different thing.

9:16But this is clearly a model that is generally intelligent. Help me understand what's going on, because it seems like maybe he wants to call it AGI, but you're not yet. So why is this not AGI? Well, it is a hard thing to define. The joke here is you ask five people what AGI is, you'll get seven answers. And I think the way we kind of look at it is it's a cumulative process, right? It's a system. And I think you have to define kind of what is it that that system is, and what do you expect it to be able to do? And for me, at least, that's a system that is reliably able to learn new things that are kind of out of distribution by virtue of its ability to reason, to think, to solve problems, to use tools, to come up with new ideas.

10:02And so do I think we're at a system that I would call AGI, no. But I think we start to see the traces and the pieces of that overall system for generalized learning start to come together in models like GPT-5, and I suspect in its successors. I don't know if we'll have a point where we are like, okay, we've crossed from a non-AGI world into an AGI world. And even if there were, I'm not sure we'd actually realize it necessarily until after the fact, because one of the things we've learned working with the models that we have is the capability overhang is significant. I think when Sam refers to the intelligence of the models and having a PhD in your pocket, we haven't yet really exploited that as a thing.

10:48In some sense, I think you could pause AI progress right here for 10 years, and you'd still have about a decade worth of new products to get built, of new ways that people We'll figure out how to use the models, even at a GPT-5 level model, in interesting products and interesting processes. And one of the kind of interesting things is I think as the models get smarter, they almost demand more from a product building perspective in terms of how you actually plug them into the system. I always kind of roughly analogize it to like, you could have a really, really smart intern and at the end of the day, they're only capable of doing a few things for you.

11:24They can take notes in meetings. They can write summaries. They can pull basic analyses together. But if you bring a PhD to work, that person has a tremendous capability set that they may not be totally effective on the job on day one, but your job is to really figure out how to expose them to enough context, enough information, give them the right tools to make them really effective later on. And that process actually takes longer to get them to their full effectiveness than it would an intern. And I think it's going to be similar with AI models. And so, you know, it is a continuous process and I don't think it will be linear.

11:57But where we are today, I would say, you know, we're probably not quite yet at something I would call like an AGI level system. Yeah. And it brings up such an interesting question, which is, does it really make sense to try to make the models smarter from here? Or is it about trying to build those ancillary capabilities? You know, I think Sam mentioned this on the media call, but GPT-3, he said, was high school level intelligence, GPT-4, maybe the level of a college student, and GPT-5, an expert. So I guess I wonder for OpenAI, is the quest to add more intelligence to the mix, or is it to focus on capabilities other than smarts?

12:32Some of the things that you mentioned, like memory and continual learning. It's going to be, I think, all of those things. Certainly, there are some unsolved problems. You mentioned a few here, and I would agree with those, that you'd expect a really smart person to kind of comes by default that our models still struggle with. And so there's open research there that we still have to do, I think, to be able to kind of close the loop on what I would call the full spectrum of intelligence. But, you know, there's intelligence like we were talking about earlier in the podcast expresses in a lot of different ways.

13:04And part of it is just your pure IQ. It's your knowledge of how things work and your ability to recall information. But then it's also your ability to reason about how to use other tools to solve problems. It's your ability to be reflective and to look back on your own chain of thought, your own line of thinking, and actually course correct when you feel like, you know, I actually went down the wrong path and maybe I didn't come up with the right strategy to solve this problem. And so that's one of the cool things we see is GPT-5 on those vectors, we can actually reliably measure as better than the previous systems we had.

13:36And for us, I think one of the real world things that we really want to understand is how do they actually perform in, you know, in the real world? How do developers use these models? How do enterprises use these models to actually apply them to existing problems, real-world problems, and see if the next models kind of do better than the last models? And so that's, for us, I think the real-world benchmark is increasingly becoming important as a sign of intelligence relative to the academic benchmarks. And how big of a priority is continual learning within OpenAI? We have a lot of priorities. I think, you know, certainly that's among them.

14:13but we feel really good about our research trajectory. Top priority, middle, low priority. It's hard to, you know, the cool thing about OpenAI is the way that we kind of, you know, I think have like systematized being able to do research. And this has really been true from the early days of the company. I joined OpenAI in 2018 is we take this kind of highly exploratory approach to research. And so we're very much not tops down, I think, in how we approach research where there's one idea and everyone kind of just gloms on to that one idea and we kind of do one thing at a time. What we really do is a lot of open-ended exploration in small teams.

14:50We explore different paths and see if those lead to new ideas that we then kind of cycle back into the kind of core idea, the main line of ideas if they work. And if they don't, we kind of, we recombine those teams into other ideas that seem to be working and then allow other, you know, new ideas to offshoot from there. And so it really is kind of feeling around in the dark a little bit. And when you find that kind of patch of grass that you're like, okay, we might be on the right path here. You kind of bring everyone to that point and then kind of let everyone feel around a little more. And I think that's kind of how it has to work.

15:20I think it's really hard a priori to know these things, you know, in advance. I think you can have intuition. And I think our researchers tend to have kind of, you know, better intuition than the average, but it really is still scientific exploration. Now, I want to talk about whether how your plus subscribers or how the people who are using these chatbots, using chat GPT, will feel the improvements. You know, there's an interesting comment from Ethan Mollock, the Wharton professor who is also experimenting with GPT-5. He says, I think it's a big step forward, but not an unexpected one. If you've been following the curve, he says, these models got gold at the Math Olympiad this week.

15:57I'm losing track of what massive advances mean. All the models are improving very quickly right now. Their question is, if you have a model that's capable of graduate level or college level biology, and then it goes to graduate level biology, the average chatbot user may not feel that even though it's gotten much smarter. So I guess I'm curious how you think this will be reflected, the increased smarts will be reflected in the average user's chat GPT experience and the plus user's experience who've been using these reasoning models for a while? Is it going to feel any different for them? Yeah.

16:39I saw something on X that was akin to what you're describing, which someone basically kind of said, I think for the upper echelon of ChatGPT users who are probably in the paid tiers, who are active on a daily basis and are really kind of expert level using these systems, it's going to feel like an improvement, but maybe a more subtle improvement. But for the average user, for the free user and we're bringing GPT-5 to our free tier, it will feel like a dramatic increase. If you actually look at kind of the way free users have used ChatGPT, most of them have actually not experienced the power of the reasoning models.

17:16They mostly are using GPT-4.0 and, you know, they mostly are kind of using it for this very kind of, you know, turn-based kind of like very quick, you know, back and forth, almost search-like ways that I think don't actually kind of express the full capability of the model. And so for a lot of people, this will be the first time using a model that has reasoning capability. And not only will it be, you know, the first time using it with reasoning, but it'll be the first time that they're experiencing a model making a decision about how long to think about a problem and how good of an answer to give relative to how hard the question is.

17:50And so we expect that like for, yeah, for the average user, it will feel dramatically different. Maybe for the kind of upper echelon of power user, it may not feel as different. So I would agree with that. And I think that's a natural thing. I think that's actually a good thing. That, you know, it is, if you've been following the kind of rate of AI progress and you're kind of exploiting the frontier at every point, yes, it probably is dizzying, but it starts to feel more continuous than if you've kind of, you know, you're using what is basically kind of the best model from a year or two ago. Right, I think you're so spot on about the average user is using it as like a search version of search.

18:28and they're like, well, what should I use when they speak to me? They're like, what should I use AI for? I'm like, just upload stuff and start talking to it about the things you upload. And I had a friend who was uploading pictures of his son's football practice and asking it for tips about, like, for coaching tips. And he was, like, fairly blown away that this thing is giving some, like, real analysis of positioning. I mean, I wouldn't use it as a football coach, but I do think that as the average user gets into these capabilities, it's going to be fairly mind-blowing. Yeah. Everyone's got a little bit of a different entry point.

19:02And that's the cool thing about it is it's really personal for everybody. We focused on health a lot with this release because that was one of the consistently common things that we heard from people as a starting point for how they've used powerful AI was when they're navigating a health journey. And so we really wanted to make an effort on making sure that if people are going to be using AI systems for health-related things, that we could serve them the best possible model. And so that was a big push for training GPT-5. Yeah, you brought up health a couple times. Do you want this to replace a GP?

19:34I mean, a lot of people are really underserved with healthcare, but I kind of worry about handing them a model that can hallucinate and saying, this is the substitute now. I don't think it'll replace GPs, but what I think it helps people do is have more agency in their journey, a little bit more control over the process of managing care. It gives people also just an awareness of the conditions. So we hear stories all the time of people managing conditions that they didn't really understand because no one actually took the time to explain it to them. And that's not because anyone did anything wrong.

20:16It's just because the health care system as it's designed doesn't allow for there to be time to allow people to understand what it is that they're managing. And so even just giving people that baseline of education of like, you know, this is the condition you're managing. It's this common, it's going to express in this ways, you're going to feel these types of symptoms. That's a huge unlock just in people's kind of psychology for what it means to be managing a disease. And, you know, I don't think, I think you still have to kind of work with a GP for care, or a specialist for care. But having something that can kind of handhold you through that journey, I think for a lot of people is really comforting and in a lot of cases has actually proven to be helpful.

20:56Obviously, we want to make sure that model is as accurate as possible. So being able to kind of push the model capability in that domain specifically has been a big area of focus. But we think now with GPT-5 and obviously with future models, we've seen consistently the rates of accuracy and the rates of hallucination go up and down respectively. GBD-5, I think, depends on how you measure it, but it's four to five times more accurate than its predecessors. And that may be more accentuated in health. I don't know off the top of my head. So we have a lot of control, I think, and are pushing in the right direction on being able to make them reliable and accurate.

21:38It's pretty interesting we're talking about things so far beyond the chatbot. Like, of course, there's the chat function, but there's coding, there's health, and of course there's enterprise or the way that businesses use these models. And businesses are notoriously slow at implementing this technology and I'm sure there's so many approvals and reviews and it's tough to get things out the door. But I do think that when you have better models, this is sort of my belief, when you have better models, you sort of are able to push that forward much faster and much more effectively. So talk a little bit about what a better model than GPT-5 will enable on the enterprise front or business front.

22:17Yeah, no, I would agree with your assessment there. I think in many ways, I always kind of say we haven't yet seen the chat GPT moment, I think, in business for AI. I think AI was an amazing tool for consumers where your search space, so to speak, is more narrow and you've got a more constrained problem. You've got obviously a much more narrow context that you're processing. And I think, you know, you can kind of take things turn by turn with very, very few kind of external dependencies. And you really just kind of let the model's pure intelligence shine. Businesses are a different category of difficulty.

22:56So you've got complex business processes. You've got a lot of multi-user dependency. You've got a lot of context that you have to process. You've got a lot of tools that have to be brought to bear. Those tools have to be used in succession in certain ways with certain guardrails. And there's not as much fault tolerance for when they don't work. And so it kind of goes back to what we were talking about earlier. I think you look at models like GPT-5 and the impact that they're going to have in business, it is that baseline of capability that's moved up. It's their ability to use tools, to think in a structured way to solve problems, to kind of recursively correct, you know, their own mistakes, to do long context retrieval, things like that, that actually, you know, these little things do matter on the edge.

23:41And you don't feel them every day in ChatGPT as an individual user, but you will start to feel them as a developer or an enterprise. And so we see this anecdotally too. I mean, we've worked with large enterprises and small startups and the entire spectrum in between on testing these models and GPT-5 specifically before release. And we get a lot of feedback from companies like Uber and Amgen and Harvey and Cursor,

24:10Lovable, JetBrains. All companies that have use cases that are highly, highly sensitive to the model's ability to reliably call tools, to deal with long context, to problem solve and reason effectively. And so it's a rising tide, I think, across the enterprise. And it's just really going to be on the developers we work with to be able to kind of understand the difference and the improvement and then implement them in the applications that they're building. Yeah, it is interesting to know that you have been already working with many companies and letting them use GPT-5 already. So has there been a sort of unified, we couldn't do this with the previous models, but we can do it now with GPT-5?

24:56Or is it sort of spread out in terms of the capabilities that it's now enabling? I would say it's been rising tide across the board. So everyone who's kind of benchmarking and all the companies that we work with typically now are pretty accustomed to evaluating and benchmarking performance across all the models that they use. But everyone has kind of reported much higher, kind of consistently higher performance on those evals. There are a few areas in particular we've seen spikes. So one is coding for sure. I mentioned companies like Cursor, JetBrains, Windsurf, Cognition, and others that we work with who anecdotally have all said that GPT-5 now feels like the most capable coding model, whether that's in an interactive coding environment or more of an agentic coding environment.