Exclusive: How GPT-5 Actually Works

0:00This is an iHeart Podcast.

0:58We'll see you next time. at washablesofas.com. Authors are subject to change, and certain restrictions may apply. This week, on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer. I first found out about my cancer on my birthday at the age of 45. I found out my cancer has spread to my pelvic bone, and from there, life just changed. About one in eight men will be diagnosed with prostate cancer during their lifetime, and the risk increases with age. Anything with cancer, you just think death sentence.

1:39And the only thing I could think about was, who's going to take care of my family? You have to go out there and build your support system. You got to build your team. In this episode, we'll explore the science behind detection, along with the practical steps men can take to protect their health. Listen to Health Discovered on America's number one podcast network, iHeart. Open your free iHeart app, search Health Discovered, and start listening.

2:09Parking shouldn't slow you down. ParkWiz gives every driver a shortcut. Book ahead, save up to 50%, and skip the hassle of circling the block. Park smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park. In business, they say you can have better, cheaper, or faster, but you only get to pick two. What if you could have all three at the same time? That's exactly what Cohere, Thomson Reuters, and Specialized Bikes have since they upgraded to the next generation of the cloud. Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds.

2:57How is it faster? OCI's block storage gives you more operations per second. Cheaper? OCI costs up to 50 % less for computing, 70 % less for storage, and 80 % less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds. This is the cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free. Head to oracle.com slash strategic. That's oracle.com slash strategic. Call Zone Media Hi, my name's Ed Zitron and welcome to Better Offline. This is also Jackass.

3:52So you've just had a cheery two-part chuckle first about how generative AI may tank our markets and our economy. So I'm going to give you a lighter one. An episode about GPT-5, which is a model from OpenAI, and why just under three years of hype have led to the software equivalent of the launch of St. Anger. Except every time Lars Ulrich hit the snare drum, it cost them$55 ,000. Now if we look at the positive reviews, we see takes ranging from Simon Willison's tepid remark that GPT-5 is just good at stuff, to Semi-Analysis's completely insane statement that GPT-5 is setting the stage for ad monetization and the OpenAI GPT, ChatGPT super app, in a piece that makes several assertions about how the router that underpins GPT-5 is somehow the secret way that OpenAI will inject ads, which is just distinctly silly it's i'll get into this in the episode a little bit but just with everything you're gonna hear you're gonna realize this is just someone just saying stuff took four bylines to do that shit too i'm also british i'm gonna say router i might say router as well because i've been here a while make fun of my voice if you really must but with that out the way here's a quote from seminary analysis's coverage before the router there was no way for a query to be distinguished and after the router the first low value query could be routed to a GPT-5 mini model that can answer with zero tool calls and no reasoning.

5:12This likely means serving this user is approaching the cost of a search query. This does not make any sense. None of this makes it like it's just a bunch of assumptions. Why would this be the case? The article also makes a lot of claims about the value of a question and how ChatGPT could, I am serious, agentically reach out to lawyers. I'm not going to edit that out because agentically is not a a fun word to say, is just complete nonsense. And in fact, I'm not sure this piece reflects how GPT-5 even works at all. Again, quoting it, the router serves multiple purposes on both the cost and performance side.

5:47On the cost side, routing users to mini versions of each model allows OpenAI to service users at a lower cost, or with lower costs even. To be fair, on semi-analysis, it's not as if OpenAI gave them much help. OpenAI's official writings about the router aren't exactly filled with details, talking in glowing terms about what it does, but not how. Here's what they say. ChatGPT's real-time router quickly decides which model to use based on the conversation type, complexity, tool needs, and your explicit intent. For example, if you say, think hard about this in the prompt. The router is continuously trained on real signals, including when users switch models, preference rates for responses, and measured correctness, improving over time.

6:27Once usage limits are reached, a mini version of each model handles remaining queries. In the near future, we plan to integrate these capabilities into a single model. And that last bit really doesn't make sense, but in any case, the launch of GPT-5 has been very, very weird. At first, some people seemed really happy about it. Chief of them, software YouTuber Theo Brown, who has over 468 ,000 subscribers. He's also known as Theo GG, who said, I didn't know it could get this good. This was kind of the, like, oh, fuck moment for me. in a lot of ways. And I've had to fight like a slow spiral into insanity.

7:07It's a really, really good model. He finished by saying, and keep an eye on your job because I don't know what this means for us long term. Pretty crazy, right? Comments on the video included people saying things like, if OpenAI is holding you hostage, blink twice, and yes, that is an adverbative quote. Another saying, this dude is everything wrong in IT today. Another saying, this video was sponsored by OpenAI, another saying, GPT-5 failed every test project I gave it today, it's a lie in my experience, maybe they haven't ramped up the GPUs. Now, from what I can tell, Theo Brown played with GPT-5 in OpenAI's offices and did all the benchmarking there.

7:44OpenAI, by the way, fucking hell, come on, you can't benchmark in their offices. Anyway, OpenAI's API-based access to GPT-5 models, you know, the thing that you use if you want to integrate GPT into your app does not root them, by the way, nor does OpenAI offer access to its router or any associated models. Important detail. Just want you to know that because we need to make sure we're very clear. Now, a week later, Theo Brown would put out another video called I Was Wrong About GPT-5, which he would open by saying, so first and foremost, I want to make sure it is very, very clear that the experience that you probably are having with ChatGPT and GPT-5 right now is not the experience that I had when I was first testing it.

8:25Brown goes on to explain that he was not paid by OpenAI at all, that he was sincerely impressed by the company and GPT-5, and that he'd actually spent over$25 ,000 in inference testing it on his own company's software, and indeed also that he turned down a grand appearance fee. Sorry, I mean, that's a very British thing,$1 ,000 appearance fee, not just like a really nice one. Brown claims he asked OpenAI to try it out, and after they declined to let him test it early on his own, he was invited to try it on camera with a small group of other people at OpenAI's offices, where they'd film his reactions.

8:57He said that the API was incredible, but that it's become apparent that the models he was using in the video were not the same as those released to the public, making a post on August 13th on XDEverything app that GPT-5 was nowhere near as good in cursor as it was when he was using it a few weeks ago, complaining that things that worked while demoing it at OpenAI no longer did, adding that there was somebody else on Twitter that said they had a similarly great experience with GPT-5 on launch that has since decayed. It isn't completely clear what happened here, but I'm going to guess that OpenAI showed Theo Brown and others in their offices some sort of heavily modified version of the model that burns significantly more compute to provide its outputs, though I'm also very suspicious of how significant the difference is here.

9:40Brown's videos attempt to show the difference between the generations that he received from model when it was good and when it was bad in this video, which I'll include a link to in the episode notes. But if I'm honest, they look pretty similar in that they're kind of mediocre. I'm not saying that as a hater, by the way. They just kind of look like shit. It's just kind of, okay, like shit. They look like regular fucking generated websites. They don't look special. The good one is fine and the bad one has weird gradients on it. This whole thing sucks though and was a clear set up by OpenAI to overstate the abilities of GPT-5, one that fell apart with the lightest brush with reality.

10:15I imagine their assumption was that Brown would post a glossy video and then walk away, and it gave Theo some credit for straight up stating he was misled. This was a desperate move, and one that blew up in the face of OpenAI along with the rest of the GPT-5 launch. People hate the model. Customers are mad at OpenAI for taking models away like 4.0, and have remained mad even with their return. And the chat GPT subreddit is almost entirely people complaining about how ineffective the new version is and how even GPT-4-0 is not the same. They got game of brain, baby. As I said in last week's monologue, I believe OpenAI has grown a fandom rather than any kind of sustainable product market fit, and they're now suffering fandom-like hate, with every minor change they make in an attempt to push GPT-5 further, further aggravating people that barely understand why they use the product to begin with.

11:01Yet at the center of the anger laid the reason for GPT-5's launch. The belief that this was somehow a cost-cutting measure where OpenAI had added a router to ChatGPT as a means of sending certain requests to cheaper models to save money. But when I hear router, I hear latency, and I never, for even a second, believed that this would somehow be cheaper to run. It didn't make sense. I'm a curious little critter, so I went and found out how ChatGPT-5 actually works. And unlike the following incredible products that you should buy, it's actually kind of a big piece of shit.

11:41Life's messy. We're talking spills, stains, pets, and kids. But with Anabay, you never have to stress about messes again. At washablesofas.com, discover Anabay sofas. The only fully machine washable sofas inside and out. Starting at just$699. Made with liquid and stain resistant fabrics. That means fewer stains and more peace of mind. Designed for real life, our sofas feature changeable fabric covers, allowing you to refresh your style anytime. Need flexibility? Our modular design lets you rearrange your sofa effortlessly. Perfect for cozy apartments or spacious homes. Plus, they're earth-friendly and built to last.

12:23That's why over 200 ,000 happy customers have made the switch. Upgrade your space today. Visit washablesofas.com now and bring home a sofa made for life. That's washablesofas.com. Offers are subject to change and certain restrictions may apply. This week on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer. I first found out about my cancer at the age of 45. Anything with cancer, you just think death sentence. In this episode, we'll explore the science behind detection, along with the practical steps men can take to protect their health.

13:06Listen to Health Discovered on America's number one podcast network, iHeart. Open your free iHeart app, search Health Discovered and start listening. So I've shopped with quints before they were an advertiser and after they became one. And then again, before I had to record this ad, I really like them. My green over shirt in particular looks great. I use it like a jacket. it's breathable and comfortable and hangs on my body nicely. I get a lot of compliments. I liked it so much. I got it in all the different colors, along with one of their corduroy ones, which I think I pull off. And really, that's the only person that matters.

13:36I also really love their linen shirts, too. They're comfortable, they're breathable, and they look nice. Get a lot of compliments there, too. I have a few of them. Love their rust colored ones as well. And in general, I really like quints. The shirts fit nicely and the rest of their clothes do, too. They ship quickly. They look good. They're high quality, and they partner directly with ethical factories and skip the middleman. So you get top tier fabrics and craftsmanship at half the price of similar brands. And I'm probably going to buy more from them very, very soon. Keep it classic and cool this fall with long lasting staples from Quince.

14:06Go to quince.com slash better for free shipping on your order and 365 day returns. That's Q-U-I-N-C-E dot com slash better. Free shipping and 365 day returns. Quince.com slash better. Parking shouldn't slow you down. ParkWiz gives every driver a shortcut. Book ahead, save up to 50%, and skip the hassle of circling the block. Park smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park.

14:40And we're back. And from here on out, I will define two things. GPT-5, referring to the model and its associated mini and nano models, and ChatGPT-5, referring to the current state of ChatGPT, which features an auto, fast, and thinking, and thinking mini model selections. You also can see legacy models, but that's not what we're talking about today, and that's also only for a little bit. It's a distinction I have to make, by the way, and make early, because the two things are different. They work in different ways, and ChatGPT5's structure introduces a bunch of trade-offs and downsides that, as I'll discuss later, make this whole thing even more wasteful.

15:14In discussions with a source and infrastructure provider familiar with the architecture, it appears that ChatGPT5 is in fact potentially more expensive to run than previous models, and due to the complex and chaotic nature of said architecture, can at times be on upwards of double the tokens per query. Tokens, for those who don't know, are basically chunks of text that the AI models do stuff with. I'm simplifying this, do not email me and correct some minor thing, nobody cares. A sentence like the quick brown fox jumps over the lazy dog will be broken into lots of smaller four-character chunks.

15:45There are different kinds of tokens, and they're all priced differently. An input token refers to the data you send to the model when you ask it a question. Output tokens are used to measure the size of its response with bigger responses, requiring more tokens. The more tokens you burn per query, the more expensive it is to run that query. The fact that ChatGPT 5 can, in certain circumstances, burn twice the number of tokens per query means that every question costs more. ChatGPT is also significantly more convoluted, plagued by latency issues, and is more compute-intensive thanks to OpenAI's new smarter, more efficient model routing system.

16:18In simpler terms, every user prompt on ChatGPT, whether it's in Auto, Fast, Thinking, or Thinking Mini, starts by putting the user's prompt before the static prompt. I don't want to lose you here. This is important. A static prompt is the invisible instructions given by OpenAI to ChatGPT and the models themselves and the tools associated with them to tell them how to operate. Instructions like, you are ChatGPT, you're a large language model, you're a helpful chat bot, do not threaten them with a knife, and so on and so forth. These static prompts are different with each model you use. a reasoning model will have a different instruction set to a more chat-focused one, such as think harder about a particular problem before giving an answer, break down problems into component answers when you get a certain thing, like if someone asks you a coding question, query a coding tool, that kind of thing.

17:02A user prompt is exactly what it sounds like, the thing that a user wants the AI model to do. The new order in chat GPT-5 becomes an issue when you use multiple different models in the same conversation, because the router, the thing that selects the right model for the request, has to look at the user prompt. It can't consider static instructions first because they may be different based on what the user asked. In fact, the order has to be flipped for the whole thing to work. Put simpler, previous versions of ChatGPT would take the static prompt and then invisibly append the user prompt onto it.

17:31This static prompt would typically be cached, massively reducing the amount of compute the model needs to perform a task. ChatGPT cannot do this. Every time you use ChatGPT 5, every single thing you say or do can cause it to do something different. Attach a file might need a different model. Ask it to look into something and be detailed might trigger a reasoning model or a different depth of reasoning. Ask a question in a weird way. Sorry, the root is going to need to send you to a different model entirely. Each time coming up with new instructions based on the subtle interpretation of what you asked it.

18:02Every single thing that can happen when you ask ChatGPT to do something may trigger the root to change model or request a new tool, and each time it does so requires a completely fresh static prompt, regardless of whether you select auto-thinking fast or any other option on ChatGPT. This in turn requires it to expend more compute, with queries consuming more tokens compared to previous versions. It's like you start at a job and every time you do a task, write an email, make a copy, attend a meeting, email someone with a threat, your workplace requires you to complete the entire mandatory onboarding training first.

18:35Wanna edit a spreadsheet? Not before you brush up on your anti-bibriary legislation first, you prick. As a result, ChatGPT may be smart, but it doesn't really seem efficient in the GPT-5 version. Now, to play devil's advocate, OpenAI likely added the routing model as a means of creating a more sophisticated output for a user, and I imagine with the intention of cost-saving. Then again, this might just be the thing it had to ship. After all, GPT-5 was meant to be the next great leap in AI, and the pressure was on to get it out the door. By creating a system that depends on an external routing model, likely another LLM in this case, OpenAI has removed the ability to cache the hidden instructions that dictate how the models generate answers in ChatGPT, creating massive infrastructural overhead.

19:18Worse still, this happens with every single turn, as in message, on ChatGPT 5, regardless of the model you choose, creating endless infrastructural baggage with no real way out that only compounds based on how complex a user's queries get or how much they change. They could be simple, but just going in different directions every time. Could OpenAI make a better router? Sure. Does it have a good one today? No. Every time you message ChatGPT, it has the potential to change model or tooling based on its own whims. Each time requiring a fresh static prompt, and short of totally reworking the architecture of ChatGPT5, there's no way to change this.

19:51And if it's an LLM choosing which model, I don't know, maybe it hallucinates. Just a guess. It doesn't even need to be the case where a user asks chat GPT-5 to think. And based on my tests with GPT-5, sometimes you can just ask it a four-word question, and it will think about it for no apparent reason. OpenAI has created a product with latency issues and an overwhelmingly convoluted routing system that's already straining capacity, to the point that this announcement feels like OpenAI is walking away from its API entirely. This, as a reminder, is the thing that people use to incorporate OpenAI's models into their apps, while also running said models on the infrastructure OpenAI rents from Microsoft, and CoreWeave at some point, as well as Oracle.

20:31And this API thing is really weird, by the way, because these are new models, but OpenAI is really not talking about the models themselves that much. Unlike the GPT-40 announcement, which mentions the API in the first paragraph, the GPT-5 announcement has no reference to it, and only has a single reference to developers at all when talking about coding. Sam Orman has already hinted that he intends to deprecate any new API demand, though I imagine he'll let anyone who will pay for priority processing, which is essentially OpenAI's way to require minimum commitments and extra payments from API customers, just so they never feel the bite of any compute shortages and throttling, which they absolutely will do to people that don't pay.

21:09ChatGPT5 feels like the ultimate comeuppance for a company that has never been forced to build a product, choosing instead to bolt increasingly complex tools onto the side of models in the hopes that one will magically appear. Now, each and every feature of ChatGPT burns more money than it ever did before. ChatGPT5 feels like a product that was rushed to market by a desperate company that had to get something out the door. In simpler terms, here, it's actually really funny. When I worked this out, I chuckled vigorously. This is just a case where OpenAI has given ChatGPT a middle manager. But now I'm giving you the chance to open up your hearts and do something better.

21:42Open up your wallets too and send money to a company that follows here. Behold my advertisements.

22:24We'll see you next time. real life. Our sofas feature changeable fabric covers, allowing you to refresh your style anytime. Need flexibility? Our modular design lets you rearrange your sofa effortlessly. Perfect for cozy apartments or spacious homes. Plus, they're earth-friendly and built to last. That's why over 200 ,000 happy customers have made the switch. Upgrade your space today. Visit washablesofas.com now and bring home a sofa made for life. That's washablesofas.com. Offers are subject to change and certain restrictions may apply. This week on a very special episode of Health Discovered, we're taking a closer look at a condition that affects hundreds of thousands of men each year, prostate cancer.

23:11I first found out about my cancer on my birthday at the age of 45. Found out my cancer has spread to my pelvic bone. And from there, life just changed. About one in eight men will be diagnosed with prostate cancer during their lifetime, and the risk increases with age. Anything with cancer, you just think death sentence. And the only thing I could think about was who's going to take care of my family. You have to go out there and build your support system. You got to build your team. In this episode, we'll explore the science behind detection, along with the practical steps men can take to protect their health.

23:53Listen to Health Discovered on America's number one podcast network, iHeart. Open your free iHeart app, search Health Discovered, and start listening.

24:05Parking shouldn't slow you down. ParkWiz gives every driver a shortcut. Book ahead, save up to 50%, and skip the hassle of circling the block. Park smarter, park faster. ParkWiz. Download the ParkWiz app today and save every time you park. Hey, it's Ryan Reynolds here from Mint Mobile. Now, I was looking for fun ways to tell you that Mint's offer of unlimited premium wireless for$15 a month is back. So I thought it would be fun if we made$15 bills. But it turns out that's very illegal. So there goes my big idea for the commercial. Give it a try at mintmobile.com slash switch. Upfront payment of$45 for a three-month plan equivalent to$15 per month required.

24:45New customer offer for first three months only. Speed slow after 35 gigabytes of networks busy. Taxes and fees extra. See mintmobile.com.

24:54And we're back. Like every great middle manager, ChatGPT5's router creates more work based on its own interpretation of what's going on. And as a separate large language model, I can't imagine it has a ton of training data available. If I had to guess, and this is a guess by the way, OpenAI has done and will do a lot of fine-tuning and reinforcement learning to make it work. Though to give it a little grace, this is a new thing that it's doing, and it's doing sort of a huge scale. The problems start, by the way, with the fact that ChatGPT5 is taking the user's initial prompt and then deciding which model to use.