An exclusive inside look at GPT-5

0:00GPT-5 is the newest model released from OpenAI and from my very first interaction I felt like this was a engineer built by engineers for engineers it writes good code it refactors it's thoughtful and girlfriend loves to call a tool if you have a good idea and you really just need to get down to what are the technical implementation of this feature I think GPT-5 is tremendously better at that than GPT-4 which again is like actually pretty light on functional requirements if your use case is getting things to humans like business users or stakeholders you might like a GPT-4 104103 output. A little bit more business-oriented, really no complaints.

0:39It's exceptional at coding. This is a highly technical model. I think it's going to be a daily driver for lots of folks.

0:49Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive here on a mission to help you build better with these new tools. Today, I'm doing something a little bit different. I'm walking you through the newly released GPT-5 model from OpenAI and giving you my honest takes on a couple workflows that I personally use. We're going to look at GPT-5 for product managers and engineers, investigate some stylistic choices that the model has made, and also go through a couple personal workflows that I find useful and see if side-by-side GPT-5 outperforms other models. Let's get to it. To celebrate 25 ,000 YouTube followers on How I AI, we're doing a giveaway.

1:33You can win a free year to my favorite AI products, including V0, Replit, Lovable, Bolt, Cursor, and of course, ChatPRD by leaving a rating and review on your favorite podcast app and subscribing to YouTube. To enter, simply go to howiaipod.com slash giveaway, read the rules and leave us a review and subscribe. Enter by the end of August and we will announce our winners in September. Thanks for listening. So before we get into how this model performs, let's talk about what the model is. GPT-5 is the newest model released from OpenAI and they were generous enough to give me a little bit of early access to play with the model and really start to understand its strengths and weaknesses.

2:20And from my very first interaction with GPT-5, I felt like this was a engineer built by engineers for engineers. This is a highly technical model, both in capabilities and style. And this is going to be one that you're really going to reach for on a daily basis if you are coding, testing the technical bounds of these LLMs or solving deeply complex problems, but it might have some pieces for the business thinkers out there, the product owners out there that might not work for your use case. And we're going to show exactly what I mean by that in just a second. Now, I have been pretty familiar with the OpenAI ecosystem for quite some time and have been using the OpenAI models almost exclusively for my own product, ChatPRD.

3:03That being said, I do work with a variety of models and model providers in my day-to-day workflows. So when I'm coding using Cursor, I'm often using Clawd4, Clawdsonnet4, Gemini 2.5, O3 from OpenAI. In ChatPurity, I again are using a lot of the OpenAI models, 4.0, 4.1, even did a little test with 4.5 when that first came out. And I use a variety of different out-of-the-box AI tools as well. So I'm using ChatGPT relatively often, occasionally go into Claude, have my whole stable of different AI coding tools, which again, choose and fine-tune their own models. So I do feel like I'm pretty familiar with the model ecosystem, at least the commercial model ecosystem, and have really developed a sense of where these models perform well for specific use cases and where they don't.

3:58And I'm the kind of user and AI power user that really selects the model for the use case. So I was really excited to get access to GPT-5 because I wanted to know the answer to the question, which is, where does this model fit on my team? I don't think of myself as a single model employer. I really think of models as part of a team and tools as part of a team. And each model has their own personality and capabilities. Each tool has its own personality and capabilities. And I think that rather than think, is this an upgrade? I think, is this an addition to my team? And where would I put them into play?

4:34So the first thing that I did when I got access to GPT-5 is I went straight to the use case that I know, love, and think about the most, which is actually ChatPRD and our core chat and document generation implementation. It's a common use case for product managers using AI to generate product requirements documents. It's a place where I've spent a lot of time prompt testing, model testing, and really optimizing the experience for both matching the stylistic tone I want for the product, as well as getting great user feedback on outputs. And we've really A-B tested this pretty significantly into depth in Chat Purity and landed most recently on GPT-4.1 and a variety of tools and prompts being the best stack for our users.

5:22And in July, we had a 96 % satisfaction rate with our documents. So that's how I'm really thinking about it. I'm thinking what model is highest performance, cost really doesn't come into play, but it will later and then do users love it. And I consider myself a proxy for the product manager and engineering users. So I feel like I have a pretty good sense of what will perform well in this use case and won't. So when I got access to GPT-5, what I did is I went ahead and used LaunchDarkly AI configs, which left me on demand, switched the model that I'm using in local or production, and I started testing GPT-5.

5:56And what I'm going to show you on my screen right now is really a side-by-side representation of the results. So GPT-4-1, our core model that we use on chat PRD is on the left, and GPT-5 is on the right. And a couple things right out the gate that I noticed, and in fact, I had to prompt around is GPT-5 when I first tested it, it spoke like a developer. This is actually tuned a little bit for prompt on the right side. It just wanted to write me markdown bullet point lists. And I gave that feedback to the OpenAI team, did a little bit of prompt engineering, and I think it's a little bit more natural language when you speak to it.

6:38But you're definitely going to see GPT-5. She loves the bullet point list. So we're going to get lots of bullets and we're going to call lots of tools. That's what something you're definitely going to see in this episode. But if you look at it side by side to start off, they are pretty similar responses. And I think that's really a representation of they share the same system prompt in context in chat purity. So this is exact same system prompt, exact same context. It's coming back and it's just really asking me questions about what I want to achieve with my product when I ask it to brainstorm new features.

7:10Now, where you start to see it diverge is what it starts to focus on when asking to brainstorm new features. And so if you look at GPT-4-1's response here, the questions are really about business impact. You get a lot of discovery around what metric you want to change. Who is your persona? What is your business goal? And I've noticed that throughout my side-by-side evaluation. This is just one example. GPT-4.1 and some of the older models just came at the problem from a more general, but more business-oriented lens. But GPT-5 on the right really came to features quickly. And I think this is an important point for product managers to note because you know us product managers, we love to ask a good why.

8:06And we really love to understand the problem. And what you see in GPT-5 is a jumping to the solution. And I think that's a reflection of the way it was trained and the place that GPT-5 fits in the sort of ecosystem of open AI models. It's very clear that the coding model wars are heating up, that the IDE wars are heating up, that the coding tool wars are heating up. And this model really feels like an answer for engineering use cases more than anything. And what I thought was interesting is we'll get to those engineering use cases. I think it's quite exceptional at writing code. But that sort of angle into execution of engineering tasks even bleeds into the conversational aspect.

8:52of the model. And so you can even see the point of view of the model, if you can call it that, is really different from 4.1, which we're using on the left, which really comes from a business point of view. You'll see very quickly GPT-5 is getting to an execution engineering point of view. So it's just something to consider as you look at these models side by side, what you're really going to get out of them and where they might be most applicable in your use case. And so right Right off the gate, we're seeing 401 be more business oriented, Fivo be a little bit more technically oriented. And then I ask it to focus on free to paid conversion.

9:28And again, we get pretty similar ideas. So again, this isn't the most radical product area to focus on. It's well trodden, well documented. You know, both of these models probably have access to best in class growth tactics. So you'll see the kind of features be very similar across the two. But if you really inspect, you will see that the description of the features for 4.1 on the left are much more user centric and much more business centric. So it's really like a who, why question. If you look at GBD5, again, I find this so fascinating. It's really a what, how answer. And I think that really sums up how I would say my interactions with this model has been.

10:19You still get a little bit more of that like business user discovery from, you know, 4.1 or 4.0, 0.3 even. QBD5 is like, tell me what to build. Tell me exactly how the features work. Give me numbers. Give me user stories. Give me something to code. And so I just thought it was really interesting to see that the ideas themselves, again, pretty similar. but the way those ideas are executed are very different. And you'll start to see the chats branch here and you'll start to see the GPT-5 chat really branch into wanting to get into technical code, which has its pros and cons. And you'll really see the GPT-4-1 model really stay in this business kind of like high level mindset.

11:01And so as an app builder focused on product managers, what am I thinking to myself? I'm thinking, well, my product's a product manager. It needs to talk to engineers, but it's a product manager. And so I'm unsure if my users are going to love GPT-5 because it skips that step of product management thinking and gets right to what to build, which again, engineering side of my brain loves. So I'm going to pull these docs up side by side and really show you what the PRD that got generated from each of these models looked like. And again, pretty similar prompts, pretty similar inputs. You can see right out the gate.

11:37I mean, I told you it's an engineer for engineers. It tried to put this code block comment at the top of the document. Again, just a pure signal. This is, you know, trained to write technical documents and trained to write code. Even when you tell it to write like a prose document, like a PRD, you see artifacts like this, which are code based, which I find very, very, very interesting. And so if I'm looking at these, these PRDs side by side, a couple of things that you're going to notice. GP5 writes more. It is significantly more detailed in its content. And I think there are pros and cons to that.

12:17I think when you're trying to define something for a engineer or a coding agent to execute, the more detailed you can get, the better. When you are trying to align stakeholders as product managers or other business users might need to do, sometimes a level of detail too far can actually obscure the primary message that you're trying to get across. And so I'm looking at these side by side. And I'm really thinking, do I want five business goals for this product? Are these the right business goals? And are they artificially too precise on the GPT-5? Or are they like perfectly precise? And so it was just something that I observed and looking at these side by side.

13:01Now, if we scroll down, really interesting. Again, the personas are a lot more detailed. There are more of them and the use cases are very specific. But on the GPT-5 model, the use cases are very feature centric. And on the GPT-4 model, they're very like what I'm trying to achieve as a user specific. And so I thought it was really interesting to just kind of compare and contrast both of these. Again, GPT-5, very detailed. Where I love GPT-5 and prefer it over the 4.1 model is the functional requirements are exceptional. The formatting got a little weird, but you can see here there's a prioritized list in a table.

13:49There's lots of details about soft warnings, hard warnings. I mean, these are the kinds of things that the best engineers are going to ask you about how this stuff works. And so if you have a good idea and you really just need to get down to what are the technical implementation of this feature, I think GPT-5 is tremendously better at that than GPT-4, which again, it's like actually pretty light on functional requirements. I think you could say the same for user experience. Again, you're just going to get a lot more detail out of GPT-5 in terms of describing the user experience in prose. And so if you are using any of the prototyping models, like a V0, a Lovable, a Bolt, a Magic Patterns, whatever those might be, the more specific you can be about describing the user experience in prose, the happier you're going to be with your prototype.

14:39And I think 4.1 is actually pretty high level, and five is is is pretty exceptional at that now the narrative is an interesting interesting one you know gpt5's a little longer i will say like it's not a terrible writer so i don't think that its prose is necessarily cold or not compelling or not lyrical which are things as somebody who has a liberal arts degree i really care about it's just a little bit more detailed and i think you know writing shorter prose is also a virtue and so you really need to think about do you need as many words is simpler better are the details really valuable here versus in in another version now again another place where I think GPT-5 obviously outperforms for one in a side-by-side is technical consideration so if you are an engineer and you need to write a tech spec I would highly recommend GPT-5 over any of the other models that I tested.

15:41It is just very specific. It speaks in the language that an engineer would understand. It's really detailed in its analysis of requirements. And so I do think it is a really nice technical writer. And I think engineering teams, docs teams are going to be quite happy with it. I honestly think product managers might not need to be writing this part of a PRD. So maybe there's a division of labor here that happens naturally or in your AI tools. But again, GPT-5 is really going to outperform on technical considerations and detail across the board. So that's a side by side. But these PRDs don't operate in a vacuum.

16:22They are artifacts generated for another purpose. And so what I wanted to do is actually generate a prototype based on those different PRDs. So if we go back to my general analysis, I thought that GPT-4.1, business-oriented, higher level, maybe easier to read as a reader because it's not so dense, not as technical, not as detailed. GPT-5, engineer, engineer, engineer, very detailed, perhaps overly so. But the real question is, do I get a better prototype one shot out of those prompts versus another? And this is where I think things get interesting. because I would say to you, if your use case is getting things to humans, you might not want to, and those humans are not engineers.

17:08Engineers, I love you. You're humans, but I'm going to put you in a different category for just the sake of this argument. If you are trying to get this to business users or other stakeholders in your company, you might like a GPT-404103 output. a little bit more business oriented, a little slightly more condensed, easier to read, not so much excessive detail. If you're trying to get this to an engineer, I think you're going to be happier with a GPT-5. And so what's interesting about the side-by-side is honestly for a prototype and visual style, I like what 4.1 prompting did into this is our V0 integration.

17:47I like what 4.1 prompted into V0 and the outcome here. It's colorful, it's clear, I understand, you know, what's happening here. I think this looks nice. Meta observation, I could not get V0 via GPT-5 to generate color. It's like all very gray and blue, but you can see on the left side with 4.1, for whatever reason, whatever prompt was behind the scene, which I'll have to go look at, we got a little bit more color and a little bit more design. It's much simpler. It looks nice. It's visually appealing. but I feel like GPT-5 over here on the right gave me, and I'm just gonna make it a little bigger so you all can see, gave me a lot more to work with.

18:30And what I mean is I tend to think of these prototypes as inspiration for implementation, not implementation itself. So I'm never like gonna ship this. This is not what chat purity looks like. It's not what our product looks like. But I'm really looking for ideas on upsells and free to paid ideas. And I just think the fact that they put so much detail into the PRD means they put so much into the prototype, which means I have a lot of components to choose from when I really want to make my product better. And so I have locked spaces. I have upgrade widgets. I have free trial details. I have I'll try it later.

19:12I have upgrade now. But I mean, I just have there is just as much in here as I want to pick. And when you're looking at prototypes as an ideation space. Honestly, I think taking a abundance mindset and generating as much as possible and be like, I'll never use that. Oh, I like this is a lot better. And so I think the verbosity of GPT-5 in terms of technical specifications and user experience actually output more interesting ideas when given to a prototyping tool. So that was a really interesting observation for me. I wasn't sure that I would love it. And I actually didn't love it on first passed.

19:47But once I started to click through, I was like, man, it really thought of a lot here. And I think that's because it was given quite a bit of detail. So that's just one little side by side on prototype generation. I want to give you one last observation in the specific chat PRD use case, which I found quite interesting, which is, I gave it a copy of our homepage, and I asked it to change things. And this is what I find interesting. As much as I thought that GPT-5 was a pretty cold, straightforward, detailed engineer, GPT-4 was much, 4.1 was much meaner to me. It was much more critical. And I thought that was kind of interesting.

20:35GPT-4.1 starts out, and this makes me feel bad about my homepage, but just says, not up to standard, very straightforward. room. GPT-5 was like, that's pretty good. Areas to improve. And what's interesting about the instructability and promptability of the model is I actually went back and gave it another pass and said, could you be a little bit more critical of my homepage? Same prompt. And again, GPT-4-1 was legitimately, legitimately critical, cruelly critical if you look at it. And GPT-5 really, again started with like the shit sandwich excuse pardon my french but it really started with here's what's not working or here's what's working here's what's not working but like you can make it better and and I think this is interesting one of the things that you really have to test as an application builder is working with LLMs is can you tune it via prompts effectively now again these two side-by-sides are using the exact same prompts I have not prompted to the strengths and or weaknesses of GPT-5.

21:42I've just simply been giving it similar side-by-side content, context, and prompting. And it was just really interesting to see how you can massage the LLM responses to meet your needs. So my general conclusion remains the same through the side-by-side, which is functionally, this thing is built to code and this thing is built to help you code. And you're going to be very happy with the strengths of that, but it might have some drawbacks on the other side, especially as an application developer, a business user, and then we'll get to it. I actually think it's got some strengths from the consumer perspective.

22:17Today's episode is brought to you by ChatPRD. I know that many of you are tuning into How I AI to learn practical ways you can apply AI and make it easier to build. That's exactly why I built ChatPRD. ChatPRD is an AI co-pilot that helps you write great product docs, automate tedious coordination work, and get strategic coaching from an expert AI CPO. And it's loved by everyone from the fastest growing AI startups to large enterprises with hundreds of PMs. Whether you're trying to vibe code a prototype, teach a first time PM the ropes, or scale efficiently in a large organization, ChatPRD helps you do better work fast.

22:56And we're integrated with the tools you love, vzero.dev, Google Drive, Slack, Linear, Confluence, and more so you don't have to change your workflow to accelerate with AI. Try ChatPRD free at chatprd.ai slash howiai. And let's make product fun again. So let's go really quickly into coding and then I'll zip back around to a couple personal use cases and we will get you to using GPT-5. So let's talk about coding for just a little bit. Before I get to that, I do have to give OpenAI true and unsponsored props here. I think that the OpenAI team continues to outperform on API design capabilities and developer support.

23:41One of the reasons that for ChatPRD, honestly, that I have centralized on a lot of the OpenAI models is that it's not the models themselves are exceptional compared to ones by Anthropic or other providers. It's really not that. It is quite simply the API designs, developer tools, ecosystems, and essential primitives that get exposed on top of these models are just much easier to work with as a software engineer developing LLM-backed tools. I've been very happy with many of the upgrades, not just to the GPT-5 model, but with the GPT-5 model, some increased improvements in tool calling, reasoning, all these sort of parameters and controls that you have over the model that as an application developer make me very happy.

24:30So I'm not going to go into that too deeply. If anybody wants to talk about it, I'll chat with you all day about it. But I think the API improvements here are worth taking a look at and you should check out the documentation. now using open or using gpt5 to code i'm going to just show you two things one it's my favorite right now and i am a model switcher that nothing stresses me out more than someone selecting auto in cursor like auto model select i cannot i cannot imagine it really stresses me out like you just leave it to the forces that be to choose your model no no no know you have to be very opinionated with your model and so I historically using cursor just as an example I'm really prescriptive with what model I choose and you can say this is all made up stuff I use Sonnet 4 a lot for front-end work I think it does pretty good front-end work I use 2503 quite a bit in the past for deeper technical work been pretty happy with it I do think 2.5 is clinically depressed it's always so sad and it's thinking so Google friends out there please just cheer it up a little bit.

25:36I don't mean my main prompts. And then I have recently been testing GPT-5 here for a couple weeks. And it's been really interesting because I got access to GPT-5 when I was shipping a very major feature. I mean, thousands and thousands of lines. And I will tell you, one, the performance of the model, it's very fast. So I've been very happy with the performance of the model. It's allowed me to do a lot very quickly. Two, it's, I mean, it's good. It writes good code. It refactors. It's thoughtful. And let's take that word thoughtful and talk about one of my primary observations on this model. Girlfriend loves to call a tool.

26:17So if you look over here on the right, man, I have rarely hit Cursor's 25 tool call limit in a single call in many, many moons. I have not hit that in a long time. And I hit it really consistently with GPT-5. It will take advantage of tools. It is a tool calling beast. And so you can see here on the left side, it's reading, it's searching, it's reading, it's searching, it's reading, it's searching. Honestly, sometimes it felt a little inefficient and ineffective. And this will be one of my questions as these get rolled out into production in these coding tools. Will token usage, will tool calling and performance start to become an issue?

26:57but man she loves a tool call the second thing you'll see here is it loves bullet points it will talk to you in bullet points all day and all night it loves loves loves bullet points and so you'll see it talk to you like an engineer might talk to you in slack lots of bullet points but that being said the code I am happy with the quality I'm happy with it's a great engineering partner. As I said, you want one of these on your team. So we didn't go too deep into coding. But again, GPT-5 is now my daily driver. I love it. And it's really great when you're actually using the code in production. So again, I'm going to repeat myself, I really do think this is a great engineer's model.

27:43And you're going to really like it for that use case. But let's switch over and look at chat GPT and how GPT five actually operates in their core product. Okay, so one thing you'll know is you'll have two options here, at least I had two options here, GPT five and GPT five thinking, I'm used thinking for specifically prototyping and design in chat GPT. So I think that with the GPT five thinking, it is possible that chat GPT really becomes a viable option for folks trying to do some high level prototyping inside an AI tool. I love the specialty tools. I love VZero, lovable bolt, all those. Of course, I work in cursor.

28:25But if you're just trying to design something, one of the things I noticed about GPT-5 is it's got great front end design taste and actually makes things that look pretty good. So I'm going to go ahead and turn on canvas, which allows chat GPT to generate some images. And I'm going to drop in a copy of the chat purity homepage. So you can see it's very pink, we love her. And I'm actually going to write just a really simple prompt here. I'm going to say design and prototype a blog for chat purity matching our style. Okay, that's it. So GP5 is going to use that reference image. It's going to think it loves to think we can actually expand this thinking right now and see how it thinks through generating this.