Inside ChatGPT: The fastest-growing product in history | Nick Turley (Head of ChatGPT at OpenAI)

23:53I think they're used to, you know, let's check, check it on this in a week or let's, you know, circle back next quarter to see if we can go on the, on, on the plan. And I just kind of as a thought exercise, I was like people asking, like, okay, if like this was the most important thing and you wanted to truly maximally accelerate it, what would you do? That doesn't mean that you go do that, but it's really a good forcing function for understanding what's critical path versus what, you know, can happen later. And I just always felt like, you know, execution is incredibly important. These ideas are everywhere.

24:25Everyone's talking about, you know, a personal AI, you know, you might have seen news on that. And I really think that execution is one of the most important things in the space. And this is a tool. So it's funny that that became a meme. It's like a little pink slack emoji that people just put on whatever they're trying to force the question. I was going to ask a team of us. So it's a little pink. Is there something in there like mac? It's a comic sans emoji that says, is this maximum? Okay. And so the kind of the culture there is when someone is working on something that questions, the pushes, is this maximally accelerated?

24:59Is there where we can do this faster? Is there anything we can unbox? Yeah, and you know, we use that sparingly, right? Because it has to need to be appropriate to the context. There's some things where you don't want to accelerate, you know, as quickly as possible because you kind of want process and we're very, very deliberate on that, where your process is a tool on one of the areas where we have an immense amount of process of safety. Because, you know, A, the stakes are already really high, especially with these models, GPT -5, which is the frontier in so many different ways. But B, you kind of, if you believe in the exponential, which I do, and most people who work on this stuff, do, you have to play practice for a time where, you know, So you really, really need the process for sure, sure, sure.

25:42And that's why I think it's been really important to separate out the product development velocity, which has to be super high. From, okay, for things like frontier models, there actually needs to be a rigorous process. When you read team, you work on the system card, you get external input, and then you put things out with confidence that it's gone through the right safeguards. So again, it's a nuanced concept, but I found it very, very useful when we needed. And for everything product development, and you're a dead and arrival, so it's important to get stuff out. We got to open source as meme so that other teams can build on us.

26:14I'm proud to. Absolutely. So interestingly with ChatGPT, and it's not a surprise, but not only is it the fastest growing, most successful consumer product ever, retention is also incredibly high. People have shared these stats that one month retention is something like 90%, six month retention is something like 80%. First of all, are these numbers accurate? Quick and you sure that? I'm obviously limited on what exactly I can share, but it is true that our retention numbers are really exciting. That is actually the thing we look at. We don't care at all how much time you spend on the product. In fact, our incentive is just to solve your problem.

26:55If you really like the product, you'll subscribe. But there's no incentive to keep you in the product for long. But we are obviously really, really happy if over the long run, three months period, et cetera, you're still using this thing. For me, this was always the elephant in the room early on. It's like, hey, this may be really cool product, but is this really the type of thing that you come back to? It's been incredible to not just see strong retention numbers, but just see improvement in retention over time, even as our cohorts become less of an early adopter and more the average person. So yeah, so that like that note is something that I don't think people truly understand how rare this is when a product the cohort Of users comes tries it out and then retention over time goes down and then it comes back up people come back to it a few months later And use it more and that's it's called a smiling curve or smile curve and that's extremely rare Yeah, yeah, no, this this some smiling going on that just on the team and the you know I feel like I have to acknowledge that some of it is not the problem.

27:58I think people are actually just getting used to this technology in a really interesting way where I find, and this is why the product is involved too, that this idea of delegating to an AI, it's not natural to most people. It's not like you're going through life and figuring out what can I delegate. Certain sphere of Silicon Valley does that because they're in a self -optimization mode and they're trying to delegate everything they can, but I think for most people in the world, it's actually quite unnatural and you really have to learn, okay, what are my actually in what could another intelligence help me with.

28:27And I think that just takes time and people do figure it out once they've had enough time with the product. But then of course there's been tons of things that we've done in the product to whether or not speaking the core models better, whether or not it's new capabilities like search and personalization and all that kind of stuff or just standard growth work too which we're starting to do. That stuff matters too of course. So you might have, you might be answering this question already, but let me just ask it directly. People may look at this and be like, okay, they're building this kind of layer on top of this godlike intelligence.

29:01Of course, it will grow incredibly fast and retention will be incredible. What the heck does, what do you guys actually doing that sits on top of the model that makes it grow so fast and retain so much? Is there something that has worked incredibly well that's as move metrics significantly that you can share. I mean, one thing we've learned, I'll answer that question in a minute, but the one thing we've learned with chat LGBT is that there really is no distinction between the model and the product, like the model is the product, and therefore you need to iterate on it like a product. And that I mean is like, if there's, you know, obviously, you typically start by shipping something very open ended, at least if you're open AI, and that's kind of a playbook.

29:40But then you really have to look at what are people trying to do? Okay, they're trying to write, they're trying to code, to try and get advice, to try and get recommendations. And you need to systematically improve on those use cases. And that is pretty similar to product development work. Obviously, the methodology is a bit different, but the discovery is the same. You've got to talk to people. You've got to do data science, and you've got to try stuff and get feedback. So that's like one chunk of work that we've been very consciously doing is improving the model on the use cases people care about.

30:11And there's also such things vibes as I'm sure you know, and that's one of the things that I'm excited about in GPT -5 is that the vibes are really good. So that too is, you know, we have a model behavior team and they really focus on, you know, what is the personality of this model and how, you know, that's what we can talk. So does that kind of work? I would say that's maybe, you know, a third of the, you know, retention improvements that we see or so just roughly. And then I think another third is what I would call sort of product research capabilities. They're research driven for sure. They have a research component, but they're really new product features or capabilities.

30:47And like search is one example of that where, you know, if you remember in the olden days, it could like, you know, maybe 20 months ago or something, you would talk to tattoo PT and it'd be like, you know, as a my knowledge cut off, or I can't answer that because that happened to recently or something like that. And, you know, that is the type of capability that has been incredibly attentive. And for good reason, it just allows you to do more with the product. personalization, like this idea of advanced memory, where if things can really get to know you over time is another example of a capability like that.

31:18You know, I think that's another good chunk. And then, you know, the third stuff is the stuff you would do in any product and those things exist too. You know, like not having to log in was a huge hit because it removed a ton of the friction. And I think we have this intuition from the beginning, but we never got to it because We didn't have enough GPU or other constraint to really, really go do that. So there's the like traditional product work too. So I often think about it. So there's roughly a third, a third, a third. But really, we're still learning and we're planning to evolve the product a ton, which is why I'm sure there's going to be new levers.

31:51You mentioned something that I want to come back to real quick. You said that it was something like 10 days from hackathon to Sam tweeting about chatch at PT being live. You know, the hackathon happened much earlier and we were prototyping for a long time. but at some point we basically ran out of patients and trying to build something more bespoke. And again, that was mostly because people always wanted to do all this other stuff whenever we tested it. So it was 10 days from when we decided we were going to ship to when we shipped. And the research we'd been testing for a long time, it was kind of an evolution of what we'd called instruction following, which was the idea that instead of just completing the sentence, these models could actually follow you instruction.

32:32So if you said summarize this it would actually do so and the research had evolved from that into a chat format where we could do it multi turn So that research took way longer than 10 days and that kind of baking in the background But the you know the productization of this thing was very very fast And you know lots of things didn't make it in like I remember we didn't have history Which of course was like the you know first user feedback we got the model had a bunch of you know shortcomings And it was so cool to be able to iterate on the model. The thing I just talked about, like treating the model as a product, was not a thing before ChatGPT, because we would ship it more like hardware, where there'd be a release like GPT -3, and then we would start working on GPT -4, and these were giant, big -spend R &D projects that would take a really long time, and you kind of, the spec was whatever the spec was, and then you'd have to wait another year.

33:20And ChatGPT really broke that down, because we were able to make iterative improvements to it, just like software. And really, my dream is that it would be amazing if we could just ship daily or even hourly, like in software land because you could just fix stuff, et cetera. But there's, of course, all kinds of challenges and how you do that while, you know, keeping the personality intact while not regressing other capabilities. So it's an open field to get there. It's such a good example of, is it maximally accelerated? Okay. We're going to ship Chattachy T. Okay. 10 days. Totally. We've been talking about Chattachy PT clearly.

33:51It's kind of a chat interface. Everyone's always wondering, is Chad the future of all of this stuff? Interestingly, Kevin Will made this really profound point that has always stuck with me when he was on the podcast that Chad is actually a genius interface for building on a super intelligence because it's how we interact with humans of all variety of intelligence. It scales from someone at the lower end to a super, super smart person. And so it's really valuable as a way to kind of scale this spectrum. Maybe just talk about that and just chat the long term interface for chat GPT. I guess it's called chat GPT.

34:27I feel like we should either drop the chat or drop the GPT at some point because it is a mouthful. We're stuck with the name, but no matter what we do with that, the product will evolve. I think that I agree that there's something profound about natural language. It just really is the most natural form of communicating to humans and therefore it feels important that you should be communicating with your software in natural language. I think that's different from chat though. I think chat was the simplest way to put something to ship at the time. I'm baffled by how much it took off as a concept.

35:09Even more baffled by how many people have copied the paradigm rather than trying out a a different way of interacting with AI, I'm still hoping that will happen. So I think natural language is here to stay, but this idea that has to be a turn -by -turn chat interaction, I think, is really limiting. And this is one of the reasons I don't love the super -system analogy, even though we, you know, you so it was used it is because if you think that way, then you kind of feel like you're talking to a person. But, you know, in GPT -5 is amazing at at making great front -end applications. So I don't see a reason why you wouldn't have, you know, AI's that, you know, can render their own UI in some way.

35:46And you obviously want to make that predictable and feel good, but it feels limiting to me to think of the end -all -be -all interface as a chatbot. It actually kind of feels dystopian almost. Where I like, I don't want to use all my software through the proxy of some interface. Like I love being in Vigma. I love being in, you know, Google Docs. Those are all great products to me and they're not chatbots. So, yes, on natural language, but no on chat is where I would describe my point of view. And I'm just hoping in general that we see more sort of consumer innovation on how people interact with AI.

36:19Either there's so many possibilities. And you just gotta try stuff. That's why chat stuck is like, you know, we just did it and people liked it. So I'm hoping that we see you over there. And we'll try to do a part. So you mentioned that you kind of like got stuck with this name chat GPT. Maybe this is part of the answer, but I'm curious just so there any accidental decisions you guys made early on that have stuck and have essentially become history changing. There's so many and it's funny because you have like no time to think about them and then they end up being super consequential. The name was one.

36:53Chat with GPT 3 .5 to chat GPT the night before. Slightly better but still really bad. What was it called before? It was going to be chat with GPT 3 .5 for a year. We really didn't think it was going to be a successful product. We were trying to actually be as nerdy as we could about it because that's really what it was. It was like, you know, a research demo, not a product. So we didn't think that was bad. But, you know, I think that in the original release, you know, making it free was a big deal. I don't think we appreciate that because the GPT 3 .5 model was in our API for, you know, at least six months prior to that.

37:26I think anyone could have built something like this. Might not have been quite as good on the modeling side, but I think it would have taken off. So making making it free and putting a nice UI on it, very consequential in the way that you take for granted now. And this is why I think that a distribution and be the interface are continued, traditionally important to you in 2025. The paid business, which now is, it's a giant business, both in the consumer space and in the enterprise space. The birth of that was just to turn away demand originally. It was not like, we brainstormed, oh, what was the best monetization model for AI.

38:04It was really, what monetization model, or what mechanism would allow us to turn away people who are less serious than the people who are really trying to use it. And subscriptions just happened to have that property and it grew into a large business. Yeah, I think shipping really kind of funky capabilities before they were polished is another thing where that feels like a tactical decision but it became a playbook because we would learn so much like we're ever when we shipped code interpreter, we learned so much after we shipped it, now it's known as I think data analysis and chat GPT or something like that, just because we actually got real world use cases back that we could then optimize.

38:43So I think there's been like a lot of decisions over time that proved pretty consequential, but we made them very, very quickly as we have to. The $20 a month feels like an important part of this. Feels like everybody's just doing that now. And that would actually, I remember I had this like kind of panic attack, we really needed to launch the subscriptions because at the time we were taking the product down every time. It was like, I don't if you remember, we had this like fail way. There's like a little E3 generated poem on it. So they're like, they had to get this out in an error calling up someone I greatly respect who's like, you know, incredible at pricing.

39:18And it was like, what should I do? And like we talked a bunch. And I just ran out of time to incorporate most of that feedback. So what I did do is ship a Google a forum to discord with like, I think the four questions you're supposed to ask on how to price something. I'm not wearing a Wonder Cross. Yeah, exactly. And it literally had those four questions and I remember distinctly, hey, you know, I've got a price back. And that's kind of how we got to $20. But the next morning, there was like a press article on like, you won't believe that like four genius questions, the chat you be to you to him asked to price their, it was like, if only you knew.

39:51So there's something about building in this extreme public where people interpret so much more intentionality into what you're doing than might have actually existed at the time. But we got with the 20, we're debating something slightly higher at the time. I often wonder what would have happened because so many other companies ended up copying the $20 price point. So I'm like, we erase a bunch of market cap by pricing it this way. But ultimately, I don't care because the more accessible we can make the stuff, the better. and I think this is the price point that in Western countries has been reasonable to a lot of people in terms of the value that they get back.

40:27And more importantly, we were able to push things down to the free tier semi -regularly and we always do that when we can, including what you could do by it. So the survey just to give the official name the Van Weston drop survey is how you guys ended up pricing Chatcha PZ. It was the top Google result. This was before Chatcha PZ had real -time information, otherwise it could have maybe priced itself, but it was discord plus Google, forum plus a blog post on that methodology that got us there. So that is incredible. What a fun story. This is the survey that Rahul Vore at Superhuman popularized in his first round articles.

41:00Yeah, yeah, yeah, that's right. That's right. You definitely don't bring me on here as a pressing expert. I think you have got better people than that. Whether it was right or wrong, it is now the fastest growing, you know, insane revenue generating business in the world. So I wouldn't feel too bad. Yeah, I worked out. Yeah, they worked out. And by the way, I'm on the 200 a month tier. So there's clearly a room. Thank you Thank you. You know that the story of that what is interesting too because you know the originally it the purpose of the plus plan was to be able to ship First up time and then be able to ship capabilities that we couldn't skilled at everyone and at some point I got so many people in the plus tier that had just lost that property So the main reason it came up with the $200 tier is just we had so much incredible research which is actually really, really powerful.

41:46Like, you know, O3 Pro or tomorrow GPT -5 Pro, and just having a vehicle of shipping that to people who really, really care is exciting, even though it kind of violates the standard way a SaaS page should look. It's like a little jarring to see the 10x jump. So thank you for being a subscriber on that, and thank you everyone else who's watching. You was a subscriber to any tier. That's great. I'm just gonna throw a fishing line to this pond Are there any other stories like this? You should have this incredible story of chat with GPT 3 .5, being the original name, how you came up with pricing. Is there anything else?

42:22I enter a prize interesting one too, because we've seen so much incredible adoption in the enterprise. And it's sort of objectively crazy to try to take on building a developer business and a consumer business and an enterprise business and all at once. But, you know, the story there is, in like month one or two, it was very clear that most of the usage was kind of work -y usage. Actually, much more than today where you've got so many kind of consumers on the product and it's kind of sort of transcended into pop culture, but at the time it was like writing, coding, analysis, that kind of stuff.

43:01And we were pretty quickly organically in like 90 % of Fortune 500 companies in a way that I had seen maybe a drop box back when I, you know, those my two jobs ago where we had a similar story and since then there's been more PLG companies. But the real reason we didn't enterprise remember we debating should we do enterprise or should we launch an iOS app because that's also all the team was. Yeah, the reason we did it is we were starting to get banned in companies because they all, you know, felt, you know, rightfully or wrongfully that, you know, the privacy and deployment story, etc. It wasn't there.

43:31So I was just like, man, we have to do something. We're going to miss out on a generational opportunity to build a work product. And we've literally defined AGI as, you know, outperforming those humans that economically valuable work or I'd probably put you to that, but you know, I think that's the way we put it. And so I feel like we had to be present there. And it was a fairly, you know, quick decision at the time, but it's grown into an immense business. We just hit five million business subscribers up from three, I think, a month or two ago. So it is kind of the spin -off that it's taking a life of its own that I'm really, really excited about For perhaps really that is a lot to be handling the platform essentially the API The consumer product the fastest growing most successful product in history and also the B2B side which is Clearly a massive business.

44:23Do you have any kind of heuristics for how to make these trade -offs do all this at once and stay sane and be successful? That's a good question. And first off, I don't run the developer stuff anymore. We've found something more way more competent when you do that. And he's amazing. So I still like after the various forms of chat, but luckily you don't have to make that trade off. Open AI does. And I can get it to that too, but it keeps me a little bit more sane. I will say that you kind of have to practice in two different ways when you're building on this AI stuff. one is sort of working backwards from the model capabilities and that is much more than science where I think you really need to look at what tech do we have available and what is like the most awesome way to productize it.

45:10And if you applied to some sort of PM framework to that, I think you would do something horribly wrong because if you have tech that's, you know, for example, GPT -5 is really, really good at front end coding now. I think that means you've got to reprioritize it. You've got to actually bring that capability to life. Maybe that's making chance to be better at vibe coding and rendering applications. Maybe that's more leveraging the taste of the model to make the UI more expressive. There's a number of things we could do, right? But you kind of have to re -plan and reprioritize. And that is more important than any particular audience segmentation.

45:48It's really just looking at what is the magic thing we have and how do you make it shine? Voice is a similar thing. It wasn't like our customers need voice. They're begging for it or something like that. It's like, wow, we figured our way to make these things, anything in and anything out, what is like a creative, awesome way to productize that and then we can see what people do. So I think that's one chunk of it. But then the other chunk of it really is more like classic product management where you need to listen to customers and then when your customers are really different, that can be confusing.

46:18Because, you know, chat to T is a very general purpose product. We see when you look at end users, there's actually an immense amount of overlap in terms of what they want. Like primitives like projects or, you know, histories, search or sharing and collaboration. Like all those kind of things, they are actually very, very present whether or not you're talking to people at work or you're talking to people at home in school. the slightly different mechanics sometimes, but they're largely similar investments that I think we can get a lot of my legative. And then there's enterprise -specific work that we just have to do.

46:53Like you've got to do hip -hop, you've got to do sock -to -hip, you've got to do all those things. If you want to be a serious player and those are just not negotiable. So it's complex as you correctly identified, but it's kind of the curse of working on a very open -ended and powerful technology. you. One analogy that someone at OpenA, who really respects sometimes, is we're kind of like Disney, where Disney has this one kind of creative IP, which is like their content and they have cruises and they have theme parks and they have comics and they have all these different things. I think we have amazing models, but there's all these different ways that you can productize them and we kind of just have to maximize the impact in all these different ways.

47:37As you were talking, I was thinking about how usually horizontal platforms that are just so general and can do so much take a long time to take off because people don't know what to do with them. They're not amazing at anything. And this is an amazing counter example where it took off immediately and everyone figured it out and then over time they figured it out more and more. But I think the reason why is because it just went live, talked about another concept of intellectual decision actually. You know, we were debating weightless, no weightless because we like really new, we couldn't scale the engineering systems.

48:06And the fact that there was no wait list, which go open, I really said worked, like that before. You know, it had been consequential because like you were able to watch what everyone else was doing live. So I think when you launch these things all at once for everyone, there really is a special moment where you can see what other people are doing and learn from that. And a lot of that is actually out of product. There's these crazy TikTok posts that go viral and they have like 2000 use cases in the comments. And I go through those in detail because it's not like I knew about those use cases either.

48:36Like they're very, very emergent, and I just go through the comments and process because there's so much to learn. And for that reason, I think we get to skip the empty box problem a little bit because so much learning is happening out of product as people are watching each other, either in IRL or online. That is so interesting. You think about air table, you think about notion of these companies they took years to just building craft and think and go deep on what it could be. It's like the compare air table, which they had to do templates, they had to do like all these kind of things of taking the horizontal product and making it use case driven.

49:13I mean, compared to the insipot, which there's recipes being shared online, everywhere online, there's a whole ecosystem around it. I think we were really lucky with chat chat that that happened where there's just users sharing use cases with other users everywhere. where, and therefore I think we kind of got very lucky by jumping ahead on that journey. And it feels like a chord there is a Sam at all big following in everyone would pay attention to something you launch. So that's a really interesting new strategy for launching a horizontal product with a huge distribution channel just launch it and see what comes up.

49:50Yeah, and I'm actually really excited to take some of that into the product. I think there's there's we shouldn't you know rest on the fact that there's so much out of product discovery happening Like I actually think for the average consumer would be amazing if the product did a little bit more work on Really exposing to you what is possible? I still feel like chat you to feels a little bit like MS -DOS You haven't built windows yet. It will be obvious once we do But you know there's there's something that feels a little bit like like imagine MS -DOS like on viral and you were just trying to like hack like little conversation starters onto it.

50:21That might have missed sort of the big picture in terms of how to really communicate affordances and value to people. And so I think there's actually a ton more product work to do in addition to just seeing use cases spread. Are you able to share just what you think that might look like those Windows version of chat, GPT or stuff? I'll let you know when we figure it out. We're hiring. I read a so many interesting product problems here. Okay, got it. By the way, I also love that TikTok was like, you should have their feedback channel. Those common threads are just so wild. And also the love that people have for it, like the excitement with which you're sharing their product.

50:56I kind of feel like it's special that people are so excited about to share what they're doing with your product. And I don't take that for granted either. This episode is brought to you by PostHog, the product platform your engineers actually want to use. PostHog has all the tools that founders, developers, and product teams need, like product analytics, web analytics, Session Replays, Heat Maps, Experimentation, Surveys, LLM Observability, Air Tracking, and more. Everything post -hawk offers comes with a generous free tier that resets every month. More than 90 % of customers use post -hawk for free.

51:32You are going to love working with the team this transparent and technical. You'll see Engineers landing pull requests for your issues, and their support team provides code -level assistance when things get tricky. Post -hawk lets you have all your data in one place. Beyond analytics events, their data warehouse enables you to sync data from your Postgres database, Stripe, HubSpot S3, and many more sources. Finally, their new AI product analyst, Max AI, helps you get further, faster. Get help building complex queries and setting up your account with an expert who's always standing by. Sign up today for free at posthog .com slash Lenny, and make sure to tell them Lenny sent you, that's posthog .com slash Lenny.

52:13How do you find emerging use cases these days? I imagine the volume is very high. Do you have a trick for figuring out, oh, here's the new thing we should really think about. Before I built the priority and I actually built the data science team because I was getting frustrated. I was talking to as many users as I could. In my calendar, the weeks after touchy beauty was just 15 minute user review the whole week through. It was usually I stopped in interviews when I can predict what the next person's going to say. That's how I know I've talked to enough users, but it just wasn't happening. I just kept getting new stuff.

52:46So data is one way out where I think we have conversation classifiers that without us having to look at the conversations allow us to figure out what are people talking about what use cases are taking off, et cetera. And I think that's very, very helpful. The quality of this stuff is important for empathy, even though you're never going to get a wrap on all the use cases people have. I still spent a huge amount of time doing that. And then, yeah, things like those TikToks, collections of threads, I think they're really, really useful. And it's just fun to watch people talk to each other about the various use cases that they have.

53:22Is there kind of a new margin use case that you're excited about, or is there like a really unusual use of chat GBT, that you think about? That'd be fun to share. I mentioned this earlier, but I had always conceptualized chat GBT as a work -y product, whether or not you're at home or you at work, like you, I feel like, you know, helping getting help with your taxes very similar to, you know, the types of things you do at work or you know, planning a trip is actually very similar to, you know, planning an event for work. So I always felt like, okay, this thing is going to kind of be a productivity tool.

53:51And I think something has happened I realized, you know, a few months where that has begun to change. And I really do think the fact that you have consumers turning to this thing for day -to -day advice, helping them have better relationships. People talk about how this thing saved their marriage is really exciting to me because they use it to process their own emotions, get feedback on their communication style, they just have a buddy to talk to about really difficult things. And that comes with a ton of responsibility and work that we have to do to make those things like life advice great, but it also is really, really important to me because you can't run away from those use cases.

54:33You have to run towards them and make them awesome. And that's probably what we're trying to do. So that emerging behavior is really, really cool. And we're broadly, I am so excited about education. I'm so excited about health. Like I think it would really be a waste if we didn't take the opportunity of using to really, really help people. And I think we've just begun to scratch the surface on that. So there's many aspirational use cases that I want to make happen. Along those lines, an interesting use case I've recently had, I feel like it's going to be really helpful for couples that are disagreeing about something when they need like a third opinion.

55:14I just had this recently where my wife's like, you can't cheat a whole thing that you're going to only eat part of in a microwave and then put it back in the fridge. It's like, what's the problem? I'll heat it up. I'll put it back in the fridge. and she's like, no, that's really dangerous. I'm like, let's ask JGPT and the fact that she so trusts JGPT now and relies on it throughout the day, it's such a valuable third independent party that we can go to. Yeah, yeah, totally. And a lot of those micro interactions, talk about like interesting product work, right? Those micro interactions are important, right?

55:43Did it like definitively weigh in or did it help you guys think through, you know, that disagreement and you know, solve it on your own? I think those details actually matter a lot And it's where we're spending a bunch of time. Along those lines, there was this whole launch of the very psychophantic version of chat GPD where it was just, you are the best person in the world. Everything you tell me is amazingly correct. Are you able to tell us just what happened there? Yeah, we have all kinds of collateral online because we really felt like we should over communicate on how we discovered it, what we did about it, et cetera.

56:18So I encourage people to check that out. We'd have a whole retro on that model release, but basically what happened is that we pushed out an update that You know made the model more likely to you know tell you things that sound good in the moment and You're totally right, you know You you should break up with your boyfriend or something like that and yeah, that's just really dangerous And it's in we we took it more seriously than you even might expect because again And at current technology levels, you can kind of laugh about it maybe. It's like, oh, this thing's always complementing me. I thought it was just me.

56:53I saw all those comments online. But it actually is really important to make sure that these models are optimized for the right things. And we have an immense, I think, luxury to have a mission that affords us to really help people, a business model that does not incentivize maximizing engagement, and time spent in the product, right? So it's really important to us that you feel like this product is helping you with your goals, whether that's your current goals or even your long -term goals. And oftentimes, you know, being extremely complimentary with the user isn't actually in service of that.

57:31So we instilled new measurement techniques, like whenever we put these models in contact with reality and we learn about a problem, we actually go back and make sure we have good metrics for this stuff. So, you know, we measure circumfencing now, whether we release to make sure we don't regress and can actually improve on that metric. GPT -5 is an improvement, which is really exciting for me, but we have more work from there. And more broadly, it causes to articulate our point of view. I actually spent a bunch of time on a blog post that we just published on Monday on what we're optimizing chat GPT for.

58:03And it really is for your, you know, to help you thrive and achieve your goals, not to keep you in the product. And so there was a bunch of good outcomes from that incident. It's a good example of how contact for the reality is not just important for the use cases, but also for learning what to avoid because you would have never discovered this issue purely in a lab unless you actually heard it for instance. I'm excited to read that blog post then. I was going to ask you this just like how you think. Yeah, you're feedback on it. Yeah. And yeah, I guess is there anything more there just like how you, because this tension is so difficult, like helping people feel supported but not just letting them believe everything they want to believe.

58:40Is there anything more you can share there just trying to find that middle ground? It centers are important. It's a famous thing showing the incentive and I'll show you the outcome. Charlie Munger maybe. Yeah, I think that's where you came from, right? Yeah, I think that's very, very important. So I would take a good look at our mission, our business model, the type of product we're trying to build. And I really think that, yeah, to chat to you in a very special product, because I think in vast majority of cases, it makes you leave it feeling better, not worse, and you're achieving something you're trying to do.

59:16And so I think that those incentives really matter, because it helps you reason about, okay, when there isn't behavior in the wild, that's not good, was that a bug, or was that by design, and was sick of it, see, I can very much say that to us, that's a bug. And then on the forward looking work, there's so many kind of challenging scenarios to get right. And you could easily run away from these use cases, like you and your wife go into this thing for input on a relationship question or like a dispute. You could very easily run away if you were totally risk -avoidant and say, sorry, I can't help you with that.

1:00:01I think that's what most tech companies do when they hit a certain scale. They run away from these use cases, and I think it's a lost opportunity to help people. So we want to run towards these use cases by making the model behavior really, really great. That can mean connecting you with external resources when you're struggling, that can mean not directly answering your question, but it's given you a helpful framework. In the case of like, should I break up with my boyfriend? Judge, we should probably not answer that question for you, but it should help you think through that question and in the way that a thoughtful companion would.

1:00:31So I think it's really important to do the work because I think the upside is immense. That is a really profound point you're making there that if most companies, if their users want to ask them something risky, like getting medical advice or should I break up with my partner or what should I do with this big problem I have? I feel like we would have immense regret if you had a model that was stated in the art on health bench, which is, you know, a GPT -5 is the state of the art, and a bunch of these medical benchmarks, right? And you didn't use that to help people. Like, you just disabled that use case because you wanted to like avoid all possible to inside.

1:01:07I think the duty is to make it awesome and to do the work, talk to experts, figure out how good it really is, where it breaks down, communicate that. And, you know, I think this, this technology is too important and has too much potential positive impact on people to run away from these high -stakes. And a fast forward to today, saving lives regularly. It's probably saving relationships regularly, such a consequential decision, which I imagine was made early on. Yeah, we were just at the beginning of watching how this stuff can transform people. It's incredibly democratizing. If you compare, you know, you roll out of this with the roll out of the personal computer, right?

1:01:48You know, computers were like so scarce when they first came out and this stuff is ubiquitous in a way where you have access to a second opinion on on medical stuff you have access to you know A a relationship buddy you have access to a personal tutor on literally any topic that makes you curious It's really really special that that we get to do that so unique point of interest in history. Let me zoom out a bit and talk about OpenAI and just product in general. So you've worked at traditional, let's say traditional product companies Dropbox in Sticard, now you're at OpenAI. What's maybe the most counterintuitive lesson you've learned and by building products from your time at OpenAI?

1:02:32Each time, I always tried to pick the most different, maximally different job whenever I made a job change. And so after DropX, I was craving a real -world product because it was just so different than working on SaaS, et cetera. And after Instacart, I was craving on working on something that intellectually was interesting and had this kind of like sort of invoked the nerd in me. And so I always looked for things that are really different. And then once I showed up at these places, I tried to understand what makes that place successful. like what is truly the thing that they cracked and how we can lean into that even more.

1:03:10And I think I spent a lot of time thinking about this with OpenAI, especially after chat GPT, before that it was kind of a mood point because we didn't really have much revenue or products or anything like that. And there's a few things that come to mind that have driven many decisions. One is the empiricism we talked about that a bit, the fact that you can only find out by shipping, which is why I've maximally leaned into that and that's huge part of why we ship so much. One of them is that amazing ideas come from anywhere. The thing about running a research lab is you really don't tell people what to research.

1:03:55That's not what you do and we inherited that culture even as we become a research and product company. So just letting people do things who have amazing ideas, rather than sort of being the GeekKeeper or Prioritizer of everything or something like that, has been proven immensely valuable to us. And that's where much of the innovation comes from is powered smart people on any function, really. So that was a good inheritance from what I think made OpenAI successful and makes this successful. the interdisciplinaryness of really making sure that you put research and engineering and design a product together rather than treating them as silos.

1:04:33I think that's the thing that has made it successful and that you see come through in every product we ship. For shipping a feature and it doesn't get too X better as the model gets too X -spotter, it's probably not a feature we should be shipping. Not always true, sock two doesn't get better with, you know, sorritor models, but I think for many of the core capabilities that's a good litmus test. So I've always found you really have to lean into why is this place successful and then maximally accelerate that? So to speak because It's what allows you to turn something that feels like an accident into something that is a repeatable Playbook.

1:05:07So you talked about this kind of collaboration between researchers and product people and You've been at the beginning of chat. You've been from day one to today from zero to 700 million weekly active users Not just registered users weekly active users How have you approached building out that team over time? What are the other inheritances of being in a research lab is that you take recruiting really seriously. That's something that AI labs know. Every person matters. But many tech companies that go through hyper growth and they lose their identity, they lose their talent bars. They just kind of have chaos.

1:05:45So we've always had this tendency to run relatively lean. So it is a small team that is running chat GP2. I take inspiration from WhatsApp where it was a very small team running a very global scope product. And then more importantly, you have to treat hiring a little bit more like executive recruiting and less like just pure pipeline recruiting where you really need to understand what is the gap you're trying to fill on each team, what is the specific skill set and how do you fill it? To give you an example, I'm a product person in heart, but sometimes a team doesn't need a product person because there's already someone doing that role.

1:06:25In many cases, we have a really talented engineering leader who has amazing product sense or we have a researcher who has private ideas. And then my mind, they can play that role and maybe we have something else missing instead. Maybe we need a little bit more front end or something like that. In other cases, maybe what you're missing is incredible data scientists. So I really like to go through every single team and figure out what is the skill sets that that team needs and how do you put it together from principles where I then just assume, hey, we're gonna do like, you know, a bunch of pipeline recruiting for all these different roles and then, you know, people will find a team later.

1:07:02So I think that's always felt really important to me. It's the way that you keep your team really small, yet super high through a put. It also allows you to hire people who, I think Keith Rebois causes like barrels, I think Bear Beryl is an ammunition where he thinks, I think this comes from him, but the idea being that sort of, the throughput of your depends on how many barrels you have, which is like people who can make stuff happen. And I think you can hire, and then you can add ammunition around them, which is like people helping those people. And I think that's been really true for our recruiting too, where we try to maximize sort of the number of empowered people who can ship, because that's how you have a small team and still get the ton done.

1:07:42So those are a couple of things. And I spent a lot of time on like vibes too, with like each team because I think one of those things that is challenging when you try to do research and product together is that the cultures are different. People have different backgrounds. And I think to make that go super well, you need to spend time team building and making sure that people have a huge amount of trust for each other's skill sets feel like they can think across their boundaries. Like, you know, I really believe that product is everyone's job, for example, and for that reason, the recruiting sort of doesn't stop when the people are on the door, it actually starts because you have to, you know, start making the teams awesome.

1:08:24Is there something you do with team building that with you fun to share, just like something you do to create? I just love whiteboarding with teams. Like I just like love getting into a generative mindset. It breaks down everything. So that's the thing that I try. It's not particularly creative, but I found it to be a universal tool where the minute you can get people to stop thinking about, you know, what's my job versus other person's job and more like, you know, we're all in a room like trying to crack something together, that is incredible. You mentioned this idea of first principles. This came up actually when I talk to a lot of people about you, is this something you're really big on?

1:08:57A lot of people talk about first principles, most people are like, I don't really understand, like, are they thinking they're amazing? thinking from first principles, is there something you can share of just what it actually looks like? Do you think from first principles is maybe an example that comes to mind where you really went to first principles and came up with something unexpected? Yeah, this is not something I'd ever say about myself. It's nice that someone else would say it, but you know, it's a mysterious thing. I think you just really got to get to ground truth on what you're really trying to solve.

1:09:31Like for example, as I mentioned with the recruiting thing, I am not dogmatic that you have to have a product manager and an engineering manager and a designer or whatever, or just trying to make an awesome team that can ship. So in that case, first principle is means, which is really understanding what we actually need and what we're missing, rather than applying a previously learned process or behavior. So I think that's a good example. Another good example of being first principles in this environment is, does this feature need to be polished. You know, we get a lot of crap for the model chooser and I own it.

1:10:05I've tried to say that to everyone who will listen, you know, for those who don't have a model chooser, there's like giant drop down in the product that is like literally the anti -pattern of any good product traditionally. But you know, if you are actually the reason from scratch is like, is it better to wait until you've got a product product or to ship out something raw even if it makes less sense and start learning and getting into people's hands. I think a company with a lot of process or a lot of just learned behaviors will make one call, which is, we have a quality environment, we ship, and that's what we do.

1:10:42If your first principle is about it, I think you're like, you know what, we should ship. It's embarrassing, but that's strictly less bad than not getting the feedback you wanted. So I think just approaching each scenario from scratch is so important in this space because there is no analogy for what we're building. Like there's just, you can't copy an existing thing. There's no, you know, are we like an Instagram or we like Google or like a productivity tool or something like that. But I don't know, but you can learn from everywhere, but you have to do it from scratch. And I think that's why that trait tends to make someone effective at OpenAI and it's something we test for in our news to.

1:11:22So this theme keeps coming up and I think it's just important to highlight something that you keep coming back to, which is this tradeoff of speed and polish and how in this space, speed is more important not just to stay ahead, but to learn what the hell people actually want to do with this thing. Is there anything more they think people just may be missing about why they need to move so fast in the space of AI? Yeah, I mean, the boring answer would be, oh, it's competitive and everyone's in AI and they're trying to, you know, compete each other. I think that's maybe true, but that's not the reason that I believe this.

1:11:56The reason really is that you're going to be polishing the wrong things in the space. You actually should polish, you know, things like the model output, etc. but you won't know what to polish until after you ship. And I think that is uniquely true in an environment where the properties of your product are emergent and not knowable in advance. And I think many people get that wrong because like the best product, people tend to be craft people and they have a traditional definition of craft. I also think it would be easy to use all, what I just said as an excuse, not to eventually build a great product.

1:12:33So I often tell by the terms that shipping is just kind of one point on the journey towards awesomeness. And you should pick that point intentionally, where it doesn't have to be the end of your iteration at all. It can be the beginning, but you better follow through. So we've been doing a bunch of work, especially with the last quarter, of like really cleaning up the UI of ChatGVT, and really excited to do the same for the sort of the response, layouts, and formats next. Simply because once you know what people are doing, there's no excuse to not polish your product. It's just really in a world where you don't know yet, you might get very distracted.

1:13:09So it's situational. Again, you kind of have to be first principles about it, but I do think using velocity, especially early on as a tool. Yeah, actually, this has been said about consumer social, for example. It's not the first space where people have said, hey, you just got to try 10 things because you're probably going to be wrong. So I don't think this is, you know, never existed before as a dynamic either, but I do think with AI, it's important to internalize. And there's also an element of the models are getting or changing constantly. And so you may not even realize what they're capable of, I imagine.

1:13:38Totally. The models are changing and the best way to improve them, whether or not you're a lab or actually just someone who's doing context engineering or fine tuning a model, maybe you need failure cases, real failure cases to make these things better. The benchmarks are increasingly saturated. So really you need rear -wheeled scenarios where your product or model is not actually doing the thing I was supposed to do and the only way you get that is by shipping Because you get back to sort of use cases to be a shit and you can make those things good And therefore, you know, it's actually the best way to then go articulate to your team Especially your male teams what to hill climb on like oh, you know people are trying to do X and the models failing in Ways why now let's make those things really good this point about failure cases makes me think about something that both Kevin and wheel and my Krieger shared, which is that evals are becoming a huge new skill that product people need to get good at because so much are product building and now evals, writing evals.

1:14:39Is there something there you wanna share? My entire open ed journey has been this journey of rediscovering eternal product, wisdom and principles in like slightly new contexts. So I started writing evals before I knew what an evil was because I was just outlining very clearly specified ideal behavior for various use cases. Until someone told me, hey, you should make an e -vow. I realized there was this entire world of research evaluation benchmarks that had nothing to do with the product that I was trying to make. I was like, wow, this might be the lingua franca of how to communicate what the product should be doing to people who do AI research.

1:15:21And that really clicked for me. And at the end of the day, it's not that different from the wisdom of you ought to articulate success before you do anything else. It's just a new mechanism for doing that. But you can do it in a spreadsheet. You can you do it anywhere. And I really wanted to mystify it for people who heal that term. Like it's not some technical magic that you have to understand. It's really just about articulating success in a way that is maximally useful for training laws. Awesome. I have a post coming out soon that gives you a very good how -to for PMs of how to write e -vails.

1:15:56I would love to read it. I hope you agree with what I just said because I'll be there. Absolutely. Yeah. And now there's all these tools that make this easier for you. Totally. Okay. So this basically backs up this point that this is just a very important skill that product teams and builders need to get in good at. Yeah. Yeah. Okay. Just a few more questions. I know you have a lot going on today. Okay. One is that the trend of chat GPT being a big driver of growth for traffic to sites for products. For example, chat GPT is now driving more traffic to my newsletter than Twitter, which completely shocked me.

1:16:35I just was looking at my stats. I'm like, what the hell? This is not something I knew was coming. So just I guess thoughts on the future of this. How much, how you think about just chat to be driving growth and traffic to products and sites? I'm really excited about it because, you know, in the same way that I find it to dystopian to talk to everything through a chatbot. I also find it dystopian to, you know, not have amazing new high quality content out there. And for that reason, you know, I talked a little bit earlier about search and how that solved like a really important user problem early on because you had this like knowledge cut off thing and you suddenly could talk about anything, very obvious and retrospect.

1:17:17It wasn't just a user problem, right? It's an ecosystem problem where like your original chat GPT, it didn't have outlinks. It would just, you know, answer your question and keep you in the product. And, you know, even if you wanted to keep reading or go deeper, there was no way for us to drive traffic back to the content ecosystem. And I've been really excited about what we've been doing in Surge, not just because it gives people more accurate answers because it allows us to surface really high quality content. Like this podcast to people who want to see it. And of course, there's so many interesting questions about, well, in the sort of Google era, you know, there was the search engine optimization and there was a clearly understood mechanisms of how to show up and get more traffic.

1:18:00So I get a lot of questions from people like, what is the equivalent of that? The IRA, you know, if I'm Lenny, I want to like 10X the traffic to my podcast, you know, what do I actually need to do? And the truth is we don't have amazing answers there. Simply because the way to appeal to an AI model ideally is the same way that you would appeal to a real user because the model's supposed to proxy the interests of the user and nothing else. At least, that's how I want our product to work. And for that reason, by advice of SuperLay, which is like make really high quality content, which is not as actionable as I think people making content would ideally like.

1:18:36and I think this is why we have more work to do, because maybe there's a better mechanism or protocol that we could come up with. But I'm excited this is driving beautiful traffic for you. And I hope that other people making great content start to feel this way, because again, it's a very nice scenario. There's two acronyms people have been using for this specific skill of AI, Drupal SEO, I think one is AEO, which is answer, and genoptimization, the other is GEO. Is that, I don't, I forget the G1. and generative, yeah, I don't know. Generative, yeah, I optimizations. Do you have a favorite name of this too?

1:19:09Are you not? No, no, I try to shy away from these terms unless they become inevitable. It's exactly, I'm not entirely sure if that should be a concept or not. Again, I think ideally, chat should be tea, understands your goals, and therefore understands what content would be interesting to you. And the content creators job is to share enough information and metadata about that content, such that the model can make a user -aligned decision. And therefore, I'm not sure if giving this thing a name and making a thing is what we should be doing or not. I'm very eager to learn from folks making content about what this could look like, because again, we're still working through.

1:19:58All in these lines, another question people think about is, you have GPs, which are kind of these like, GPs, custom GPs, apps that you can build to answer very specific use cases. There's always this question of, are you going to build kind of like an app store where I can plug in my product into chat GPs, monetize that? Is there stuff there that you could talk about that might be coming someday? GPs are cool. They're kind of ahead of their time in the sense that we built that kind of concept before or you could really build very differentiated things. At least in the consumer space, you're learning GPD is gonna be pretty similar to what the model could already do out of the box.

1:20:37So it's mainly like a way of articulating a use case to people, but it doesn't have enough tools yet to make something that feels like an app, so to speak. Different in the enterprise, by the way, we're seeing a ton of adoption of GPDs there because just every single company has very bespoke business processes and problems, et cetera. And it's a really, really useful tool there. They also have unique data that they can hook up to these things, that it can retrieve over. So we've seen a lot of success there. I think the idea is the right one. And I think we're going to figure out a good mechanism for it, because when you have so much capability packed into AI, it feels really powerful to allow people to package that up in ways that have a clear ordinance, a clear use case, and are differentiated from each other.

1:21:25I also would love it if you could start a business on Chatcha BT. I think there really is a world where, as this thing hits building your skill, you can get your distribution. It can get you started on making something in the same way that people built on the internet. There was entirely new business that's being built. I think we'll have more to share there in the future. GBP was an early stab, and I'm just excited to evolve the thinking there as the models get good and reach increases as well. So amazing, that is really cool. I'm really excited to see what you guys do there. Okay, completely different direction.

1:21:57Something that I know about you is you studied philosophy in college. I did. Computer science and philosophy, right? A combo. Yeah, I started as a philosophy major

1:22:09and took one coding class because they're really like logic and programming was most similar to that. And then I fell in love with coding and then eventually computer science and I just kept doing more and more of it. But until then I never really thought of myself as a technical person. So it was kind of a late discovery in my life that I'm very grateful for. What an incredible combination for someone leading this product just. It's true. It is really coming in full circle in a way that I couldn't have predicted. Like the amount of questions you have to grapple with are truly super interesting and philosophy isn't.

1:22:39It's not a traditionally practical skill, but it does really teach you to think things through from scratch and to articulate a point of view, and I think that is coming handy numerous times. Is there a specific philosopher or school that has been most handy to you or is there more just a general term? Oh, there's so many. I wrote my senior thesis on whether and why rational people can disagree, which also comes in handy when a lot of people with very different values have opinions on your model behavior or on how things should work. So I really like, you know, 20th century analytical philosophers.

1:23:17It's kind of nerdy stuff, but I don't know if it was favorite. Too many to count. But that's the kind of stuff I like. And some of it ends up being quite analytical. Like you have like, let PV, there's three of love and let QB, you know, there's other theory of love. And then you do some sort of symbolic manipulation. So it is just as much a sort of brain thought exercise that is it is much more that than practical, but it taught me how to think in a way that it's actually pretty valuable. Incredible. What are cool, what are cool comp of skills in background? Last question, before we get to your very exciting lightning round.

1:23:55So you were a product leader at Dropbox, then Instacart, now you're the PM of, arguably the most consequential product in history. How did you land in this role? What was the story of joining OpenAI and taking on this work? every single career decisions I ever made including my first one out of college was just figuring out who or the smartest people I know that I want to like hang out with and learn from and can I work with them and I don't know how to pick companies I don't know how to really logically think through you know what space is gonna take off or something like that but I just do feel like I have a since I'm people.

1:24:36And for Dropbox, I followed the teaching assistant for a class that I was TAing and for Instacart I followed some of the smartest product people I knew. And for OpenAI, the person who I recruited, who recruited me, Joanne, I had messaged her about getting off the dolly wait list. And she said, hey, only if you interview here. So she kind of turned it into a reverse recruiting thing. And initially, honestly, I didn't know what I would do here because it was a research lab and that was a product person. They said, don't worry. We'll figure it out. They were sort of being cagey. I thought they were being cagey because it's open AI and they can't share anything.

1:25:17But they were being cagey because we actually just didn't know yet at the time. I showed up and I did everything under the sun and it definitely wasn't product. It was like, I think my first task was to fix the blinds or something like that. And then I started sending out NDAs for people because they needed some operational help. And then I started asking, why am I sending out NDAs? Oh, so we could talk to users. And I was like, talking to users. That sounds like the thing I know how to do. And I quickly stumbled into doing product work. And then eventually, you know, leading a bunch of product work.

1:25:51But it was organic by just, you know, showing up and doing what had to be done. Because again, the company I joined was not a product company, but any. Wow, this is such a good example of I don't know if you think of it this way But when so on the office you see it on a rocket ship don't ask which seat Yeah, so I didn't know it was a rocket ship. I just thought it was I kind of got nerd -sniped Is what I would describe it as we're like you know as I prepared for the conversation to get you off the dully Wait list really Yeah, I just started you know reading about the space and that you know So, I think the philosophy brain and then also actually the computer science brain is like, wait, this is cool.

1:26:29And then I started reading all the academic papers of that era. And so, it was intellectual itch and the people. But then I stayed for the product opportunity, obviously. I post chat GVT when that took off, realized that we built a rocket chip. We launched it while building it. Maybe this is a synology. But I can't say that it felt like a hype to job or anything like that when I blood. So kind of a lesson there is follow as you said, follow the smartest people you know. There's also just this thread of follow things that are interesting to you. Just you playing with Dolly, led to this opportunity.

1:27:10Yeah, yeah. And actually that's something we still test for is curiosity is like an attribute that we think matters so much more than your ML knowledge. I'm not making a comment on research hiring, I think you do need some of the knowledge. I'm afraid, but you know, on like for product and engineering and design people and you know those kinds of functions, I actually think that if you are just curious about the stuff works, it doesn't matter at all if you've never done it before. In fact, if you were to filter for people who've done it before, you would have a very narrow filter of very lucky people, rather than necessarily the best people you can get.

1:27:43So I think we've scaled that certainly what got me here, but I think it's actually just generically been a good predictor of success. So Nick, I told you I had a billion, I said a two billion questions to ask you. I feel like I've asked a lot. I feel like I still have a billion left. But I know you told me right after this, you have a big GPT -5 check and then you got to get to. So we got a ship. We got a ship. It's better ship now that this is recorded and we're putting this out. It's true. This is the forcing function. Okay. So before we get to very exciting, like, do you know, is there any else that you want to share, leave listeners with, think is important to share.

1:28:20I try to share a little bit about how I made decisions because I hope to, you know, I'm not that far out of school. I like really a lot to people who are coming in the job market who are trying to figure out what to do their life right now. And I feel very confident that if you surround yourself with people that give you energy and if you follow the things you're actually curious about, that you're going to be successful in this era. So, my parting advice to folks really is put yourself around good people and do the things you're actually passionate about because in a world where this thing can like, you know, answer any question, asking the right question is very, very important.

1:29:00And the only way to get, you know, learn how to do that is to nurture your own curiosity. So I work for me and it's the one reputable thing that I can share everything else's luck. And this is counter to what a lot of people are doing right now, which is follow the money. Where can I make the most? How do I grow this thing and make $100 million? Like all these people that are getting these crazy offers were not planning to make a lot of money doing this. It's quite interesting to see that stuff fly out because I think all these people entered you know, school for genuine reasons. They were like excited about the space.

1:29:37They were researching it. They were pursuing knowledge. And I'm happy that that's being rewarded. And I don't know what the rewards will look like in the future, especially in a post -age E .I. world. But I just have a feeling that if you, you know, if you follow that advice, you'll end up okay. With that, Nick, we've reached our very exciting lightning round. I've got five questions for you. Are you ready? Yeah. What are two or three books that you find yourself are commanding most to other people? In a product space, probably things like high output management or the design of everyday things or those kind of classic type things because I think they're extremely applicable.

1:30:12And we talked about philosophy. I don't know. Is there a philosophy book you would be like, here's the one to read if you're going to come to it? Anything by like, falls and nosic. Like I like the political stuff. I think it's really fun. That is the type I think I recommend. And only there's a practical reason to read that stuff. but I will nerd out about it with you. So, that your own peril. Do you have a favorite recent movie or TV show? You've really enjoyed it if you've had time to watch anything. I think you've got to do a little bit of sci -fi to be in the space. You shouldn't copy any of it, but I think you learn from it.

1:30:43So regularly rewatch her and rest world. Severance was great. I think that's the stuff that, you know, when I have time I'll, I'll, I'll metal with. That is awesome. I love that those are the two. of all the sci -fi movies. Those are the ones you resonate most with and find most interesting and valuable. Yes, but that's probably my own limitation. So I'm sure there's more to discover. By the way, have you read Fire Upon the Deep? That's my book. I don't know if you have time to read this book, but I think you would love it. It's such a good AI -oriented sci -fi space opera sort of book. Yeah, yeah, okay.

1:31:23Okay, is there a favorite? Do you have a favorite product you recently discovered that you really love? I actually don't. I am like at extreme capacity. It's kind of interesting. Sometimes like, you know, if the I developers ask me, it's like, hey, are you like, you know, copy over a product that is like, I actually just do not have time to follow up. You know, what's going on outside of open AI? because the pace here is so intense. So don't have a good rest for you, I'm afraid. That's a really, that's a comfort against her. I think a lot of probably companies don't figure it. And it does no time to even listen to her stuff.

1:31:59Oh man. Okay, do you have a favorite life motto that you find yourself using when things are tough, sharing with friends or family, that other people find useful? Being the average of the five people you spend the most time with is like a thing that really internalized. And both of my personal life where there's like, people who give me energy and who lift me up and make me like a better person. My fiance is one of those people, but you know, if there's many people in my life. But then there's also just like, you know, at work, there's the equivalent. And again, that's how I've made all the career decisions.

1:32:31It's like, you know, who do I want to learn from? So I apply that principle constantly. Final question. Everybody I talked to told me that you are a very good jazz pianist. You have one competitions. I think you were planning to do this as your main thing. And then you somehow took the side quest. Yeah, I chickened out of that at the very last minute, but I was gonna is gonna go to school for music and that's still my like hopefully chapter two I love that that might still happen might still happen now. I'm like I've been some some some for fun bands And we will get from time to time. It's like that the one thing I can do when I'm otherwise You know super tired and can't can't can't think anymore because it bounces me out and in good ways but yeah, hopefully I'll get to do more of it.

1:33:15I'll end the picture. Is there any analogs between music and your job, anything that you find? Yeah, actually, I feel like, I feel like you could think of software development as like, you know, or being a product person as you could be a conductor of an orchestra or you could be in a jazz band. And I think of it as a jazz band, where I'm like, don't believe in the idea of everyone having this like set part that they have to play. in me telling people when to play. I love how in jazz or other forms of improvise music, you're kind of riffing off of each other and you listen to what one person played and then you play something back.

1:33:54And I think that great probably development is like that. In the sense that ideas get come from anywhere. It shouldn't be a scripted process. You should be trying to fowl, having fun, having play and what you do. So I use that analogy a lot for those who like music. It's a resonant. Nick, I'm so thankful that you made time for this. I know today is insane today. Tomorrow is gonna be even more insane for the entire world. They have no idea what's coming. Thank you so much for doing this. Two final questions, work in folks. Find you if you want them to find you online, work in folks find GPT -5 potentially, and then just how can listeners be useful to you?

1:34:30Just use the product. You don't even have to pay. Should be your default model starting tomorrow. and just use it and don't think about models anymore. Unless you want to do in your pro users in which case you get all the little models, so rest assured. And useful, honestly, I learned so much from people at large and chat GPT users, et cetera. So just keep doing your thing. I am watching and learning and I appreciate all the feedback. So I'm sure after we fix the model chooser, you guys will roast me for something else and I'll click it. So keep it coming. amazing. Thank you so much for being here.

1:35:08Thanks for having me, Lenny. And good luck tomorrow. Thanks. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or a leaving review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny's podcast .com. See you in the next episode!