“Engineers are becoming sorcerers” | The future of software development with OpenAI’s Sherwin Wu

24:58And really what, you know, because what the one person billion dollar startup implies is that there's, you know, one person can just have so much more agency and so much more leverage using one of these tools that it is just super easy for them to get everything done that they need to for their business to, you know, ultimately create something that's a billion dollars. but I think there are a couple other implications of this so one of them is if it's easy for a person to create a one person or if it's possible for a person to create a one person billion dollar startup it also means it's way easier for people to just create startups in general like I actually think this will one second order effect of this is I think there's going to be a huge startup boom and small SMB style boom where anyone can build software for anything one one, you're kind of starting to see this play out in the AI startup scene where software's become a lot more vertical oriented where like these verticals, like creating some AI tool for some vertical tends to work quite well because you really lean into that particular domain.

26:03You like really understand the use case for it. And so if you play out AI, there's no reason why you can't have like 100x more of these startups. And so I think one world that we might end up seeing happen is in order to enable a one-person billion-dollar startup, there might be like a hundred other small startups building bespoke software that works extremely well to support other types of small, small, one-person billion-dollar startups. And so I think we might actually enter into a golden age of like B2B SaaS and just like software and startups in general. And so I think that's a really interesting trend to kind of see because as it gets easier and easier to build software, as it's easier and easier to run a company, you might actually just end up seeing way more of these startups.

26:53And so the way I've been thinking about it is like, yeah, there might be one person billion dollar startup, but there might be like a hundred, you know, a hundred million dollar startups. There might be tens of thousands of$10 million startups. And as an individual, it's actually pretty great to have a$10 million business. That's like enough for yourself or life at that point. And so, you know, we might really see an explosion in that way. And I feel like people aren't really, you know, pressing that in. There's another kind of like third order effect to this. And again, all of these, I guess, you get to the further and further out predictions, I think there's a lot of uncertainty.

27:29I think if we end up moving to this world where you end up with these kind of micro companies building software that works for one or two people who own the company and are working there, I think the startup ecosystem will change. I think the VC ecosystem will change. We might end up in a world where there's just like a handful of big players that are offering platforms and supporting all of these startups. But, you know, the types of venture scale return startups that can really 100 or 1 ,000x your investment might actually end up shrinking if you end up having a bunch of these, you know, smaller 10 to$50 million companies, which are not great for venture solid returns, but are great for the individuals, the high agency individuals who are now, you know, really leaning to AI to build these businesses for themselves.

28:12I love how many order effects we've been through. I want to hear the fourth order effect now, Sherwin. I'm just joking. Fourth order is too gigabrain for me. I can't think that far ahead. It's like Inception where just everything gets slower every time you go deeper into someone, every layer. Okay, so the billion-dollar startup, I think about this a lot because I'm not going to be a billion-dollar startup because what I'm doing is not venture-scale in any way and not super high leverage. But just seeing how many support tickets I get from just like the most ridiculous things, it's hard for me to imagine one person.

28:51Like I'm bearish on this billion dollar startup. I just want to share this thought simply because of the support costs. Even if AI is helping you at a billion dollars, just like unless your ACVs are very high and you have very few customers, it's just dealing with support. And people are like, you know, like they can solve their own problems, but they're like, I'll email support. I'll ask about this thing. Just dealing with that is hard to scale is in my experience. So unless you have, in my opinion, unless you have a bunch of contractors, which I don't know, does that count as a single person company?

29:21I feel like it's very difficult to scale a billion dollar startup and not have someone helping you with at least the support work. And AI, I think will only take you so far. So I think that's true. And actually, I think my view on it is slightly different, which is I think that your, you know, Lenny's podcast might end up becoming a billion dollar startup. But what I think might happen is instead of you kind of being the one person who has to dispatch an AI to solve and fix those support tickets, I think what might end up happening is there might be a whole smattering of other startups that are building software and super and and super tailored towards what you might need.

30:03And so there might be 10 or 20 startups that build support software for podcasts and newsletters. And that might be a one-person startup. It doesn't need to be a big one. And they might be able to just code up this product very, very easily. They're able to build their own thing. And because it's so tailored and unique and hopefully useful for you, it might be something that you purchase as the one-person billion-dollar startup. I would buy that. Yeah, there's like a question of like what you in-house and what you like kind of outsource. And what I think might happen is because the cost of writing software and building products is collapsing so much, you might end up outsourcing a lot of this and in doing so reducing the size of your company.

30:44And so that's kind of the world that I think might end up happening. Again, there's like high uncertainty in what might play out here. But the end result still might be a one like one person driving this like high, high, massive leveraged company that might actually reach a billion dollars. I could see that. I also think about Peter at Claudebot slash Moldbot slash OpenClaw of just like how he barraged he is right now by all these asks and emails and pings and DMs and PRs just like, I'm curious to see. And he's not even making any money out of this thing. Yeah, I can't imagine what it's like to be him right now.

31:13It must be like absolutely insane. It's probably like, you know, like the months after we launched ChatGPT, the craziness that was. As one man. He's coming out on the pod, by the way, in a week. Oh, that's exciting. Yeah. maybe the fourth order effect is distribution becomes increasingly important because there are so many freaking things trying to get your attention so people with an audience and platform i think become more and more valuable which is good good stuff okay uh i wanted to come back actually to your management stuff so i really loved your insight about spending more time with top performers has been really successful to you just thinking about you as a manager of a team that is building the platform that powers basically the entire AI economy, like every AI startup is building on your API.

32:00Clearly, you're doing a great job. What other kind of core management lessons have you learned? What do you find is really important and key to your success as a manager of engineers and just people? Yeah, I think a lot of the lessons that I've learned here, I don't know how specific it is to the OpenAI API or some of our enterprise products in particular. I think my management philosophy has obviously changed over time, but I think it's probably stayed the same more than it's changed over time. One of these principles is kind of what I talked to you about before, which is spending a lot of time with top performers, actually spending, and to be very concrete, it's more than 50 % of your time with your top performers, with maybe your top 10 % performers, and really, really trying your best to empower them.

32:48The way that I think about it is kind of come back to this analogy of software engineer as a surgeon, which comes from the mythical ManMonth book. So it's actually, it's funny. So I pull it from the book, but in the book, they actually describe this world where I think they were like predicting the future because I think the book was written like in the 70s or something. They said that software engineering might end up moving into a world where the software engineers are like surgeons or like in a surgery room there's like one person doing the work um and you know there's the one person like cutting whatever and like doing all the surgery and everyone else in the room is there to just support them right it's like the nurse and like the assistant the resident and the fellow and then the surgeon's like i need a scalpel and they give them a scalpel and then uh they're like i need you know this tool and that's machine and they'll bring it over everyone's there to just like you know support the one uh surgeon and so the the the mytho mammoth actually predicted that that is kind of the direction that software engineering is going to go.

33:45I don't think that's exactly played out where like, you know, it's much more collaborative and like it's not only one person doing the work, but I've always really liked that analogy. And that analogy is actually what I strive to kind of like emulate in my own management philosophy, which is software engineering isn't really like surgery, where it's not just one person doing work, but the way in which I like treating the people on my team and the way that I act as a managers, I want to empower them, make them feel like they're a surgeon. And in so far as like, making sure that I'm supporting them and making sure they have everything that they need to do their work.

34:18And it feels like they have an army of people kind of supporting them, and looking around corners and giving them everything that they need, when it's really just me as the manager. And so like, the example that I give is looking around corners and unblocking people, especially from an organizational perspective is extremely, extremely useful. And again, going back to the AI conversations, even more important nowadays, right? Like, if people are just like cranking PR after PR, the main thing bottlenecking progress and shipping something tends to be organizational or like process oriented. And if you as a manager can kind of look around corners and kind of unblock the team, if you can, like if the surgeon needs scalpel, but the manager kind of already has a scalpel ready for them, that's the best case scenario.

35:01That's kind of the way that I approach management and especially engineering management. And so that's something that's really, really stuck with me over time. And even though, you know, software engineers aren't exactly surgeons, that metaphor has always kind of stayed in my mind as of the rest of my career. I love that. And I feel like I wonder if that's something I can help with is look around corners and predict here, this engineer is going to be blocked by this decision. We need to figure this out. We need to get it. Yeah, that's actually a really good point. I haven't tried this yet, but I wonder what would happen if I ask ChadGPT hooked up to the company knowledge, you know, like what are the active blockers?

35:36Look through all the Notion docs. What are, maybe Slack messages, you know, it's probably in Slack somewhere. What are the active blockers on my team? And is there something I can do to help? Now, that's very interesting. I have not thought about that, but you're right. We just had an insight right here. Yeah, yeah, yeah. And it's, I think even more interestingly, what do you anticipate will be a blocker for this engineer or this team in the coming months? Yeah, you ask the model, you ask the AI to do the second and third order things. Anticipate that and anticipate what the bloggers will be next month, too.

36:04I think we've got a good idea right here. Yeah, yeah.

36:31funnels, dive into retention, and explore their growth metrics. Where other tools stop, Datadog goes even further. It helps you actually diagnose the impact of funnel drop-offs and bugs and UX friction. Once you know where to focus, experiments prove what works. I saw this firsthand when I was at Airbnb, where our experimentation platform was critical for analyzing what worked and where things went wrong. And the same team that built experimentation at Airbnb built Epo. Datadog then lets you go beyond the numbers with session replay. Watch exactly how users interact with heat maps and scroll maps to truly understand their behavior.

37:06And all of this is powered by feature flags that are tied to real-time data so that you can roll out safely, target precisely, and learn continuously. Datadog is more than engineering metrics. It's where great product teams learn faster, fix smarter, and ship with confidence. Request a demo at datadoghq.com slash Lenny. That's datadoghq.com slash Lenny. Okay, I'm going to shift to talking about the API and the platform that you all build. So you work with a lot of companies implementing your API, your platform building on your tools. You told me that you find that a lot of companies actually have negative ROI on their AI deployments, which I think is what a lot of people read about and feel and think.

37:51And it's interesting actually seeing that. What's going on there? What are they doing wrong? What's happening in the world of AI and deployments in ROI? Yeah, so to be clear, I don't like explicitly see quantitative numbers around this. You know, it's actually really hard to measure these things. But especially from observing some companies kind of trying to do AI, I would not be surprised if a lot of AI deployments are actually, you know, negative ROI. I mean, part of this too is I think there's also general sentiment from folks around the country, like basically outside of tech, that AI is being forced onto them.

38:27And I think part of this is probably a symptom of some negative ROI, AI deployments. A couple of things I've observed around this. So one thing is, and I think I come back to this again and again, like, I think we in Silicon Valley just forget that we live in a bubble. Like, we are so, like, Twitter is a bubble, sorry, X is a bubble, Silicon Valley is a bubble, software engineering is a bubble. Most people in the world, most people in the US are not software engineers, are not very AI-pilled, are not following every single model release. And so we're just like highly out of the loop on how to use this technology.

39:05And so, you know, like we always talk about all these like best practices for codecs, all these like codecs-pilled people within OpenAI. I'm sure everyone on X who posts are like crazy power users of these AI tools. You know, they lean into skills, they lean into agents.md. MCPs. Yes, yeah, all of that. And when I talk to some of these companies, and I talk to the actual employees using these, it's like the most basic thing that they're trying to do. And they like have very little understanding of exactly how the technology works. And so that's kind of like one big observation for me, which is like, they're asking very simple questions of these things.

39:46They're really not pushing it just yet. And so that kind of goes back to, that kind of ties into what I think more companies do or like what could do or what a more ideal AI deployment setup looks like. And this is kind of how we've run things within OpenAI too. The companies where I think it started to work really well have a combination of both top-down buy-in. So it's like the C-suite is like, you know, we want to become an AI first company. So there's buy-in, they buy the tools, they have, you know, exec support. but it also has bottoms up adoption and buy-in. And so what I mean by that is it has like actual employees doing the work who are really excited about the technology and are willing to learn, evangelize, build best practices, and kind of like knowledge share within the organization.

40:32We've seen this a lot internally. So like obviously OpenAI has always wanted to be a very AI-centric company, but when it really started taking off was with the introduction of Codex and these tools where like people, like actual employees themselves could start applying it to their work. And I think you really need this because at the end of the day, everyone's work is like very different. It's like very unique. Software engineering is different than finance, is different than operations, is different than go-to-market and sales. And so there's like a lot of these like last mile intricacies of work that needs to really be done in a bottoms up fashion.

41:06And so my sense is a lot of these, these AI deployments don't have, like don't have bottoms up adoption. Like it was like an exec mandate and it's extremely top down and it's very divorced from what the actual work looks like. And as an end result, you end up with a giant workforce that doesn't really understand the technology is like, I know I'm supposed to use this. And maybe it's like on my performance review, too, but I'm not sure what to do. And they look around. No one else is doing it. There's no one else to learn from. And so my recommendation for companies kind of pushing this is find, or maybe even staff a full-time team internally that is this kind of tiger team internally that can explore the full extent of the capabilities, apply to specific workflows, do the knowledge sharing, create excitement within folks who might want to use this technology.

41:52Because in the absence of that, it's actually very difficult to pick up. And who would you put on this tiger team? Is it like engineer-led? Do you find in your experience? is a cross-functional sort of team. Yeah, it's interesting. Also, a lot of companies don't have software engineers. And so the pattern I've seen is it tends to be these software engineering adjacent, basically technical people, but are not software engineers. I think those are the ones who tend to get most excited around this. It's like maybe the support team operations lead who doesn't code, but loves using these tools and is like an Excel wizard or something.

42:34And so it's like technical adjacent or like coding adjacent and pretty technical. Those are the kinds of people I've seen in these companies who just really light up and get excited around this. And you can usually build a team around that. But yeah, it's oftentimes not software engineers. Software engineers, I think we'll understand this, but not every company has software engineers. It's actually kind of a rarity. They're hard to find, they're expensive. And so it's these other types of folks. What I'm hearing is the anti-pattern is top-down. This is very the CEO, founder, exec team, just like, we are going to go AI first.

43:05We're going to lean into AI. Everyone's going to be judged on their performance using AI tools, how much your productivity is increasing thanks to AI. And without that being just top-down and not creating a team that is bottom-up, spreading the gospel, you find that doesn't work. Yeah, yeah, exactly. And the advice is find the people that are most excited. And instead of kind of having them spread out through the organization, what you find works is create a little AI kind of evangelist team that finds ways to use it and kind of spreads it across the work. Yeah, I mean, another, it's kind of like hearing you play back to me, another way to think about it, kind of tying back to my own management philosophies is find the high performers in AI adoption and empower them.

43:48You know, let them build hackathons, let them, you know, hold seminars, do knowledge sharing, kind of create the seeds of excitement internally. Okay, amazing. There's a couple hot takes I want to hear from you, something that I've seen you talk about and share. One is you've shared that talking to customers and listening to customers is not always the right strategy in AI and it might often lead you astray. I don't know if it's that hot of a take. I think the main thing here is, so obviously you should talk to your customers. It's like you still talk to customers. I just think the AI field, especially what I've seen over the last three years, working on the API and seeing all that evolve, is the field and the models themselves are just changing so, so quickly.

44:36They tend to disrupt themselves, especially around the tooling and the scaffolding space. So there's this quote that I read actually earlier this week from an X article by this guy named Nicholas, who's the founder of a startup called Fintool, where I think he was sharing a lot of the best practices that he has learned through building AI agents for financial services, I think at a startup Fintool. And he had this phrase that I thought was really good, which is, the models will eat your scaffolding for breakfast. Like if you look, if you rewind back to 2022, right when ChatGPT launched, these models are pretty raw.

45:10And there was like all this product scaffolding and things, especially in the developer space, to basically try and steer the model and build a scaffolding around it to get it to do what you want. Like agent frameworks, there's like vector stores, I think was like really popular back then. And just like a whole smattering of tools here. and as you've kind of seen the feel play out that the models have just changed so much and gotten so much better that they ended up, yeah, literally eating some of the scaffolding. And I think this is even true today. So I think the article from Nicholas actually, you know, the current scaffolding, which is fashionable, is skills, files-based context management.

45:50I could see a world where at some point, you know, that's no longer useful, where the model can actually, you know, manage all that themselves or like, you know, So, or there might be, you know, it's hard to predict, but like might move on to some new paradigm where you know, I already need this file-based like skills type thing. You have literally seen this play out, right? Like the agent frameworks, I think are a little less useful now. There was a period of time like 2023 where we thought vector stores is gonna be like the main way for you to, you know, bring organizational context into the models.

46:19And you need to, you know, vectorize and embed every bit of your corpuses. And then you need to do all this work to like figure out the vector search, to like optimize that, to pull out the right information in the right time. All of that is scaffolding because the model, you know, was not good enough. And it turns out, you know, in this case, it turns out as the models get better, a better approach is actually to take out a lot of that logic and trust the model and give it a set of tools for search. It doesn't need to be a vector store. You could actually just hook it up to any type of search.

46:47It could literally be files on a file system like skills and agents MD to kind of steer it as well. Obviously, there's still a place for vector stores. I know a lot of companies are to using it, but the entire scaffolding around that and building an entire ecosystem around that and assuming that's the only scaffolding that you need has really changed. And so tying this back to the like, you know, you don't always have to listen to your customers because the field is changing so much at any point in time. You know, a lot of people are kind of in this local maximum. And if you just blindly listen to your customers, they'll be like, yeah, I want a better vector store.

47:20Like, I want a better, you know, agent framework for this. and if you had just kind of only chased down that path, it actually would have led you to build something that again is the local maxima. Whereas as the models get better, we've had to reinvent and kind of rethink the right abstractions and the right tools and frameworks to build around these models. And the cool slash exciting slash kind of crazy, annoying part is it's a moving target. And so yeah, like the current smattering of tools and frameworks right now will likely need to evolve and change pretty significantly over time as the models get smarter and better.

47:57But that is just the nature of building this space. I think that's what makes it exciting. But it also means when you talk to customers, you kind of need to balance the exact feedback that they want with where you think the models are going and where you think things will trend over the next one or two years. It's interesting how this is, the bitter lesson is, you know, this big lesson that AI and ML folks learned, which is just like, the less you overcomplicate, the less logic you add to machine learning, to AI, the more it'll be able to scale and grow and just like take it all away and let it just just compute basically just give it more power to to get.

48:31Yeah, there's literally a version of the bitter lesson applied to like building with AI where you know, we were trying to architect all this stuff around and turns out the models are just kind of, you know, eat it all away. And and and honestly, like open AI API team has like been guilty of this, where we kind of like took some, left and right turns when we shouldn't have. But yeah, the models still end up, models get better and we're all learning the bitter lesson day in and day out. So what would be the key takeaway for folks building on, say, the API or just building agents and having to build a little bit of this around for now?

49:05Is it just, yeah, what would be the advice? My general advice, and I've been giving this to people for a while and I think still true today, is make sure you're building for where the models are going and not where they are today. you know the it's clearly a moving target and I think a lot of the companies that I've seen startups that I've seen really really do well is they build a product for an ideal type of capability that is like maybe 80 % of the way there today and they end up having a product that kind of works but is just almost there but then as the models get better suddenly it might click and then their product now is incredible because it works, you know, like maybe with like, oh, oh three at some point, it suddenly works with 5.1, 5.2, suddenly it unlocks it.

49:51But they're building these products with the like the model capability improvements in mind. And with that, you end up creating an experience that's way better than if you had assumed that it's static in the first place. And so that'd be my general advice, which is, you know, build for where the models are going and not where they are today. You end up building a better product. You may need to, you know, like wait a little bit, but like, you know, the models are getting so much better so quickly. You often don't need to wait that long. So to follow that thread, where are like in the next six to 12 months, where is the API heading?

50:23Where's the platform heading? Where are the models heading? As much as you can share, I know there's a lot of secrets here that maybe you're more excited about, or do you think that people should start to prepare for however much you can share? I mean, so the obvious one is how long of a task these models can do coherently. So there's like the meter benchmark that I think tracks software engineering tasks and how long of a task can these models do 50 % of the time, 80 % of the time. I think we're at something like multi-hour tasks being able to be done by software engineering tasks being able to be done by these frontier models 50 % of the time.

51:02And then I think 80 % is something like just under an hour. But the The sobering thing about that chart is they plot all the previous models on this chart as well. So you can really see the trend of this. That's something that I'm really excited about, which is, you know, I actually think products today really optimize for tasks that the model can do for, like, minutes at a time. Like, even codecs and, like, the coding tools, I'd say, like, you know, it's in the CLI. You're kind of, like, seeing it be interactive. It's really, you know, quite optimized well for, like, maybe at most 10-minute type tasks.

51:33I have seen people push codex to the limit into like multi-hour long tasks. But again, I think that that's more of the exception. But if you follow this trend, like I think like in the next 12 to 18 months, we could see models that could do multi-hour long tasks very, very coherently. At some point, it might reach like, you know, six hours a day long task where you kind of like dispatch it and have it do, you know, do things on its own for a while. The types of products you build around that will look very different. You want to give the model feedback. You obviously don't want it to completely run wild for a day.

52:05Maybe you do, but you probably don't. And then the universe of things you can have the model do really expand. So that's something that I'm really, really excited about seeing. Another thing over the next 12 to 18 months where I think would be really cool is improvements in the multimodal models. And actually by multimodality, I'm mostly thinking about audio here where the models are pretty good at audio. I think they're going to get a lot better at audio over the next 6 to 12 months, especially the native multimodal model, the speech-to-speech ones. I think there's also interesting work being done around new types of models and architectures on the multimodal audio side as well.

52:47But audio, especially in the enterprise and in a business setting, I think is a hugely underrated domain still. Everyone talks about coding, it's all text. But we're talking in audio. A lot of the world's business is done via audio. A lot of services and operations are done via talking in audio. And so I think that area is going to look very exciting in the next 12 to 18 months. And I think there will be even more unlock for what we can do with audio models there as well. Amazing. So quick summary, expect agents and AI tools to run longer to that trajectory to continue to increase and then audio and speech becoming a bigger deal, more first party and native and better and core to the experience.

53:34Yeah. Extremely cool. Okay, I want to go back to one of your hot takes, another hot take that I've seen you discuss. You're very bullish on business process automation as an opportunity in the world of AI. Talk about that. Yeah, this goes back to the thing that I said previously, which is we live in a bubble in Silicon Valley. And a lot of the work that we do, that we're used to software engineering, product management, building products, is very differently shaped than the work that goes on that runs our entire economy. And I see this in and out when I talk to customers. If you talk to any company that's not based in, it's not a tech company, there's a lot of business processes.

54:17And so what I mean by this is, I generally delineate it as software engineering is kind of like open-ended knowledge work. And this is why I think tools like Codex tend to be quite good because it's exploring and you're giving it these open-ended things. But software engineering is fundamentally pretty open-ended and it's not very repeatable. So you build a feature, you're not trying to build the exact same feature over and over again. And a lot of tech jobs are in this space. I think data science is kind of in this space as well. even some of the like strategic finance stuff but as you move further and further away from software engineering and like what what is core in tech a lot of jobs are just business processes they're like repeatable things uh repeatable operations um that's you know some manager at a company has kind of like iterated on um there's usually a standard operating procedure that people want to do uh and you don't want to deviate from it that much you know there's like in software engineering, the ingenuity is deviating.

55:18But a lot of the work being done in the world is actually just running through these procedures and operations. Like if I call a support line, they're running through one of these. If I call my utility company, there's a bunch of processes and things that they can and cannot do for me. And so I'm just extremely bullish on this general category of like, and I think it's underrated because it's so different from what we think about it in Silicon Valley, people tend to not think about it. But how can we apply AI and some of the tools and frameworks that we have towards this business process automation, towards automating and making easier repeatable business processes with high determinism that is fully integrated with business data and business decisions and different systems within an enterprise?

56:07And how How can I actually make that process better? Because I actually think there's a lot of opportunity and a lot of work to be done in that area. And we just don't talk about it because it's a little bit less in our wheelhouse. So your take here, just to make sure I fully understand it, is you think there's a much bigger opportunity outside of engineering for AI to impact productivity of companies and also jobs of these folks that are doing these kind of repetitive, easily automated tasks? Impact jobs and also just impact how work is done. like so much of work is done in this way. Like you think about, you know, like what a, like basically I talk to customers all the time, big enterprises, like how will AI transfer my company?

56:48Like how will it run in a world with AI in like 20 years? And, you know, software engineering is part of the story, but there's so much more on the business process side. And I actually think it might look even more different on the business process side. And the work there is pretty substantial. It's actually interesting. I don't know, like from an absolute percentage or absolute basis, I don't know if it's bigger or smaller than software engineering. Like software is pretty huge and pretty expensive as well, but it is pretty massive. And it's definitely bigger than, you know, it's bigger than you would think it is based off of how people talk about it or don't talk about it on X or Twitter.

57:23Okay, going in a slightly different direction, having built the platform, building the API, people building on the API, the biggest question on people's minds is always just, how do I not have OpenAI squashed my idea and build their own thing and then, you know, destroy this market I created? What's the general policy? What's the general philosophy of how startups should think about where OpenAI is unlikely to go? My general answer here is the market is so big and so massive. Like, I actually think, you know, startups should just not overly think about where OpenAI or these labs are going. I've talked to a lot of startups that have not worked out, startups that are doing really well.

58:08Every startup that I've seen that has kind of fizzled out is not because OpenAI or Big Lab or Google or something has come to squash them. It's because they built something and it really didn't resonate with the customers. Whereas the ones that take off, even in very competitive spaces like coding, Cursor is huge at this point. And it's because they built something that people really love. And so my general advice is don't overly stress about this. Just build something that people like and you will have a space in this. I can't overstate how big of an opportunity there is right now. The opportunity space and building with AI is so big.

58:42A good example of this is the space is so big that the Overton window of what is acceptable and not acceptable for VCs to do has completely changed here. VCs are investing in competitive companies left and right. It's just like the space is so big because the opportunity is unlike anything that we've seen before. and while that affects how VCs operate, from a startup perspective, it's like the most empowering thing in the world because even if you just build something that some people really, really love, you will end up with a massively valuable business. And so that's why I tell people, don't overly think about it.

59:16The other thing I also think is important to remember, at least from an open AI perspective, one thing that we've always held very near and dear, which both Sam and Greg helped reinforce from the top as well, is we actually view ourselves fundamentally as a ecosystem platform company. The API was our first product. We think it's really important for us to foster this ecosystem and continue to support it and not squash it. And so if you kind of look at the decisions we make, this is all weave through it. Every single model we've released in one of our products gets released in the API. Like even, we release these codex models now that are a little bit more optimized for the codex harness, but they always find their way into the API.

59:54and like all of our, you know, customers end up using those. We don't hold back on any of that. We think it's really important to keep our platform neutral. And so, you know, we don't block competitors. We allow people to have access to our models. We also want, you know, like we've recently been testing more of like the sign in with ChatGPT, you know, product as well. And so we want to foster this ecosystem. I think it's really important that we do so. The general like thinking about this is like, you know, a rising tide like lifts all boats. And, you know, we might be an aircraft carrier, like, pretty big at this point, but we think it's important to raise the tide because everyone kind of benefits and I think will benefit as well.

1:00:32Like, our API itself has grown pretty significantly because we act in this way. And so I'd really encourage people not to view OpenAI as this kind of, like, you know, thing that'll just shove people out of the way, but instead focus on building something valuable. And we, you know, remain committed to providing an open ecosystem. term why why is that important to open ai just this focus on building a platform creating a way for people to build businesses just like is that just that's been the vision from the beginning we want this to be a platform it's been the vision from the beginning it comes goes back to our charter actually like our mission um so the open a mission has always been to one to build agi so you know where i was thinking that but then the second thing is to like spread the benefits of it to all of humanity.

1:01:18And there's kind of like a lot of, you know, the main part, there's all of humanity. Like, and obviously, ChadGPT is trying to do this. You know, we're trying to reach however many, you know, the whole world. But very early on, and this is why we launched the API back in, I think it was like 2020 or something like really early. We don't think we as a company will be able to reach all of humanity, right? Like there's, I don't know, every corner of the world is like pretty deep. And so we actually feel like in order for us to fulfill our mission, we need to have some platform style thing here where we can empower other people to build, you know, the customer support bought for podcasters and newsletter hosts because we're not going to be able to do it ourselves.

1:01:57And so we've largely seen this play out with the API. This is why we, you know, we talk to so many of our customers and really, you know, love seeing the diversity of things built on. But yeah, it's been there since A1 because it's kind of, we view it as an expression of our mission. And you haven't even mentioned the app store that you guys are launching, the ChatGPT App Store. Yeah. Is that under your umbrella, by the way, or is that a different org and team? It's a different team. So it's under ChatGPT. We obviously collaborate very closely with them. And, you know, they built like an apps SDK, which is a built-in close collaboration with our team.

1:02:28But that is more within the ChatGPT umbrella. But that is also another, like that's another example of this, right? It's like ChatGPT is like, we kind of like have these 800 million weekly active users who are just coming over and over again. It's a great asset to have as a business, but man, would it be better if we could somehow allow other companies to come in and take advantage of this as well and build for this audience as well. And then ultimately, we think it'll help us expand that group as well, right? And so it all kind of comes back to the mission and we find that being a platform, being open tends to help here.

1:03:05Just that number, 800 million, I think it's MAUs. No, no, no, it's weekly. Weekly active. almost a billion people using weekly. It's absurd how these numbers we're just used to now, but that's insane, unprecedented. Yeah, it's mind-boggling for me to think about from a scale perspective, honestly. And the way I think about it is like 10 % of the world, and growing by the way, it's shooting up. Come to chat GPT and use it every day. Sorry, every week. At this point, I just want to double down on this point. you're making open ai's mission was to make ai available to all of humanity and i think some people diss that they're like oh you know it costs money and it's like uh like the fact that it it's there's a free version of chat gpt that anybody can use that is not so different from the most powerful ai model that exists in the world for free that's not gated that anyone can use like if you have if you're a billionaire there's only so much more you can get out of ai than what someone you know in a village in africa can can get and i know that's always been really important to open ai yeah yeah i mean like uh that that's why i think we've leaned into the health work we've leaned into like i got like uh education is going to be very interesting here um the other insane kind of trend here is the free model has gotten so smart over time like the free model back in 2022 was you know like uh well it's good at the time but it's like nothing compared to what you get today because you get gpd5 today uh and so the like you know raising the floor across the world is kind of you know something that we're really trying to do and we view it as part of our mission the other flip side of this by the way is like you know kind of talking about like the billionaires or whatever i know people love saying like you're using the same iphone that like you know steve or sorry like mark zuckerberg's probably using or like the billionaires are using but for like 20 a month you're basically using you know like using the same ai that you know the billionaires are using.

1:05:00For like$200 a month, you get the same pro model that all the billionaires are using, but they're probably not using pro for everything. They're probably just using the plus tier ones for their day in and day out. And so, yeah, this kind of like democratization and just like spreading of this benefit like across all of the world is something that's really meaningful to us and something that drives a lot of what we do. One last question, just for folks that are thinking about building on the API or just like, oh, wait, I could do cool stuff with OpenAI's models and APIs. What does your API and platform allow people to do?

1:05:33Like, I know you can build agents on top of the platform. Just talk about what you allow. So fundamentally, the API offers a bunch of developer endpoints. And these developer endpoints basically let you sample from our models. The most popular one that we have right now is one called Responses API. And so this is an endpoint, and it's optimized for building long-running agents, so agents that'll work for a while. So what you can basically use, you can, at a very, you know, low level, you're basically just giving the model text, the model will work for a while, you can kind of, you know, pull it to see, see what it'll do.

1:06:07And then you'll get the model response back at, at some point. That's like the lowest level primitive that we have for people. And that's actually what a lot of people use. That's the most popular way of building on top of API. With that, it is like super unimpanied and you can do basically whatever you want. It's like the lowest level thing. We've also started building more and more kind of like layers of abstraction on top to help people build some of these. And so next layer up, we have this thing called the Agents SDK, which has also gotten extremely, extremely popular. This allows you to use, you know, the Responsys API or some other API endpoints that we have to build what you might more traditionally think of as an agent, like, you know, an AI kind of working in an infinite loop.

1:06:46It might have sub-agents that it delegates to. It starts building all this framework, All the scaffolding, actually. We'll see where this all goes. But it makes it a lot easier for you to build these kind of agents, giving it guardrails, allowing it to farm out subtasks to other agents and kind of orchestrate a swarm of agents. The Agents SDK kind of allows you to do that. And then above that, we've now started building tools to help also with kind of like the meta level of deploying an agent. So we have this product called Agent Kit. and widgets, which are basically a bunch of UI components that you can use to very easily build a very beautiful UI on top of either our API or agent's SDK.

1:07:32Because a lot of times these agents kind of look very similar from a UI perspective. And so there's AgentKit. We also have a smattering of evals products, evals API, where if you want to test and see if your agent or your workflow is working, you can test it in a very quantitative way using our eDolls product. And so yeah, I view it as these various layers. They're all kind of helping you build what you want with our AI, with our models, and with increasing levels of abstraction and how opinionated it is. And so you can use the whole stack and it very quickly allows you to build an agent or you can go down the stack as low as you want to basically response is API and build whatever you want because of how low upload is.

1:08:17Sherwin, is there anything else that you want to share? Anything else you want to leave listeners with? Anything we haven't touched on that you think might be helpful before we get to our very exciting lightning round? The only thing I'd leave folks with is, yeah, I think the next two to three years are going to be some of the most fun in tech and in the startup world that we'll have in a very long time. And I would just encourage people to not take it for granted. I entered the workforce in 2014. routine. It was great for like a couple of years. I felt like there was like a period of like five to six years where it wasn't very exciting in tech.

1:08:52And then in the last three years, it's just been the most insanely exciting, energizing period of my career. And I think the next two to three years is gonna be a continuation of that. And so I would encourage people not take it for granted. I'm trying to not take it for granted. At some point, you know, this wave is going to play out and it's going to be a lot more, you know, incremental. But in the meantime, we're going to get to explore a lot of really cool things, invent a lot of new things and change the world and change how we work. And so that's the main thing I'd leave folks with. I love this message.

1:09:19I want to spend a little more time on it. When you say don't miss it, what do you recommend people do? Is it just build, lean in, learn, join a company, building really interesting things? Like what's your advice to folks that are like, okay, I don't want to miss the boat? Yeah, I would just say engage with it. So it's basically like what you said, lean in, building tools on top of this is part of the story. Just using the tools. Like you don't need to be a software engineer to lean into this. I think a lot of jobs are going to change here. So just using the tools, understanding the limitations of what it can and cannot do so that you can kind of watch the trend of what it can start to do as the models improve.

1:09:58And yeah, and so it's basically like getting used and getting used to this technology and getting familiar with it instead of kind of like laying back and letting it pass you. On the flip side of that, there's a lot of, I think, stress and just anxiety around like there's so much happening. How do I keep up? I got to learn out. Clotbot this week. Oh, God. Is there something you learned about it? Just not like you're at the center of this. How do you not get overly stressed and worried about missing things that are going on and just stay on top of news? What are some things you've done learned?

1:10:28Yeah, so I think I'm personally a bad example of this because I'm basically chronically online on X and our company Slack. So I actually try and absorb. I end up absorbing a lot of it. What I will say, though, is just like from observing other folks who are less, you know, addicted to this stuff like I am. Yeah, a lot of it is noise. Like you don't need to, you don't need to have like 110 % of this kind of pass your mind, like go into your mind. Honestly, just leaning into like one or two different tools, starting small is already like, you know, more than you need here. I think just the combination of like the frenetic pace of the industry X as a product just creates like this insane kind of like, this insane pace of news, which is honestly very overwhelming.

1:11:17The main thing is you don't need to know all of that to really engage with what's happening right now. And even something as simple as just install the Codex Client, play around with it. Install ChaggyBuchin, connect it to a couple of your internal data sources, Notion, Slack, GitHub, and see what it can and cannot do. All of that, I think, is a part of it. Amazing. Sherwin, with that, we've reached our very exciting lightning round. I've got five questions for you. Are you ready? Yeah, yeah, absolutely. First question, what are two or three books that you find yourself recommending most to other people?

1:11:50I'll talk about one nonfiction, one-on-one fiction book. The fiction book was, I just finished reading it. I really recommend it. There is no anti-mimetics division by QNTM. I think it's like an online author, but I saw it being shared on X. It's like a science fiction-y kind of book. um and it was i basically devoured it in like two days um it was it's super super well written super fascinating it's about a government agency that's fighting you know things that make you forget it um and so it's just a very like smart like creative book that that and fresh uh honestly in terms of like source material uh that that that i really like so i'd recommend that one uh the book is also unintentionally hilarious so like it's like meant to be like this like sci-fi almost like horror style book, but it was, it was, it made me laugh a couple of times.

1:12:40So that's the, that's the fiction book. Nonfiction, so I'm going to cheat and I'm going to recommend two of them. So in the last year, I've been reading a lot more about China and kind of like the US-China relations. And I think there are two books that came out in the last year that have been, you know, really, really eyeopening for me in that regard. First one is the Dan Wang book, Breakneck. That one was really, really good. I really liked his analogy of like the lawyerly, US is the lawyerly society. China is the engineering society. And there are pros and cons to each. I read it and I was like, hmm, yeah, it does seem like we're run by lawyers in the US.

1:13:13So that's one. And the other one is the Patrick McGee book on Apple in China. It was super, super interesting. I'm a huge Apple fanboy. If you could see my desk right now, it's all Apple stuff. But just like, one, it was just super fascinating learning about Apple's relationship to China. And then two, it just like had a lot of inside information about Apple as a company that I found fascinating. So it was also quite a page turner and also, you know, very, very timely, timely book as well. The Antimimetics book sounds amazing. I'm buying it right now as you're talking. Yeah. Yeah. It's like, I think it's only like a couple hundred pages.

1:13:46I literally finished it in two days. It was just like so, so good. Okay. Great tip. Okay. Favorite recent movie or TV show you have really enjoyed? Yeah. That one's tough because, you know, I have two kids and a busy job. And so I really haven't had much time to watch TV shows. I will say in the last couple of weeks, I watched a couple episodes. I'm actually a big anime guy. And so I watched a couple episodes. There's a new season of this anime called Jujutsu Kaisen that's out. So season three of JJK was really good. In general, I'm a huge fan of Japanese anime. I think they create the most novel and unique plots and universes that Western media has shied away from.

1:14:33And so generally a big fan of that. But yeah, I haven't really watched much, but saw a couple episodes of JJK recently. Extremely understandable in your role. Yeah. Favorite product you recently discovered that you really love? Yeah. Okay. So I recently had to set up Wi-Fi and home networking. and I went all in on Ubiquiti routers and security cameras. I'd never heard of it before I had to do this. I always just had a very simple setup. And it's just such a well-built product. I don't know if you've used it before, but it's basically like the Apple of home networking. So beautiful products. But the thing that actually makes it extremely good is that software is good.

1:15:12And so they have a really great mobile app to help manage all of the home networking. And so basically Ubiquiti, you can use it to buy wireless routers. You need Ethernet wiring throughout your house to use it. But I actually think what makes it really good are security cameras. So if you have security cameras that are plugged into Ubiquiti ecosystem, they have an incredible mobile app, an Apple TV app, an iPad app, to kind of see the live feed of your cameras. And so they're a little pricey, but not that pricey. But it's been just an incredible product experience. All right. I went Euro, so I made a mistake.

1:15:49Good tip. Heroes are pretty good too, but I'm fully converted to ubiquity at this point. Okay, good tip. Okay, two more questions. Do you have a favorite life motto that you find yourself coming back to in work or in life? Yeah, the one that I always repeat to myself is never feel sorry for yourself. There's a lot of things that are going to happen at work, in life, and reminding yourself to never feel sorry and that you always have a sense of agency to kind of pull yourself up is something that I've had to tell myself a lot. and also something that I repeat to a lot of other folks as well. Last question.

1:16:23So in your previous life, you worked at Opendoor, where you led work on basically figuring out how much to pay for houses. You basically built the model that told the company, here's how much we'll pay for this house. What's like a variable in the price of a house that you didn't expect is really important and impacts the price of a house? There's a bunch that were surprising. I'll maybe list the couple of most interesting ones. So power lines and like high voltage power lines like are super, super actually impact your price quite a lot. I didn't really fully internalize this until I went to like Dallas and observed like when your house sits next to one of these giant like, you know, voltage lines is like buzzing.

1:17:03And most people have families. You don't want your kids kind of near there. So I think that was one that really, really kind of surprised me. That makes sense. yeah and then the other one which which was something that was always something really difficult for us to uh quantify uh was floor plans uh and so it is very important like yes of course it's really important but just like quantifying what a good floor plan is like and what a really bad floor plan is like we were doing all these things like how wide is the kitchen and like is it a what style of kitchen is it and then like where's the master bedroom and and so it was just really really hard to quantify but i remember floor plan was a big one because like we'd have a home that like wouldn't sell and then our uh ops team would go in and be like yeah that's the floor plan issue so like how do you how could you tell us like you go inside you just feel it it feels you know the floor plan feel feels off uh so yeah those are ones that were uh surprising and then the last one that was more impactful than i thought is um general like curb appeal and like even like the front door uh and so i actually think there's a zillow book on on this where the front door replacement tends to be the highest roi uh for homes um but just like the feel of like as you walk up to the home as a buyer, what you're interacting with and the first moments of the house, I think was, I'd underrated its importance.

1:18:18That is extremely interesting. And I love that you had to figure how to do all this in code and not walk around with these houses. Yeah, and then floor plans. I have a bunch of stories around like for floor plans, there's like, it's not digitized. So there's like a handful of people who have like paper floor plans of like all these homes in like Phoenix and Dallas. Yeah, a lot of fun stories from the open door days. Okay, Sherwin. Thank you so much for doing this. This was incredible. Where can folks find you online? And how can listeners be useful to you? Yeah, so I'm online on Twitter, on X.

1:18:48I'm just at Sherwin Wu. And yeah, I mostly just tweet about OpenAI and API and some of the products that we're launching. And then how folks can be useful to me. I love hearing about things that people are building. And so if you're working on a startup, if you're hacking on an idea, you know, would love to just reach out to me on X. I would love to hear about what you're building. and learn about how OpenAI can help support you. Amazing. Sherwin, thank you so much for being here. Yeah, thank you, Lenny. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

1:19:24Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.