Inside OpenAI | Logan Kilpatrick (head of developer relations)

23:36And then again, giving the model enough context to answer the question, I think again, that prompt actually might work really well for somebody who like has it, like if you were interviewing like Tom Cruise or something like that. So he has a lot of information about them on the internet. It probably works a little bit better. So the advice there's just give more context. It doesn't tell you, hey, I don't actually know that much about Logan, so give me some more information. It's just like, here I go. Here's a bunch of good questions. Exactly. It wants to, like, it so deeply wants to answer your question.

24:04Like it doesn't care that it doesn't have enough context. It's like the most eager person in the world you could imagine to answer the question. And without that context, it's just hard to do to give them anything of value. If we got T -shirts printed, they should say, like, context is all you need. Context is the only thing that matters. Like it's such an important piece of getting a language bottle to do anything for you. Any other tips, just as people are sitting there, maybe they're good, they have chat GPC open right now as they're crafting a prompt. Is there anything else that you'd say would help them have better results?

24:37We actually have a prompt engineering guide which folks should go and check out and it has some neat examples. It depends on sort of the order of magnitude of how much performance increase you can get. There's a lot of really small silly things, like adding a smiley face increases the performance of the model, like telling the, you know, you've seen, I'm sure folks have seen a lot of these silly examples, like telling the model to take a break and then answer the question. All these kinds of things, and again, if you think about it, it's because the corpus of information that's trained these models is the same things that humans have set back and forth to each other.

25:13So like you telling a human, like when I go take a break and then I come back to work, like I'm fresher and I'm able to answer questions better and like to work better. So very similar things are true for these models. And again, when I see a smiley face of the NSL -1's message, like I feel empowered that like this is going to be a positive interaction. I should like be more inclined to give them a great answer and more effort on the thing that they asked me for. Wow, wait, so that's a real thing. If you had a smiley face, it might give you better results. Again, it's like the challenge with all this stuff is like it's very nuanced and it's also like, it's a small jump in performance.

25:46You could imagine like on the order of like one or two percent, which for a few sentence answer is like might not even be a discernible difference. Again, if you're generating like an entire saga of texts, like the smiley face like could actually make a material difference for you, but acrylic something small and textual and by not. Okay, good tip. Amazing, okay. We've talked about GBT's. I think maybe might be helpful to describe what is this new thing that you guys launched GBT's? And I'm curious just how it's going this because this is a really big change and element of open AI now with this idea that you could build your own little kind of mini and I'm almost explaining it, your mini open chat GBT.

26:24And then people can, I think you can pay for it, right? Like you can charge for your own GBT or is it all for you right now? It's all for you right now. Okay, it's all for you. Okay, in the future I imagine people will be able to charge. So there's this whole store now. Basically it's the whole app store that you guys have launched. How's it going? What's happening? What surprised you there? What should people know? Yeah, it's going great. And again, historically the thing that you would have to do, let's say for example, you have like a really cool chat GBT use case where you would have to do to share it with somebody else is like actually go in and like start the conversation with the model, like prompt it to do the things that you wanted to.

26:59And then you would share that link with somebody else before the action has actually happened and be like here now you can like essentially finish this conversation with chat GBT that started. So GBT's kind of changes this where you take all that important context, you put it into the model to begin with and then people can go and like chat with essentially a custom version of chat GBT. And the thing that's really interesting is, you know, you can upload files, you can give it custom instructions, you can add all these different tools, like a code interpreter is built in, which allows you to like do like math essentially, you have browsing built in image generation built in.

27:33And you can also like for more advanced use cases if you're a developer, you can like connect it to external APIs. So you can connect it to the notion API or Gmail or all these different things like have it actually take actions on your behalf. So there's so many cool things that people are unlocking. And what's been most exciting to me actually is like the non -developer persona is now empowered to like go and solve these like really, really, really more challenging problems by giving the model enough context on what that problem is to be able to solve it. Going back to like context is all you need.

28:04Like this is very true in the context of GBT's. And if you give it enough context, like you can solve much more interesting problems. There's so many things that I'm excited about with this. Like I think monetization when it comes to the store later this quarter, I think is going to be extremely exciting. Like when people can get paid based on who's using their GBT's, that's going to be a huge unlock and like open a lot of people's eyes to the opportunity here. I also think like continuing to push on making more capabilities accessible to GBT's for people who can't code is really exciting like having to, even for me, as like someone who is a software engineer, like it's not super easy to like connect the notion API or the Gmail API to my GBT.

28:45And like really I'd love to just give it like one click signed in with Gmail then all of a sudden it's like, my Gmail is accessible or like someone else can sign in with their Gmail and make it accessible. Also, I think over time like all those types of things will come, but today it's really like custom prompts is essentially like one of the biggest value ads with GPD's. Awesome. I have it pulled up here on the, on different monitor and canva has the top GBT currently. And I was trying to play with it as who's chatting just to see I was going to make a big banner that said it's the context stupid and it doesn't.

29:15I'm not doing some right, but I'm not paying that much attention to it because we're talking. But yeah, this is very cool. Just maybe a final question there. Is there a GBT that you saw someone built that was like, wow, that's amazing. That's so cool. Something that surprised you and I'll share all sure one that was really cool. But is there anything that comes to mind? Masked it. I think my my instinct is the Zapier. All of the stuff that Zapier has done with GBT's is like the most useful stuff that you can imagine. Like you can go so far with what in I don't know how it's like packaged for Zapier's GBT right now.

29:49You can actually as a third party developer integrate Zapier without knowing how to code into your GBT. So they're pushing a lot of this stuff and then basically like all 5 ,000 connections that are possible with Zapier today. You can bring into your GBT and essentially enable it to do anything. So I'm incredibly excited for Zapier and for people who are building with them because there's so many things that you can unlock using that platform. So I think that's probably the most exciting thing to me for people who aren't developers. Awesome. Zapier's always in there getting their connecting things.

30:23Yeah, they're great. So the one that I had in mind, so I had a buddy, my in Seiki who's the CEO of a company called Runway built this thing called Universal Primer, which helps you learn. It's described as learn everything about anything. And basically I think is kind of this socratic method of helping you learn stuff. So it's like explain how transformers work in LLM's. And then it just kind of goes through stuff and then asks you questions, I think, and kind of helps you learn new concepts. And I think it's the number two education GPG. I love that. Seiki's incredible. So yeah, that's true. Let me tell you about a product called Arcade.

30:58Arcade is an interactive demo platform that enables teams to create polished on brand demos in minutes. Telling the story of your product is hard. And customers want you to show them your product. Not just talk about it or gait it. That's why product four teams such as Atlassian, Carda, and Retool use Arcade to tell better stories within their home pages, product change logs, emails, and documentation. But don't just take my word for it. Quantum metric, the leading digital and analytics platform created an interactive product to our library to drive more prospects. With Arcade they achieved a 2x higher conversion rate for demos and saw five times more engagement than videos.

31:38On top of that they built a demo 10 times faster than before. Creating a product demo has never been easier. With browser -based recording, Arcade is the no -code solution for building personalized demos at scale. Arcade offers product customization options, designer -approved editing tools, and rich insights about how your viewers engage. Every step of the way, ready to tell more engaging product stories that drive results? Head to Arcade .Soph2Ware. Slash Lenny and get 50 % off your first three months. That's Arcade .Soph2Ware. Slash Lenny. I want to talk about just what it's like to work at OpenAI and how the product team operates and how the company operates.

32:16You worked at your two previous companies where Apple and NASA which are not known for moving fast. OpenAI is known for moving very fast, maybe too fast for some people's taste as we saw with the whole board thing. What I'm curious is just what is it that OpenAI does so well that allows them to build and ship so quickly and it's such high a bar? Is there a process or a way of working that you've seen that other companies should try to move more quickly and ship better stuff? There's so many interesting trade -offs in all of this tension around how quickly companies can move. I think for us, if you think about Apple as an example, if you think about NASA as an example, just older institutions, lots of over time, the tendency is things slow down.

33:04There's additional checks and balances that have put in place which drag things down a little bit. We're young and a new company. We don't have a lot of that institutional legacy barriers that have been put in place. I think the biggest thing, and there's a good Sam Tree somewhere in the ether about this from, I think, 2022 or something like that. Finding people who are high agency and work with urgency is one of the most, if I was hiring five people today, those are some of the top two characteristics that I would look for in people. Because you can take on the world if you have people who have high agency.

33:46And not meeting to either get 50 people's different consensus because you have people who you trust with high agency and they can just go and do the thing. I think it's one of the most important thing. I'm pretty sure if you were to distill it down. I see this in folks that I work with. Folks are so high agency. They see a problem and they go and tackle it. They hear something from our customers about a challenge that they're having and they're already pushing out what the solution for them is and not waiting for all the other things to happen. That I think traditional companies are stuck behind.

34:23Because they're like, let's check with all these seven different departments. Try to get feedback on this. People just go and do it and solve the problem. I love that. It's self -fung to be able to be a part of those situations. That is so cool. I really like these two characteristics. Because I haven't heard this before. Maybe the two most important things you guys look for. High agency, high urgency. To give people a clear sense of what these actually look like when you're hiring, you shared maybe this example of customer service. Someone's hearing some bug and then going to fix it. Is there anything else that can illustrate what that looks like?

34:55High agency and then some more question on urgency other than just like move, move, move, move, move, move, move. I think the assistance API that we released for DevDay. We continue to get this feedback from developers that people wanted these higher levels of abstraction on top of our existing APIs. A bunch of folks on the team just came together and were like, let's put together what the plan would look like to build something like this. And then very quickly came together and actually built the actual API that now powers so many people's assistant applications that are out there. And I think that's a great example of like, it wasn't like this top down, someone's sitting there being like, let's do these five things and then like, okay, team go and do that.

35:38It's like people really seeing these problems that are coming up and knowing that they can come together as a team and like solve these problems really quickly. And I think the assistance API and there's like a thousand or one other examples of teams taking agency and doing this. But I think that's a great one at the top of my head. That makes me want to ask just how how does planning work at OpenAI? So in this example, it was just like, hey, we think we need to build this. Let's just go and build it. Imagine there was still a roadmap and priorities and goals and things that that team had. How does road mapping and prioritization and all of that generally work to allow for something like that?

36:13I think this is one of the more challenging experiences that we've been experiencing. He says at OpenAI, there's so many, like everyone wants everything from us. And like today, especially in the world of chat GBT and how large and and well used are APIs, people will just come to us and say, hey, we want all of these things. I think there's like a bunch of like core guiding principles that we look at. Like one, going back to the mission, like, is this actually like going to help us get to AGI? So there's a huge focus on like, you know, there's this like potential shiny reward right in front of us, which is like, you know, like optimize user engagement or whatever it is.

36:51And like, is that really the thing? Like maybe the answer is yes, like maybe that is what is going to help us get to AGI sooner, but like looking at it through that lens, I think is like always the first step of deciding any of these problems. I think on the developer side, there's also these like core tenets of like reliability. Like, hey, you know, it would be awesome if we had additional APIs that did all these cool things like new new endpoints, new modalities, new abstractions. Are we giving customers a robust and reliable experience or API? And like that's often like the first question. And I think there have been times where we fall in short on that.

37:25And like, you know, there was a bunch of other things that we've been thinking about doing. And like really bringing the focus and priority back to that reliability piece. Because at the end of the day, nobody cares if you have something great. If they can't use it robust and reliably. So there's like these core tenets. And I think like again, we have like very other than all the principles about how we're making the decision. I think like the actual planning process is like pretty standard. Like we come together. There's like H1, Q1 goals. We all sprint on those. I think the real interesting thing is like how stuff changes over time.

37:58Like you think we're going to do these like very high level things. Like, you know, new models, new modalities, whatever it is. And then like as time goes on, there's like all of this turmoil and change. And it's interesting to have like mechanisms to be like, hey, how do we how do we update our understanding of the world that our goals as everything sort of the ground changes underneath of us as is happening in the craziness of the AI space today? It's interesting that it sounds a lot like most other companies. There's H1 planning. There's Q1 planning. Are there metrics and goals like that that you guys have?

38:29Okay, ours or anything like that? Or is it just here? We're going to launch these products. I think it's like much higher level. Well, I actually don't think open AI is like a big okay or company. I don't think teams do okay. I don't have a good understanding of why that's the case. Whether or not I don't even know if okay. Are there like still the industry? You're probably talking to a lot more folks about like, yeah, who are making those decisions? So I'm curious is that something that you're seeing for folks like is it still common for people to do? Okay, ours. Yeah, absolutely. Many companies use their cares love of the carers.

38:57Many companies hate okay. I'm not surprised that open AI is not an okay or driven company. Alongs lines, I don't know much you can share about all this stuff, but how do you measure success for things that you launch? I know there's this ultimate goal, AGI. Is there some way to track? We're getting closer. What else do you guys look at when you launch? Say, DPT store or assistance or anything. That's like, that was exactly what we're hoping for. Is it just adoption? Yeah, adoption is a great one. I think there's like a bunch of metrics around like, you know, revenue, number of developers that are building on our platform, all those things and a lot of these and I don't want to to dive.

39:30I'll let Sam or someone else on our leadership team, like go more into the details. But I think like a lot of these are like actual abstractions towards something else. Like even if revenue is a goal, it's like revenue is not actually the goal. Revenue is a proxy for getting more compute, which is then like actually what helps us get towards getting more GPUs so that we can, you know, train better models and like actually get to the goal. So there's all these like intermediate layers, where like even if we say something is the goal and like, you hear that in a vacuum and you're like, oh, okay, I just want to make money.

40:02And it's like, well, really money is the mechanism to get better models so that we can achieve our mission. And I think there's a bunch of interesting, interesting angles like that as well. I don't know if I've heard of a more ambitious vision for a company to build artificial general intelligence. I love that. I imagine many companies are like, what's our version of that? Before we leave this topic, is there, is there anything else that you've seen open ID really well? That allows it to move this fast and be this successful. You talked about hiring people with higher agency and high urgency. Is there anything else that's just like, oh, that's a really good way of operating?

40:39Imagine part of it's just hiring incredibly smart people. Like I think that's probably an unsaid thing, but yeah, anything else. I think there's a non -trivial benefit to using Slack. And I think like maybe that's controversial and maybe some people don't like Slack, but opening up such a Slack heavy culture and like it really the like instantaneous real -time communication on Slack is so crucial. And like I just love being able to like, agon different people from different teams and like get everybody coalesced. So like everybody is always on Slack. So it's like even if you're remote or you're on a different team or in a different office, like still much of the company culture is like ingrained in Slack.

41:16And it allows us to like really quickly coordinate where like, it's actually faster to send them to Slack, but sometimes then it would be to like walk over to their desk because they're on Slack and they're going to be using it. And I saw if you saw the recent Sam and Bill Gates interview, but Sam was talking about how Slack is his number one most used app on his phone. I'm like, I don't even look at the time saying, I'm like, okay, it works. And like I don't want to know how long I'm using Slack, but I'm sure the saleswrestly people are looking at the numbers and they're like, just exactly what we've wanted.

41:45So I also love Slack. I have a big promoter of Slack. I think there's a lot of Slack, but it's such a good product. I've tried so many alternatives and nothing compares. I think what's interesting about Slack for you guys is one of the, like you don't know if someone in there is just an AGI. That is not actually a person that's just there working in the company. I know there are real people. There is no AGI's yet, but I think like yeah, even Slack is building a bunch of like really cool AI tools, which like I'm excited to. And that's why like there's so much cool AI progress. And like at the end of the day, it's so exciting from being like a consumer of all these new AI products.

42:20Like Google's a great example. Like I'm so happy that Google's doing really cool AI stuff because like I'm a Google docs customer. And like I love using Google docs and like a bunch of their other products. And like it's awesome that people are building such such useful things around these models. How big is the opening I team at this point, whatever you can share, just to give people a sense of the scale? Yeah, I think the last public number was something around like 750 near the near the end of last year, 780 or something like that near the end of last year. And we're growing, we're still growing so quickly.

42:49So I don't want to, I won't be the messenger to share. The specific update never's like the team is growing like crazy. And we're also hiring like across all of our engineering teams. So folks are NPM teams. So folks are interested. We'll love to hear from folks who are who are curious about joining. Maybe one last question here. So you're growing maybe getting to 1000 people. Clearly still very innovative and moving incredibly fast. Is there anything you've seen about what opening I does well to enable innovation and not kind of slow down new big ideas? Yeah, there's a couple of things. One of which is the actual research team who like, you know, sort of cede most of the innovation that happens at OpenAI is intentionally small.

43:29They're not like, you know, most of the growth that open AI and seeing is around like our customer facing roles, our engineering roles to like provide the infrastructure to protect the team and things like that. The research scene is like again, intentionally kept small and there's all of this talking. It's really interesting. I just saw this thread from one of our, one of our research folks who was talking about in a world where you're constrained by the amount of GPU capacity that you have as a researcher, which is the case for OpenAI researchers. And it also researchers everywhere else. Like each new researcher that you add is actually like a net productivity loss for the research group.

44:05Unless that person is like up leveling everyone else in like such a profound way that like it increases the efficiency. Like if you just add somebody who's going to go and like tackle some completely different research direction, you now have to share your GPUs with that person and everyone else is now slower on their experiments. So they're really interesting like trade off that the that research folks have that I don't think like product folks like I add another engineer to like our API team or to our some of the chat GPT teams like you can actually write more code and do more and like that's actually like a net beneficial improvement for everybody.

44:40And that's always not the case in the case of researchers, which is interesting. In a GPU constraint world, which hopefully we won't always be in. I want to zoom out a bit and then there's going to be a couple follow -up questions here. Where are things heading with OpenAI? What's kind of in the near future of what people should expect from the tools that you guys are going to have in lunch? Yeah, new new modalities. I think chat GPT like continuing to push all of the different experiences that are going to be possible. Like today like chat GPT have really just like text in text out or I guess like three months ago it was just text in text out.

45:13We started to change that with now you can do the voice mode and now you can generate images and now you can take pictures. So I think like continuing to expand like the way in which you interface with AI through chat GPT is coming. I think GPT's is our first step towards the agent future. Like again, today when you use a GPT it's really you send the message, you get an answer back almost almost right away. And that's kind of the end of your interaction. I think as GPT's continued to get more robust, like Gil actually built a say, hey, go and do this thing and like just let me know when you're done.

45:43Like it might take, I don't need the answer right now. I want you to like really spend time and be thoughtful about this. And like again, that's if you think back to all these human analogies, like that's what we do with humans. Like I don't expect somebody when I ask them to do something meaningful for me to like do it right away and like give me the answer back right away. So I think pushing more towards those experiences is what is going to unlock like so much more value for people. And I think the last thing is GPT's as this mechanism to get like the next you know, few hundred million people into chat GPT and into AI.

46:16So I think like if you have conversations with people who aren't close to the AI space, oftentimes you talk about even if they've heard of chat GPT, you'll have to have heard of chat GPT, but if they have, they're like they show up in chat GPT and they're like, you know, I don't really know what I'm supposed to do with this. There's blank slate. I can kind of do anything. It's like not super clear how this solves like my specific problem. And I think the cool thing about GPT is you can package down like here's this one very specific problem that AI can solve for you and and do it really well. I'm like I can share that experience with you and now you can go and try that GPT.

46:47Have it actually solve the problem and be like wow, like it did this thing for me. I should probably spend the time to investigate like these five other problems that I have to see if AI can also be a solution to those. So I think so many more people are going to come our line and start using these tools because very like narrow, vertical tools are what's going to be like a huge amount for them. So in the last case, a classic horizontal product problem where does so many things and people don't know what exactly it should do for them. So that makes a ton of sense. Just being a lot more template oriented use case specific helping people on board makes a tons of sense.

47:22A common problem for so many sales products out there. The other ones you mentioned, which is really interesting. Basically, more interfaces to more easily interact with opening i voice, you mentioned audio and things like that. That makes tons of sense. And then this agents piece where the idea is instead of just a chat, it's like a good to do this thing for me. Kind of along those lines, GPT -5, we touched on this a bit. There's a lot of speculation about the much better version. People just have these wild expectations, I think, for where GPT is going. GPT -5 is going to solve all the world's problems.

47:55I know you're not going to tell me when it's launching and what it's going to do. But I heard from a friend that there's kind of this tip that when you're building products today, you should build towards a GPT -5 future, not based on limitations of GPT -4 today. So to help people do that, what should people think about that might be better in a world of GPT -5? Is it just like, it's faster, it's just smarter, is there anything else that might be like, oh, I should really think I'm approaching my product? If folks have looked through the GPT -4 technical report that we released back in March when GPT -4 came out.

48:28GPT -4 was the first model that we trained where we could reliably predict the capabilities of that model beforehand based on the amount of compute that we were going to put into it. And you could actually, we did like a scientific study to show like, hey, this is what we predicted and here is what the actual outcome was. So it'll be one, I think, just as somebody who's interested in technology, but interested in it, see like does that continue to hold for GPT -5 and hopefully we'll share some of that information when whenever that model comes out. I also think you can probably draw a few observations.

49:00One of them, which is GPT -4 came out, the consensus from the world is everything is different. Like, all of a sudden, everything is different. This changes the world. This changes everything. And then slowly but surely, we come back to reality of like, this is a really effective tool and it's going to help solve my problems more effectively. And I think that is like the undoubtedly the lens in which people should look at all of these model advancements. Like GPT -5 is like surely going to be extremely useful and like solve some whole new echelon of problems. Hopefully they'll be faster. Hopefully they'll be better on all these ways.

49:35But like fundamentally, the same problem that exists in the world are still going to be the same problems. You now just have a better tool to solve those problems. And I think like going back to like vertical use cases, like I think people who are solving very specific use cases are just now going to be able to do that much more effectively. Like I don't think that's like going to people have these unrealistic expectations that like GPT -5 is going to be like doing back flips in the background in my bedroom while it also like writes on my code for me and like talked in the phone with my with my mom or something like that.

50:06I'm like, that's not the case. Like it is just going to be this like very effective tool, very similar to GPT -4. And it's also going to become like very normal very quickly. And I think like that is actually a really interesting piece. If you can plan for the world where people become very very used to these tools very quickly. I actually think that's like an edge and like assuming that this thing is going to like absolutely change everything. And in many ways, I think it's actually like a downside is like the wrong mental framing to have of these tools as they come out. Kind of along these lines, you guys are investing a lot into B2B offerings.

50:42I think half the revenue last I heard was B2B and then half is B2C. I don't know if that's true, but that's some I heard. What is it that you get if you work with OpenAI as a company as a business? What is the what is it? What is it? What is it? What's it? Gold? And what do you get as a part of that? Yeah. So I think a lot of our B2B customers are using the API to like build stuff. So I think that's one angle of it. I think if you're a Chatchy BTB customer, we sell teams, which is the ability to like get multiple subscriptions of Chatchy BT packages together. We also have an enterprise version of Chatchy BT.

51:18There's a bunch of like enterprise -y things that enterprise companies want around like SSO and stuff like that related to Chatchy BTB enterprise. I think the coolest thing is actually being up in the like share some of these like prompt templates and GBT's internally. So again, you can make like custom things that work really well for your company with like all of the information that's relevant to solving problems that your company and like share those internally. And to me, that's like, you know, you want to be able to collaborate with your teammates other cool things you create using AI. So that's a huge unlock for companies.

51:48I think that those are like the two biggest value ads. There's like higher limits and stuff like that on some of those models. But I think being able to share like your very domains to set up applications is the most useful thing. And I think if you're a company listening and you think a lot of employees are using Chatchy BTB basically the simplest thing you could do is just roll it up into a business account with single sign -on that probably saves you money and makes it easier to coordinate and administer. Yeah, there's also like a bunch of security stuff too. Like if you want to control like you don't want people to use certain GBT's from the GBT store because you're like worried about security or privacy and stuff like that, you don't want your private data going in places.

52:25It makes a lot of sense to sign up for that so that you have a little bit more control over what's happening. Okay, got it. There's a launch happening tomorrow. I think after recording this, can you talk about what is new? What's coming out? I think this is going to come out a couple weeks after recording. But just what should people know that's new that's coming out from opening? I am tomorrow in our time in our role. Yeah, updated. So there's a few different things. A couple of quick ones are updated. GBT for turbo model, update the preview model that we released at dead day. There's an updated version of that.

52:55It fixes this. If folks have seen online, people talking about this sort of laziness phenomenon in the model, we improve on that and it fixes a lot of the cases where that was the case over the model. Be a little bit less lazy. The big thing is the third generation embedding model. So we were talking off camera before recording about all the cool use cases from that. So it's folks have used embeddings before. It's essentially the the technology that powers like many of these like question and answering with your own documentation or your own corpus of knowledge and lay you were saying you actually have a website where people can ask questions about recordings of the podcast.

53:33Lennybot .com check it out. Yeah, Lennybot .com. My assumption was that Lennybot .com is actually powered by embedding. So you take all of the corpus of knowledge. You take all the recordings, your blog posts, you embed them. And then when people ask questions, you can actually go in and see the similarity between the question and the corpus of knowledge and then provide an answer to somebody's question and reference like in empirical fact like something that's true from your knowledge base. I'm like, this is super useful and people are doing a ton of this is like trying to ground these models in reality in what they know to be sure like we know all the things from your podcast to be at least something that you've said before and to be true in that sense.

54:12And we can bring them into the answer that the model is actually generating in response to a question. So that'll be super cool. And these new B3 embedding models again, you know, state of the art performance. The cool thing is actually the non -English performance has increased super significantly. I think historically people really were only using embeddings for like it only worked really well for English. And I think now you can you can use it across like so many new languages because it's just so much more performant across those across those languages. And it's like five times cheaper as well, which is wonderful.

54:47I there's there's no better feeling than making things cheaper for people. I love it. I think now it's like you can embed I'm pretty sure it was like 62 ,000 pages of text for one dollar, which is which is very, very cheap. So lots of really cool things you can do with embeddings and excited to see people embed more stuff. What a deal. Final question before we get to our very exciting lightning round. Say you're a product manager at a big company or even a founder. What do you think are the biggest opportunities for them to leverage the tech that you guys are building? GPT -4, all the other APIs.

55:26How should people be thinking about here's how we should really think about leveraging this power in our existing product or new product whichever direction you want to go? Yeah, I think going back to this theme of like new experiences is really exciting to you. I think consumers are just going to be like like you're going to have an edge on other people if you're providing AI that's not accessible in a chatbot. Like people are using a ton of chat and like it's a really valuable service area. Like it's clearly valuable because people are using it. But I think products that like move beyond this chat interface really are going to have such advantage and also like thinking about how to take your use case to the next level.

56:07Like I've tried a ton of chat examples that are like very very basic and like providing a little bit of value to me. But I'm like really this should go like much further and like actually build your core experience from the ground up. Like I've used this product that allows you to essentially like manage or like view the conversations that are happening online around like certain topics and stuff like that. So I can go and look online like what are people saying about GPT -4? And like that what I just said out loud what are people saying about GPT -4 is like the actual question that I have and like in an normal product experience and they like I have to go into a bunch of dashboards and like change a bunch of filters and stuff like that.

56:45And what I really want is just like ask my question. What are what are people doing? What are people saying about GPT -4? Like get an answer to that question in like a very data grounded way. And I've seen people like fall part of this problem where like let me go here's like here's a few examples of what people are saying and like well that's not really what I want. Like I want this like summary of what's happening and I think it just takes a little bit more engineering effort to make that happen. But I think it's like that is the magical unlock of like wow this is an incredible product that I'm going to continue to use instead of like yeah this is kind of useful but like I really want more.

57:21Awesome I'll give a shout out to a product I'm not an investor but I know the founder called visualelectric .com which I think is doing exactly this it's basically a tool specifically built for creatives I think specifically graphic design to help them create imagery. So you know there's like Dolly obviously but this takes it to a whole new level where it's kind of this canvas infinite canvas that you can just generate images edit tweak them and continue to do it right until you have the thing that you need visual. Is it similar to Canva? It's it's even more niche I think for more sophisticated graphic design I think is the use case but I'm not a designer so I'm not I'm not the target customer but I will say my wife is a graphic designer she'd never use AI tools I showed her this and she got hooked on it she paid for it without even telling me that she's going to become a paid customer and she just started she created imagery or dog all in all this art and now it's like on our TV she the art she created is now sitting it's like we have a frame TV and that's the image on our TV so anyway I love that what was it called again visualelectric .com anyway anything else you wanted to touch on or share before we get to a very exciting lightning round I've made this statement a few times on my other places but like for people who are have cool ideas that they should build with AI like this is the moment like there are so many cool things that need to be built for the world using AI and like again if if I or other folks on the team at UrbanHack can be helpful in like getting you over the hump of like starting that journey of building something really cool like please reach out like there's just the the world needs more cool solutions using these tools and would love to hear about like the awesome stuff that people are building.

59:01I would have asked you this at the end but how would people reach out what's the best way to actually do that? Twitter linked in my email should be signable somewhere I don't want to say it and I can make it stand with a bunch of emails like you should be able to sign my email if you needed online somewhere but yeah Twitter and LinkedIn is usually like the easiest place and how do they find your Twitter? It's just Logan Tobattricker I think my name shows up as Logan .gpt or LoganGpt. official LoganK. Yeah awesome okay and we'll link to it in the show notes amazing well Logan with that we reached a very exciting lightning round are you ready?

59:34I'm ready. First question what are two or three books that you recommended most to other people? I think the first one that I read a long time ago and came back to recently is the Wonderham Schoolhouse by Salcon incredible yeah I don't want to lie to you around so I won't say too much like incredible story and AI is what is going to enable Salcon's vision of like a teacher per student to actually happen so I'm really excited about that and the other one is that I always come back to is why we sleep. I yeah sleep sleep and sleep science are so cool if you don't care about your sleep like it's one of the the biggest up levels that you can do for yourself.

1:00:13What is a favorite recent movie or TV show that you really enjoyed? I'm a sucker for like a good inspirational human story so I watched with my family recently over the holidays this grand tourism movie and it's a story about somebody who like this kid from London who grew up like doing like sin racing which is like a virtual race car and did this competition and it up becoming like a real professional race car driver through some competition and it's just like really cool to see yeah someone go from driving a virtual car to driving a real car and like competing in a 24 hour limon all that stuff.

1:00:50I used to play that game and it was a lot of fun but yeah I think if any clue how to drive a real car race car so that's inspiring. Do you have a favorite interview question they'd like to ask candidates that you're interviewing? Yeah I'm always curious to hear what people's like the thing that they so strongly believe that people disagree with them on. What do you look for in an answer that seems like wow that's a really good signal? I'm often times it's it's just an entertaining question to ask in some sense but it's also it's interesting to see like what somebody's like deeply held strong belief is I think that's it and you know not not to like judge whether or not I believe and not to like just curious to like see why why people feel that way.

1:01:34What is a favorite product that you recently discovered that you really like a thunder narrative of sleep I have this I have this really nice sleep mask from this company called and the not being paid I just say this but it's called like meant meant to sleep or something like that it's a weighted sleep mask and it feels incredible when I I don't know maybe I just have a heavy head or something like that but it feels it feels good to wear as a weighted sleep mask and I I really appreciate it. I have a competing sleep mask that I highly recommend I'm trying to find it it's an email people about it a couple times in my newsletter for gift cards okay my favorite is called the wow wow sleep mask w a what do you like about it?

1:02:15w a o a w a o a w a link to it in the show it makes a lot of room it's like very large and in their space for your eyes so like your eyelashes and whatever eyes aren't pressed on and it's just it just fits really nicely around the head in my wife we both wear our masks at night just speaking of sleep really helps the sleep that's not like I love it yeah it doesn't have the weight in this piece so it might be worth trying but everyone I recommend of this to is like that changed my life thank you for helping me sleep better and so we'll link to book a limit sleep mask look at us that's the adult uh two more questions do you have a favorite life motto that you often come back to share with friends or family either in work or in life yeah I've got it it's on my post -it note that I right behind my camera and it's measuring hundreds I love this idea of measuring things in hundreds and it's for folks who are like at the beginning of some journey I I talk to people all the time they're like yeah I've tried this thing and it hasn't worked and if your mental model is to measure in hundreds I measure in hundreds the five times that you failed at something you failed and tried zero times and I'd love that it's like such a great reminder that everything in life is like built on compounding and multiple attempts at stuff and if you don't try it that's times like you're never going to be successful at it I love that I could see why you're successful at OpenAI and why you're a good fit there final question so I asked open I asked chat GPT for a very silly question give me a bunch of silly questions to ask Logan Kilpatrick head of developer relations at OpenAI and I went through a bunch I have three here but I'm gonna pick one if an AI started doing stand -up comedy what do you think would be its go -to joke or funny observation about humans I think today I think it you if you were to do this like I think the go -to question would be something along like the so -and -a -i walks into a bar and likely because again it's trained on some distribution of training data and like that's like the most common joke that comes up and that's probably like I'm wondering if you came up with a joke right now whether or not that would shell up in in one of the examples I love it what would be the joke though we need the joke we need the punchline I'm just joking I know you can't come up with amazing that's what we Logan thank you so much for being here two vinyl questions even though you've already shared this information but just for folks to remind them work in folks find you if they want to reach out and ask you more questions and how can listeners be useful to you yeah Twitter and late and Logan Kilpatrick or Logan .gbt on Twitter please please shoot me messages like I get a ton of dm some people it's always like really really interesting stuff I think the thing that I can that I would love to have help on is like if people find bugs and things that don't work well and chat you be to you like I oftentimes like see people be like this thing didn't work really well and the the key and I think we as open a addent you a better job of like messaging this to people but having like shared chats or like actual like tangible reproducible examples are like the two things that we need in order to like actually fix the problems that people have like the model laziness was a good example where it was kind of hard to figure out what was going on because people would be like oh the model is lazier but like it's hard to figure out like what were the prompts they were using what was the examples all that stuff so send those examples as you come out on things that don't work well and we'll we'll make stuff better for you.

1:05:49Amazing and I'll also just remind people as you're if you're listening to this and you're like oh okay cool a lot of cool ideas for open AI and chat gpt what you need to do is actually just go to chat chat the chat that open AI dot common try the stuff out there's a lot of just like theorizing but I think once you actually start doing it you start to see things little differently and at this point every day I'm in there's doing something like asking for ideas for questions doing research on a newsletter post and it's just like a tab I'm always coming back to and I know there's a lot of people just like talking about this sort of thing and I just want to remind people just like go sign in play with it ask a question then something working on and just see how goes and keep keep coming back to is there anything else you want to share on those lines inspire people to give this a shot.

1:06:34I love it I think that the phrase of like you know people being worried about humans being replaced by AI and I've seen this narrative online that it's like it's not AI that's going to replace humans it's like other humans that are being augmented and like using AI tools that are like going to be more competitive than a job market and stuff like that so go and try these AI tools like this is the best time to learn like you're going to be more productive and like empowered in your job and the things that you're excited about so yeah excited to see what people will use Chatchuby T4 and then you could expense your account I think it's 10 or 20 bucks a month a lot of companies are paying for this for you so ask your boss if you can just have it expense and make sure you use the latest version anyway Logan thank you again so much for being here this is awesome buddy makes for me and thoughtful questions hopefully those weren't all from Chatchuby T.

1:07:20No only the last one I did have a bunch of others I was I had in the in the belt or in the pocket I don't know if the matter was in the back pocket that's the matter for but I did not get to them because we had enough great stuff so now that was all me human yeah thank you thanks Logan Lenny dot A I love it Lenny bot dot com check it out okay thanks Logan bye everyone thank you so much for listening if you found this valuable you can subscribe to the show on Apple podcasts Spotify or your favorite podcast app also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast you can find all past episodes or learn more about the show at Lenny's podcast dot com see you in the next episode