Open AI leadership shuffle, new diffusion models, and starting the cult of Q*

0:05Hi, NoPriors listeners. Time for a host-only episode. This week, Ilad and I are going to talk about what's going on at OpenAI, of course, video, QSTAR, what might be next in research, and some predictions. Okay, Elad, we have to start with the saga from this past week. What is your take on the outcome and the second order effects? From a second order effect perspective, this seems like overall positive news for everybody involved. So it looks like on the OpenAI side, they're back to being in a really positive, stable situation. I think they still have like the leading model in GPT-4. They've reworked the board, which seems like a positive thing.

0:45So, you know, imagine if this had happened two years from now, three years from now, et cetera. So it seems like it would net increase the stability of the company and governance and a few things like that, or the nonprofit and the company and governance. So as an external viewer, it seemed like a painful thing to go through. But the flip side of it is, it seems like they're moving forward and moving ahead in a positive way. and then in parallel I think it may have ramifications to other areas that we can talk about if useful like what are the second order aspects of this but be great to hear what you think yeah I think the first obvious lesson is that governance matters right and this isn't an area where I think most companies are that experimental but I think a lot of entrepreneurs are likely to think twice about placing their destiny in the hands of groups with explicitly mixed incentives now.

1:35I'd say generally nonprofit governance is not every organization, but as a class known to be kind of abysmal, right? Because performance is hard to measure objectively. And so it often ends up more about politics and specific relationships and status games than outcomes. The clarity around how much, you know, any board matters was kind of a wake up call for people. The second lesson that a lot of people will take away or I think should from this whole saga is that money matters, right? The factors of production are labor and capital and compute is the AI specific form of capital. Microsoft holds the compute here and that clearly matters.

2:15This is amazingly well managed and supported by Sai and Kevin. And then the class of like really special labor here, the team rebelled and the board obviously underestimated the level of support that Sam and Greg had from them. And then I think one thing that is often unsaid because it's a little bit less idealistic is that a lot of OpenAI people were very upset this last week about not just the destruction of the mission, which I think was absolutely like genuine, but also destruction of the value they built. And been promised a piece of with the$86 billion tender offer. It's just a reminder that labor and capital are like leveraged.

2:58They're stakeholders. And there's no free lunch or control without skin in the game. And I think there likely shouldn't be from the view of many of the people involved in this. Yeah, I think you're raising an important meta point, which is basically what are the incentives that different organizations have in place? and ignoring OpenAI, I think there's a lot of boards which added board members for reasons that were politically motivated or motivated by different regulations getting passed that forced certain board changes, et cetera. And you also see that in executive teams. And I think it's really important for people to go back and rethink, okay, who should be on my board and why?

3:33What are they representing relative to the board? What expertise do they provide? Or what insights are they bringing? Or what strategic views are they bringing? Same with your executive team. And also, what are their incentives? and there's this view that's kind of been moving around Silicon Valley in terms of the professional managerial class, right? People who have alliance not to the organizations they work at, but external incentives. And those external incentives could be speaking at TED or going to Davos or getting kudos or an award from a specific organization versus doing what's actually their duty as a representative of the various shareholders of a company in the context of a corporation.

4:14And so there are these fiduciary duties that may be being breached by other incentives for different actors who've been added over the last five, 10 years to boards, to executive teams, et cetera. And I think it's really worth rethinking, who do I want on board and why? And it also comes back to some of the companies that have been resetting things relative to politics. I think Shopify did a great job, for example, of saying, we're performance-based culture, We're focused on, you know, a very specific mission. We don't want that mission to creep. We're about, we're not a family, you know, like if your uncle shows up drunk and does something bad, you forgive him.

4:51If a board member shows up and does that, then, you know, you don't want them on your board, right? They're being irresponsible. There's also that broader context of like, how do you want to think about alignment, incentives, culture, motivations? And, you know, is this a good moment in time to sort of pause and rethink some of those things relative to your own company. Yeah. One friend at OpenAI who, I guess, publicly declared that this reignited their belief in clear incentives and good intent in capitalist structures, that has actually seemed somewhat radical in many Silicon Valley companies over the last few years.

5:29But I think that is going to get rethought when you see what happens when there are unclear or misaligned incentives. Yeah, there's two actually really good quotes to that. There's one, which is something which I'm going to get wrong, which is something along the lines of like, capitalism is the best way to take care of people that you don't know. You know, it's the means of actually growing the pie in many cases and providing for others through the sort of incentive of markets. But the other one is a Charlie Mungerism. And unfortunately, Charlie Munger passed away earlier today. And obviously, he was sort of a giant of industry.

5:56And he had this great saying, which is, anytime I think I understand the importance of incentives, I realize that I'm underestimating the importance of incentives. If we just think about what the other second order, like more commercial effects are, I do think that there is an interest in owning models more in open source models and in at least understanding like reliance on a single vendor. What do you think here? Yeah, I think there's a couple people who have built solutions during the last week that, for example, Braintrust now has a AI proxy where you can use the OpenAI SDK to effectively query multiple different models, including Mistral and LAMA through perplexity, as well as a variety of other things.

6:42GPT-4, GPT-3.5, I think potentially Anthropic. And so it just allows you to be able to both load balance your queries or prompts, but also interchange models more easily so you can actually look at performance across them. I think Chima similarly has done something over the last week that they've released that helps with some of the proxying and other things. And so I think there's solutions like that that have started to be accelerated in a market that would have happened inevitably. right i think everybody the journey that i see people often take is they'll prototype on gpt4 they'll look at how good it is and then they'll either keep it on gpt4 particularly if they need like advanced channel logic or other things or if they need very high throughput and performance and low cost then sometimes they'll either switch to gpt 3.5 or they'll see if they can fine-tune something right and that's the only people with enormous scale like i don't see very many fine tunes happening in general unless you know somebody has enormous scale and or proprietary data.

7:36They just don't want to get out, right? So they'll fine tune Mr. Lama or something. So already, I think people were thinking about that. And then that means you need to build an orchestration layer, you may need the proxy, you may need a variety of things. So the dimensions that people were evaluating an LLM provider on, or whether or not they wanted to control or host or fine-tuned themselves just became more clear, right? Where like reliability became more important, but the reliability, latency, cost control, capability questions were sort of naturally there. And to be clear, like OpenAI leads on capability in many areas, in unique capability in some, right?

8:22Like code generation, GPT-4V, right? You can do amazing things with that and people should go build on those tools. I think the ecosystem will mature and OpenAI is a great partner. But I think the questions are just much more obvious for anybody relying on these models now. Yeah, and I think honestly, a lot of the bigger enterprises I knew always wanted to make sure that if they really needed to, that they could second source something. So I don't think this is a new thing. In other words, one could argue that no matter what OpenAI does, there'll always be at least one or two other suppliers or vendors or partners for advanced LLM simply because the market always wants an alternative, even just for negotiation leverage.

9:03And so if you look at other markets, for example, in the router world, one of the main reasons Juniper exists is because everybody wants to make sure that they can push on Cisco for pricing. And so they always want to have a second source. That's why Juniper is always 10 to 20 percent the size of Cisco, right? It's just second sourcing or AMD versus Intel for a very long period of time. So I think often markets will end up with other players just because big enterprises always want to have that option if they need it, even if it isn't as good. And if anything, I think OpenAI kind of emerges more stable through this in ways that people didn't expect simply because there's going to be more stability at the board level in a way that people didn't understand perhaps that there could have been instability, right?

9:45This is a strengthening event and a focusing event for the company, at least from what I can tell external. Do you want to talk about Pika and video? Yeah. Yeah. There are a couple of really amazing launches happening in the video space. What's the cause for this? Like we suddenly have text to video generation and avatar cloning in different ways. What do you think is going to happen in this space? Interesting shift has been happening because basically if you go back a year and a half, MidJourney launched, Staple Diffusion came out, Dolly 2 came out. And there's a whole wave of people saying that they were going to go build on diffusion models.

10:23And ImageGen was like the thing that everybody was going to go do for like two months. And then ChatGPT came out and then everybody was like, oh my God, I need to go work on LLMs and language and natural language and NLP and all this stuff. And so the entrepreneurial ecosystem went through this sort of zigzag where everybody was going to do ImageGen and a bunch of companies started going down that direction. And then the LLM stuff really kind of was substantiated through ChatGPT. And then most people went that way. And a handful of founders stuck around on the diffusion model side. And diffusion models, you know, are really popping up.

10:55And obviously there's like Image Transformer and a bunch of other stuff, but they're mainly being used for image gen, for video and for audio, actually. And so there's a wave of people who've continued to work and crank on this. And they're starting to come out with really interesting products. For example, Pika is a great example where it was two Stanford PhD students who'd been working on diffusion models for some time. and they made this really amazing creative text-to-video engine. There are other companies like Ahajan or Synesthesia or others that are doing, let me use these diffusion models to clone an avatar or to generate an avatar of a person so that they can either go into the metaverse al-azak or alternatively, they can use it for marketing purposes.

11:35They can use it for internal training. They can use it for all sorts of applications. And then there's some really cool audio-based things coming out too, which I think are starting off more tools to create music or just simulate voice in the context of a soundtrack or you like make EDM and you want to add voice to it, right? And you can just now kind of do some really interesting things there. So it seems like there's this really interesting renaissance that's happening in part due to diffusion model work and in part due to a handful of founders not getting distracted by LLMs, which are super exciting, obviously, but wanting to do things in video.

12:10It's a really exciting trend. And I'm guessing the success of some of these companies and their traction and growth is going to pull more people over to work in this area again. I think it was just an area of less emphasis for the last year relative to language. One of my favorite dynamics that happens in sort of technology ecosystems is that once people show that something is possible, like a lot of talent floods in, right? And you kind of get, you get a lot of competition, but you also get innovation coming in waves. That could be with Mistral developing open source models that are actually interesting from a reasoning perspective at relatively small parameter size.

12:55Or it could be Demi and Chenling and the Pika team creating text-to-video generation models that are really interesting quite efficiently from a training perspective. And I know we're both investors here, but I've seen a huge wave of people interested, as you said, in media diffusion of different kinds now that they know it's possible. And it has real benefits, right? Because it's very cheap to do. And from a data set perspective, the data is reasonably straightforward to get. I mean, it's hard to get, but it's not as hard as the entire internet and transcribing voice from videos and all the rest of it.

13:33The original stable diffusion model supposedly was trained on like 600k of gpu and these models cost in the millions to train not tens of millions at least initially right and so that's another big difference relative to the really big foundation models and language models and all the rest right and so you can actually imagine that in the language world you're going to have a lot more platforms that people build on in the diffusion model world image video etc you're going to have more people kind of grow their own right and people should still potentially try things on stable diffusion first just to test it out.

14:02It's back to the, you know, no GPU before product market fit, but they can still train their own model in a very economic way relative to like the amount of money a startup would raise. So I think it is a more accessible thing in some sense, unless you just go and build on somebody else's LLM, which people should do for most things initially as well. Yeah. One of the things I think is really interesting about this space as well is we've actually had leading researchers like say, you know, we're still very, very early in Video, video generation is so hard. It's data intensive. The data is like, as you said, it's not the whole internet, but it's problematic in that a lot of the training has happened on short clips.

14:37People aren't sure how to caption. It's expensive to generate. You have like complex, like sliding window approaches and others to try to deal with the like temporal coherence problem. There are many more unknowns about how to progress this type of product technically. Do you mean video specifically? Or video specifically? Yeah, because the teams for all these things are actually quite small, right? The Pika team is reasonably small. The mid-journey team for a long time was tiny. And so I actually think this is a good example where you can do a lot with very small teams. And to your point, there's all sorts of technical challenges.

15:09But the reality is you can get to the cutting edge with like a handful of people in these fields, which isn't necessarily true as much for, you know, other types of models or certain types of models, at least. So I do think it's striking how few people you actually need to do something really interesting here. And to your point, there's other challenges coming. three, four years, maybe it becomes harder. I agree with you. You don't need more than a handful of people. Like there's empirical evidence now, maybe a slightly different point, which is instead of like having to have a certain size of team to go deliver an LLM at scale, there are more degrees of freedom in how you would innovate technically in this area.

15:46And there's more disagreement on like how to progress. And I think that's actually just interesting for startups. Yeah, I think they should just use Qstar. That's probably. I think that's the main solution to most problems, I feel, in AI today. Okay, well, do you want to give me some investment advice given Qstar? Should I go home? Yeah, you should do mainly Qstar-centric companies. And so, you know, if they're doing Qstar, you do it. If they're not doing Qstar, you don't do it. So that's one big piece of advice. I think one of the things I would like to go back to and talk about is your point of view on like, Like, hey, a bunch of people moved away from diffusion models to LLMs.

16:23One of the reasons that people moved away from diffusion models to LLMs is because there's a lot more sort of text and code in enterprises that is obvious, right? Working with images and video and audio felt more verticalized, like where the B2B use case is. And I think what we're seeing increasingly is the creative fields are actually pretty commercial, right? So one of the things I'm most inspired by, and I think there's a lot of money in, is if you look at Mid Journey, one of the biggest knocks on them from investors or naysayers early on was, well, like, how many people want to make images?

17:12that's not a hobby, that's not a social network, like what percent of the population are artists. And this was clearly wrong just in terms of the scale that Midjourney has already reached. There's probably two pieces here, right? Like one, these tools like Pika and Heijen and Audio Generation and Midjourney, they make the pie bigger for creative fields, especially since If you look at Pika or HeyGen, they're really focused on all creators rather than just like the film industry professional. And like if you go look at the mid-journey use cases, and then I suspect the Pika and HeyGen use cases over time, they're very commercial, right?

17:55Like a lot of the things that you named or that people are experimenting with are really about communication, marketing, and advertising. Yeah, I think if you just look at it as market cap of incumbent, right? Adobe is a$280,$300 billion company. Like that's huge, right? And so I don't think the creative world is small, right? I think a lot of creator economy companies have failed in the past, which is a different thing, depending on how you think of the creator economy, right? In terms of, you know, celebrity-based marketing or whatever. But if you actually look at enterprises, obviously they use enormous amounts of imagery and video and other things to reach with and interact and brand and, you know, associate with their customers.

18:32And you look internally, you need to create imagery for slides or for other things and communication. And, you know, and so to some extent, you're kind of looking at different proxies and you say, OK, well, what are some of the proxies on the text-based side? And you could say, oh, you add up Microsoft and, you know, a few other companies like that. And you're kind of getting a rough proxy for some form of text. Not really, but, you know, I'm just simplifying things dramatically. And then you add up Adobe and a few other companies and that's your proxy for ImageGen, right? And so I think both are big.

19:05And then the question I think always with the diffusion model-based companies was, where are the biggest application areas? And the application areas also were a little bit driven by, well, where will incumbents play a role? Where will you get blocked by other companies in the ecosystem versus it's a natural new greenfield thing? Most things aren't truly new capabilities. Most things are just making certain things dramatically easier. There's the duality of that. There's the market expansion. it and more people can do this thing. And then there's a value contraction. Hey, you can do this a tenth or a hundredth the cost.

19:39And so often in markets like this, you see both of those things happen at the same time. You're simultaneously growing the market and shrinking it. Yeah. I mean, you have the, um, the, um, famous, uh, strategy, like your margin is my opportunity, right? I was going to say, I mean, well, these are actually higher margin things. I think really what you're doing. If you look at the sort of, um, my understanding, I need to look up these numbers again, but I think it's something like software spend is like, I don't know, I'm making it up half a trillion dollars,$500 billion a year. And then services spend is like three to five trillion a year.

20:13Right. And so really what you're doing is you're taking services revenue, which is very people intensive and low margin, and you're converting it into higher margin software revenue, but less of it. So maybe you take that five trillion and you turn it into$2 trillion, but it's 80 % margin$2 trillion versus 30 % margin, right? The margin dollars actually expand. And you see that in other industries. That's kind of what Anderil is doing in defense, right? They're taking a cost plus model, right? You buy a drone from Lockheed Martin for a million dollars and you get paid by the government 5 % cost plus, which means you get 5 % as your margin.

20:49So you make 50K off of it. And Anderil will sell the same drone or, you know, up the margin, right? But that's 50K. And so you're making the same margin on a 10th of the price. And so I think one of the ways I think about Anderil as a company is they're taking very bad, low margin revenue from other defense companies and turning it into higher margin, healthier revenue, right? Yeah, I'm going to edit the quote and just say like your ASP is my opportunity, right? There's a like a democratization that happened in the like latter half of the SaaS revolution, or really most of the SaaS revolution, which is instead of there's 100 very large enterprises that have some sort of CRM because it costs X dollars to deploy and implement Siebel, then you have tens of thousands of companies who can buy Salesforce and then companies that figure out how to efficiently distribute S &B SaaS on the internet, even though that is still hard.

21:54I think one of the things that is interesting here is, let's just take video generation as the example, the ASP of, you know, hundreds to thousands of dollars an hour, make it single digit dollars to generate per minute and expand the audience to many more people, right? So that's the democratization that is happening. Somebody told me a joke the other day, like somebody really negative on AI investing. There's only five businesses in AI that have breakout traction right now. Foundation models, waifus, mid-journey, co-pilot and inference platforms. I think there's both truth in it. And like, also, it's not that funny of a joke because you're like, it's true.

22:41Like there is a set of things that people are figuring out that are really early, many of which feel pretty different than like, yes, it's useful to look at the incumbent vendors like Adobe, but you're not fighting really the video editing software spend. You're eating into the production spend. The early internet version of this, by the way, is there's only five things you do on the internet. You go to Yahoo to look for links. You buy Pez dispensers and auction on eBay. You buy some books on Amazon. And then there's probably like some, like two bullshitty companies that you would have quoted as like hybrid things, right?

23:21So if you were looking at the internet circa 96, 97, whatever, you probably would have had a pretty short list of real use cases. And then a bunch of stuff you just thought was kind of dumb, right? And you'd be like, look, you're not changing anything. Like you're still using Microsoft Office and you're still using whatever shrink wrap software, you're still watching TV, right? And so I feel like we're kind of in that era of AI. The stuff that's going to happen right now is the really easy, low-hanging fruit, and then a bunch of dumb things are going to get built that aren't going to work. And dumb is not meant in a pejorative way.

23:51It just means it's very hard to tell what's actually a good idea in a new market like this. And that was true of the internet, and that was true of mobile, and that was true of cloud. There's a lot of these waves where there's a bunch of stuff that gets built, right? so I think it's the same thing right it's a very positive sign this has been such massive traction in such a short period of time for so many companies if you think about it it's actually kind of amazing so I'd actually take the other side of that but I totally get the point yeah well with the AI focused fund I agree with you I didn't change the name of your fund you should call it like conviction star or something conviction star yeah but spelled with a Q, like conviction.

24:32Yeah, conviction star. Okay, LPs, you heard it here first, conviction star. Very exciting. I should send you a t-shirt. Thank you. Please do. That'll actually be the swag for No Priors season one. If you're a guest, you're going to get the No Priors tequila and then a conviction star t-shirt. Yeah, that's very good. I'm very excited about the tequila. Anybody with a podcast has to have a tequila. We could actually call the tequila a conviction star with a Q. Okay. That'd be pretty amazing. I'm serious. Okay. It could be like a Q-shaped bottle. You know how they have like the really cool bottles for different things?

25:05A lot of Gil guys are no priors branding. Brand marketer. Yeah. I think I'm moving to LA and starting the brand. If you haven't yet been a guest, please write into the show and for the low, low price of one GPU, we'll ship you a bottle. Yeah. We're looking for a brand marketer to join the team too. Or not, actually. So if you work for Mr. Beast, just call me. Alad's going to do it. Okay, Alad, thank you for joining me on No Priors. Thank you for joining me. And I look forward to getting my Conviction Star t-shirt and tequila. Exciting. Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces.

25:47Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.