Listener Q&A: AI Misconceptions, The Reality of Regulation, Infinite Context, Incumbent AI Execution and Startup Ideas

0:05Hello, NoPriors listeners. We're excited to just do another Hangout episode with me and Alad and answer listener questions. I think a fun one to start with would always be a place where we're disagreeing with the market. So I'll ask Alad, what are people getting most wrong about AI right now? Yeah, I guess there's two or three things that I wouldn't say they're necessarily getting wrong, but I just feel there's some misconceptions about. The first one is, I feel like a lot of people are kind of treating this as an extension of the last decade of machine learning that we've seen in the sort of convolutional neural network and RNN world.

0:41And everybody keeps talking about it as if it's that old world and they keep emphasizing certain aspects of data and other things which are important, but not as important as they used to be. And in reality, we've had a technology disruption. We've shifted to two very different architectures, diffusion-based models, which is a statistical physics model for ImageGen. And then on the language side, we moved to these large language models, which some people are now calling foundation models. and you know fundamentally that's different from the prior wave of NLP in terms of capabilities in terms of the way it works but also in terms of insights around things like you know just the fact that you now have this really interesting like chain of logic or chain of thought style processing of information and the ability to act and synthesize information in a way that never existed before for NLP for example and so I think one big misunderstanding is oh this is just ML and we've been doing ML for 10 years and it's the same thing and it's totally different so I think that's one big sort of area that I keep seeing people get things wrong.

1:36Or at least, you know, there's these misassumptions. Second is I keep getting pinged by people saying, hey, what's working? What's working? And there are a few things that are truly working at scale, you know, OpenAI and MidJourney and a few other things. But the reality is it's been six months since ChatGPT came out and most people became aware of this. You know, like, you know, I think we both started investing or being involved with the area on the general viz side much earlier than that but the sort of starting shot for the industry was six months ago and then gpt4 came out maybe three months ago and so everybody's acting as if this is an old thing and again i think this ties into the prior point this is not a normal extension of what nlp used to be like this is a fundamentally new set of capabilities and so when people are saying well look no enterprises are adopting it very much at you're like well it's been six months since most people realize this was that important in six months is one planning cycle for a big enterprise, right?

2:30So people are just planning what to do. So I think that's a second one. I actually went on a walk with a growth investor that I actually have plenty of respect for. But they asked the question of like, oh, like, what's working? What should we invest in? Do you have anything that, you know, popped off that we should keep an eye on? I'm like, yes. And we actually had a very similar conversation around what adoption in the enterprise looks like. And my prediction based on the sort of like negative point of view that this investor had of, oh, like it's not enterprise ready. Like this is just a hype cycle.

3:03And like, we're not actually going to be investing here. Like we explored it. We met 100 companies. We're done. I think other people will arrive at that conclusion. And like any other wave of enthusiasm, you'll see people abandon it as well as a technology. Yeah, I think people just just prematurely assuming it's a continuum from before and therefore there's nothing new here. And I just think that's wrong. And I think people will realize that. I actually don't think it's going to, I think there's going to be more hype rather than less coming simply because certain things are really starting to work at scale from a revenue perspective and really quickly.

3:38And we've seen the first wave of viral apps in terms of things like Lenza or other things that really ramped quickly and then went away, but those are signs of real traction and real usage. And so I just feel that it's very early in the hype cycle. There's more to come, but it's a different technology. And I just think people don't appreciate that. Or I should say, it's an extension of some technologies in some ways, but fundamentally, the capabilities are very different, right? You're still using deep learning, but it really implies a different modality of how these things work. And then I think the last place, the third area that I think people are kind of getting things wrong is the mad rush to call for regulation by people working in the industry strikes me as very unusual and a bit naive in terms of what that actually means.

4:17And so I'm on some private chat groups where people are talking about regulation and they're like, oh, well, the regulators will assign a panel of experts from the industry who will then decide what we're going to do. And I said, no, that's not how it works. What happens is you have some agency established and just like people keep talking about how it's just like nuclear. And you're like, you do realize that once the NCR, the Nuclear Regulatory Agency existed, we had no new nuclear designs approved for the last 50 years due to this agency existing it's not going to help us it's going to hurt the the collective motion forward and there's so much at stake in terms of the positive things this technology can do for the world like if you think about global equity around health care around education this is the single biggest motive force to make anybody around the world irrespective of their upbringing or background or or any other aspect of diversity.

5:09This is the biggest single thing that can impact things for the positive for healthcare and education going forward is two examples of global equity. And so, you know, that's one of the other areas that I kind of feel like I differ from the pack. I actually am quite a bit of a doomer in the long run, but in the short run, I think that regulation or misregulation can really take things down a dark path. So those would be my three. How about you? Yeah, let's talk about, let's talk about nuclear first and we'll go back to that because you brought it up. I think it's actually really interesting that people want to put it in this bucket with nuclear and climate of existential risk.

5:42And I'm not saying that there isn't a version of technical progress here that leads to existential risk, as I think you implied. But it's funny in the way these things relate, because as you said, like nuclear regulation, I think it's like broadly agreed upon in the scientific community that nuclear power is if you look at the tradeoffs of waste management and then you look at the progression since second, third generation reactors have been developed and the absolute like complete freeze on nuclear power in the United States and in many other countries because of proliferation concerns. concerns, like that's a huge impediment to one of the major contributors to actually, you know, at least creating some progress on the climate change front.

6:30Yeah, it's proliferation and broader safety concerns because the U.S. is still at something like 20-ish percent nuclear power. Japan is 30 percent. France is 70 percent. So people have been using nuclear and they've been using 60-year-old reactors this whole time without any real issues. If you actually look at death rates or things like that. Right, right. What I'm pointing out is that there have been two generations of improvement in terms of efficiency and safety of reactor development. And specifically and concretely because of the regulatory cost to get a new reactor license, we're building none of something that is newer and safer.

7:05Yeah. And to your point on safety, the existing reactors are incredibly safe. If you actually look at death rates from nuclear, they're incredibly low. You know, there maybe was one person who died during Fukushima from the nuclear reactor when, you know, the biggest earthquake and tsunami in decades hit the entire coast of Japan. And there was something like a dozen other reactors on the coast that were completely unaffected, right? And so it's just very overblown and overstated. and I kind of worry that in the short run the same thing is happening on the AI side again I'm a long-term doomer like I think there's real risk but in the short run I think we're bundling hate speech and bias and all sorts of things with existential risk and I've never seen the technology industry rush into the arms and embrace of regulators which they don't understand and have never dealt with before and if you look at the crypto community or you look at people who've in healthcare or in ed tech or other areas, they understand what it really means to be regulated and the potential for capriciousness, right?

8:06There are some really great regulatory actions, but there's a lot of things that tend to be arbitrary at times too. And so I just think it's a very odd moment in the history of the technology industry in terms of how people are acting. But I guess on a more positive note, do you want to talk a little bit about some of the areas that you're excited about in terms of either big picture benefits of AI, or I know you've been thinking a bit about Kodja and other areas. Yeah, I think maybe adding to your point on global equity, there's one way to look at these models as, you know, encoding a huge amount of the knowledge that is available publicly on the internet, most simply, right?

8:43And the idea that you don't want to give that to as broad an audience as possible, when it is so cheap to offer, like, some very flawed representation of knowledge in the world, to me is ridiculous, right? And so I think it's a sort of sub point within education and like structured education with curriculums and the ability to train in many different fields from science to primary education or whatever it is. But I think of it very much as like, do you want people to have, you know, information access in a way that people think of like the web and search as something that you want universally accessible.

9:23Yeah, on Cogen. So this is one of the areas I am most excited about, not only because like we have seen a product, you know, as you said, one of the few that is like massive, has real impact on workflow and now generating real revenue for Microsoft and for others. But because I think we're really early, right? Like today, you know, you have Copilot looking at your five open files, your five most recent files and doing function completion. And like that already has such a massive impact on productivity. But I think we're really early in figuring out how to deliver more context to these models and have them work in different ways, right?

10:07And in different user experiences. And so I think, you know, Gravely and the whole co-pilot team, they tried a bunch of things, including like everybody starts with chat and ended with this continual autocomplete, which is a great experience. But I think there are a lot of really interesting ideas that are floating around now from, I think, the more obvious like search or to the very fun like, okay, well, if you just had like, you know, AI Elah, the junior developer that takes any JIRA ticket or shortcut or linear issue and makes a best effort guess at actually writing that code with the context you give it and submits a pull request.

10:49Like, I think people would really love that interface. And the question is just how can you get some of this stuff to work? And I think we're going to see a lot of progress on that front. Yeah, it definitely seems like there's a few different companies starting to work in that area now too. Like there's magic.dev. I think, you know, there's another company that Redpoint just announced they were backing. Yeah, this is Jason Werner's company. I do think it begs this like broader question that's become more interesting as the context window for the available models has expanded, right? First, OpenAI to 32K and Anthropic to 75 and 100K.

11:25And you see these somewhat exciting research headlines like, you know, two million, two million token context window. And the reality is, first of all, that's it's not quite true. And second, like, you know, I have a strong point of view here that context will expand to fit the window. Right. Like the the amount of information or the instruction you can give to a general intelligence is much more than a hundred K. And given the relationship of the context window to attention architectures and the quadratic limitation here, I think that it's going to be a continual area of investment of figuring out how to get efficient context into a model as a product insight.

12:09Right. And I'll give one like very concrete example. We held a like an AI DevTools hackathon for a bunch of like college age new grad builders at the conviction offices this past weekend. And thankfully, with the support of Anthropic, we're playing around a bunch with big context in Windows. But even if you just want to feed in, for example, like Kubernetes documentation, you hit that window very, very quickly. Right. So I think there's a lot of enthusiasm from builders about what you can do with that window. But, you know, catastrophic forgetting and the idea that like 100K or even a million tokens is going to solve all our problems.

12:49And like you just ask your questions and dump in all context naively is ridiculous to me. I think it's a really important area of product and research work. Yeah, I think in two years it'll be reasonably solved in terms of the size of context windows will be usable. Like I'd be happy to take that bet, but we'll see. Like, I think the context windows are going to get very large. I agree they're going to get very large, but do you agree that people will just fill it? And it's like, it will continue to be a question of how to be efficient with that. Up to a point, yes. I guess the question is, say that you had a billion token context window.

13:20Is that big enough? Like, where does it asymptote in terms of value? And you can think of it in a few different ways. Like, in your code gen example, the question is, like, how big do repos get, right? Or you can ask in the context, I know some people who want to use LLMs for private data. or I should say, you know, private data across many different users where they want to be able to differentiate and dump all the specific user data into the context window. So they don't sort of have any crosstalk in terms of data. And, you know, it's a data privacy approach in some sense. And so the question is like, how many tokens do you need to represent all the data that's relevant to a person relative to a specific query that you're worried about relative to PII or other sensitive information, right?

14:01And so I agree, like the windows will get bigger and bigger and bigger, but eventually some of these things asymptote. And it may take years or decades to asymptote, right? That happened with CPU on your computer or bandwidth, you know, is still probably limiting for certain applications, although not all. So I think it's an interesting question of like, what number is it enough? And you could probably come up with some heuristic for that, just based on the types of documents that you want to have access to and the specific use cases that you're working against. Yeah, I guess my view would be, I don't want to take the bet against the like billion token context window with you.

14:35But my point is more that we're, I don't think we're close to the asymptote today where people don't worry about it. And I think even so, even if that window gets very large, like the state of work today is that ordering matters. And like you, and there's very little research as to like what sort of structured information, maybe you just described it as like, okay, as soon as your context window gets to be, you know, 30 to 100K a million tokens, like your prompt engineering becomes a much broader field, right? That's probably the generalization. Prompt engineering, and then like, probably a little bit different than that, but just like, how do you structure data that is put into context against these models?

15:16Yeah, exactly. Yeah, I think there's a data structure piece of it, and then there's the sort of memory piece of it. And so I think both of those things will start to feed into, how do you make trade-offs between different aspects of what you feed into a model and how do you do that? So yeah, I think it's very exciting times. There's a lot of very basic things to work out. And again, I think this is back to that point of it's very early in this field. And I think a lot of people are assuming that we have this ongoing continuum from the past and aren't these all solved problems or why are these things still open questions?

15:48And I think the reality is just so freaking early. Let me ask you about something that I know you've been paying attention to. So I think like two fronts that are interesting with regards to China. One is like what the reaction to hardware sanctions as of last October have been. And two would be just like there's been a ramp, as you might expect, in terms of funding for Minimax and Baidu. What's your thinking on all this? I think it's kind of the expected shift, right? And so if you look at the Chinese internet or software ecosystem, it's always been walled off from the US and there's been a focus on building local heroes or local incumbents.

16:29And so that's part of what led to the rise of WeChat and a bunch of different messaging apps because the US-based ones were just blocked in China, right? You couldn't use Twitter and you couldn't use Facebook and you can use all sorts of different applications. And so that gave a lot of room for local incumbents to grow up and really become the dominant platforms in these countries. I remember, and this may have just been a rumor, when I worked at Google, I visited the China office there briefly when they still had a China office. And I remember rumors of the fiber to the Google data center getting repeatedly cut as a way to effectively take down the services Baidu was just getting started, right?

17:03So it sort of gave Baidu space to grow as an alternative search engine. And again, that may be incorrect. That's just kind of the rumor that was floating around back then. And so, you know, you saw the ability to create these local heroes, And it seems like the same thing is now happening on the LLM side. So Minimax raised$250 million at a$1.2 billion valuation. To your point, Baidu just announced$145 million AI venture fund. And so it'd be surprising if the Chinese didn't invest very heavily in creating local heroes in this very important technology location. And this leads back to questions around, well, how does this translate into different aspects of competition between different countries and regions?

17:44And how do you think about that relative to how you think about what to regulate or not regulate in the context of the U.S.? Because fundamentally, I think these companies are going to go at this area and this technology very hard and aggressively because it's both very important, but also it can tie into national security or other concerns. So I think it's definitely worth watching. Yeah. And I think one of the things that's also worth just noting in terms of the profiles of people starting these companies, the Minimax team is former SenseTime folks. Right. And for our listeners that don't spend as much time in the Chinese ecosystem, SenseTime was a like broadly a facial recognition company that did a lot of work with the Chinese government.

18:26Right. And so I think, you know, I think one vein of discussion as part of the debate around regulation and whether or not having these models developed in the United States or in the EU is a good thing. People have been discussing like, well, what does AI with American values in mind mean? I think it's probably very different than something that starts with a perspective of like a government facial recognition company. Do you have any thoughts on the hardware sanctions? Yeah, so right now, NVIDIA is the engine for everyone's progress. And Jensen is laughing all the way to the bank about that and much deserves sitting on his empire.

19:10But I do think that if China cannot get A100s, H100s, top-of-the-line hardware, like GPUs with good interconnect to train models, they will invest heavily in solving that problem. And so one of the things that has been quietly happening is investment by both the like networking and hardware players that are domestic, like a Huawei, in, you know, chips and systems to support AI training. And this is not like a trivial pursuit, right? You have a small set of companies that have created like TPUs at Google or other accelerators that actually work for at-scale transformers training. But I think it will happen over time, right?

19:55It's not an unsolvable problem with the right talent and enough capital. And so there is a piece of me that feels that some of the regulatory attempts to control model development are misguided. And also, they don't really take into account what you need to go create these models and what those sort of impacts are going to be. Because what is happening, not that I'm against these sanctions around AI training hardware, but what it's doing is encouraging a domestic industry in the long run. Right. I think separately from that, there's another interesting vein that you and I have talked about where like you're trying to it's probably two different technical approaches.

20:40Right. One is figuring out how to make more heterogeneous. I guess it's all sort of the same thing, but making more heterogeneous compute work for both inference and training. And so this is companies like Together or Foundry or even at the sort of compiler layer, like something like Hippo. And I think that that's interesting depending on like what layer of abstraction you're at, if that is a compiler, if it's scheduling and abstraction. But I think that there is more innovation to be had at the infrastructure layers, given it is a constraint to the ecosystem in a way that it hasn't been, at least not in a way that's been like paid attention to in a very long time.

21:22Right. Yeah. I guess one other big thing that's happened over the last week or two is all the different incumbent announcements. Right. So there's all the different things around Microsoft, Google Duet, incorporation of all these things into various products in terms of AI enablement. Do you think this impacts any startup opportunities? I think it definitely does. It also is, to me, not that different from the past, right? Let's sort of like separate into the different components of whether or not this is going to actually work at the incumbents and whether or not that means like there's still startup opportunity around some of these platforms.

21:57Like there was always risk in building on other people's platforms, right? Be that Facebook games or being part of the G Suite ecosystem or building on Shopify, right? Like this is not a new question for startups. And I think the name of the game has been like, can you create sufficient value on a broad enough platform that you can break out? Or is it a big enough independent business, right? You look at something like Klaviyo and like, you know, there's both relationship with Shopify as a platform vendor. And like it's a multi hundred million dollar business looks like there was enough there.

22:29And so I think it's like very idiosyncratic to every situation. I think the other consideration would just be like Salesforce is an amazing company in terms of many dimensions of execution. But it does have a repeated pattern of like announcing, you know, interesting products that like never seem to see the light of day or get broad customer development. Right. And so I think that, you know, to your point, it's a really young industry. you're asking companies that have become larger and more difficult to drive at a rapid pace facing technology disruption. And I think you're going to see varied levels of execution on that, right?

23:07So six months in, that's like one planning cycle. That planning cycle hasn't been executed at the biggest companies. They're saying they're going to have all of these AI features and AI, you know, development tooling if you're Microsoft. And I think like we should see if any of that stuff gets shipped over the next quarter or two. Yeah, I think it'll take 12 to 18 months for a lot of larger companies to really start to have anything. Do you have a point of view on this? Yeah, I think it's overstated in terms of the impact. Like I think it's expected, right? In other words, if you don't expect Microsoft to incorporate this into every product, then you're probably misunderstanding what Microsoft is doing.

23:45I say that with no inside knowledge with Microsoft, I'm just saying they've been very open on it. They were very early to open AI. They've been moving very aggressively. It takes time to rule all these things out at scale. Same with Google. They're going to start incorporating it in lots of spots. Now, sometimes incumbents try to do something and it backfires or doesn't work very well like Google+. And sometimes you do something and it works extremely well because of your distribution and cross-sell like Microsoft Teams. And so I think there's always the potential for some mesexecution or the potential for bumps along the way.

24:12But you should assume that many of the more savvy incumbents are going to adopt this reasonably soon. And reasonably soon for an incumbent may mean 12 to 24 months. But if they suddenly cross all everything to their existing base, then it can really hamper a startup's ability to function, right? And so as a startup, you should kind of be asking, well, what will happen if the incumbent adds this? And do I still have an advantage? And what if it takes three years versus one year? Does that matter? And if it matters, then great, you have an opportunity. If it doesn't matter, then two, three years later, you can still get really hampered.

24:44Then you have to rethink what you're doing or ask about how you build the fensibility or what else you build against it. So I think this is all expected. And I don't think it's that surprising. And I think we'll see more and more announcements in the next 12, 24 months from lots of big companies saying they're doing stuff. Honestly, the thing that surprised me the most was how fast Adobe moved. I didn't expect them to actually launch products this quickly. And so I I was pretty impressed by that. And Adobe, of course, is one of the better run large technology companies in the world. And you could almost measure the rapidity with which somebody adopts this technology almost as a metric of management competence.

25:18The most competent companies, or at least the ones closest to this technology, will adopt it soonest. And the ones that are still figuring out a lot of other stuff will take a little bit longer to figure it out, but we should assume this is all coming, just like mobile. Like some companies took 18 months to launch a crappy mobile website. And then three years later or five years later, everybody has apps. And 10 years later, B of A has a great mobile app, right? So, you know, I think these things happen with time. Yeah, we would all buy a LODs ETF that is predicated on signals of actual product execution in AI amongst incumbents rather than announcements.

25:58If you're gonna, you know, put that side hustle together. Yeah, I do think that this does create new openings to go against certain incumbents that were untouchable before. And I do think that there is almost this room for a Rippling or Ashby-like or HubSpot-like fat startup approach, right? So this feels to me like the first time that maybe Salesforce is vulnerable or certain ERP vendors are vulnerable or others where you have a lot of very dense interconnected software with lots of connectors and integrations to other applications as a moat, right? And the big moat for many of these companies is A, people know that the brand and they want to buy them and all the rest of it and they're already running on them so it's hard to displace but secondly is just the breadth of the product plus the breadth of integrations that you have to do if you're implementing an enterprise resource planning system like a sap or a net suite right or if you're trying to displace sales force right and so i do think the fact that you can now build things pretty rapidly that interact in a really rich way with each other and where you can fundamentally take advantage of sort of the ability to munge and interact with data and create your own connectors and everything else that AI gives you, it does seem like it creates an opening for the first time for some of these really broad-based product areas.