The Race for AI—Search, National Infrastructure, & On-Device AI

0:00In 2024 alone, I think we were more than 700 pieces of state -level legislation that were AI -specific. The average query on perplexity is 10 to 11 words. The average search on Google is 2 to 3 keywords. You've had an enormous amount of NVIDIA's purchasing orders come from the balance sheet of governments. You can actually reimagine the AR experience. If you're a founder like that who has the guts, then your impact on humanity ends up being quite generational. Here we are again, inching even closer to the end of 2024. And as we near 2025, here are a few dates to give you some perspective. We've had 24 incredible years of Wikipedia, 18 years of the iPhone, and 16 years since the Bitcoin White Paper release.

0:46So as we look to 2025, and the speed of innovation is only increasing, we continue our coverage of A16Z's big ideas. Together with the dozens of partners who are meeting daily with the people building our future. Last year we predicted. A new age of maritime exploration, programming medicine's final frontier. AI proskeems at Nebunet. Democritizing miracle drugs. On deck this year, closing the hardware software chasm. Game tech powers tomorrow's businesses. Super staffing for healthcare. And throughout our four -part series, you'll hear from all over A16Z, including American dynamism, healthcare, Fintech, games, and more.

1:26However, if you'd like to see the full list of 50 big ideas, head on over to a16c .com slash big ideas. And of course, if you missed it, check out part one, all about the intersection of hardware and software. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the company's discussed in this podcast. For more details including a link to our investments, please see A16Z .com slash Disclosures.

2:11Today in part two, we'll be talking about the topic of the day, the month and quite frankly the year, artificial intelligence. There are certainly AI, AI, AI, AI. And the race is on, whether it's across companies like Google and the Disruptor's Jacing Way 10 Blue Links, or sovereign countries trying to capitalize on the next frontier, or even device companies, figuring out their role as AI meets the edge. That is all on deck today. There's a bit of an innovator's dilemma here and I'm excited to watch it play out. That was Alex Zimmerman and I'm a partner here on the Growth Fund. The search monopoly ends in 2025.

2:53Google controls 90 % of US search, but its grip is slipping. Its recent US antitrust ruling encourages Apple and other phone manufacturers to empower alternative search providers. More than just legal pressure, Gen AI is coming for search. ChatchyBT has 250 million weekly active users. Answer engine perplexity is gaining share, growing 25 % month -on -month, and changing the search engagement form. Their queries average 10 words, three times longer than traditional search, and nearly half lead to follow -up questions. Clawed, GROC, meta AI, PO and other chatbots are also carving off portions of search.

3:4460 % of US consumers used a chatbot to research or decide on a purchase in the last 30 days. For deep work, professionals are leveraging domain -specific providers like Causley, Consensus, Harvey and Hebrew. ads and links historically aligned with Google's mission, organize the world's information and make it universally accessible and useful. But Google has become so cluttered and gained that users need to dig through the results. Users want answers and depth. Google itself can offer its own AI results, but at the cost of short -term profits. Google as a verb is under siege. The race is on for its replacement.

4:32Maybe start by setting the stage. So how big is the search market today? The search market is enormous. Anyone listening to this uses search. Virtually anyone with the internet uses search. But to put some numbers around it, Google that I just mentioned, they're the biggest game in town. They're approaching 200 billion of revenue annually. They're still growing double digits. it's highly profitable. Microsoft Bing, which has been the number two player for a long time, mid -single digit market share. So pretty small. They have 12 billion of revenue. This is a massive, massive market, one of the largest out there.

5:10And it does feel like there are forces reshaping this industry. So tell me about those. So there's certainly AI, AI, AI. But before we jump into that, we can level set with the legal pressure that's mounting on Google. So earlier this year, Google was declared a monopoly. The court ruled that they're spending billions of dollars to phone manufacturers, in the case of Apple, tens of billions of dollars, is monopolistic. It's anti -competitive and it's preventing their competitors from gaining share in the marketplace. They are the default search engine on all these phone manufacturers, and it basically makes it impossible for any of the others to gain share.

5:50So the exciting technological change, of course, is around AI. The Gen AI search providers are fantastic. And the market has been dominated by Google for close to 25 years at this point. And as a monopoly, they're not really innovating. They have no incentive to until now. If you think about the Google experience as it is today, it's just a long list of links. and the first few are sponsored, they're ads. And I as a user, when I make a query on Google, I then need to make the decision. I have to sift through the information on that site and find the answer that I'm looking for. That's actually a pretty long process.

6:33Instead with Gen AI, I can just get the answer. So on chat GBT, perplexity, Claude, or when I'm chatting with my AI friends, on character AI, or on Po, I get an answer immediately. And that's just a much, much better experience. And I hear tons of people saying they're trying these new tools, but maybe you can give us a sense, is this shift really stark? Are people really moving over? People are definitely moving over. And I'd say there's four main reasons. One is the one we just talked about, the shift from links to answers. A second one would be how they are personalized, how the answers feel interactive, how they are conversational.

7:16The average query on one of these new services has a follow -up question. So it's not just about that initial engagement, it's about the ongoing conversation. The third difference is about how these new services engage with complex queries. So the average query on perplexity is 10 to 11 words. The average search on Google is 2 to 3 keywords. As you can imagine, their ability to leverage that information and get you what you want is much higher. On Google, you're going to get a list of links. One of those links might have one side of the debate. Another link may have the other side. You may never make it to that second side.

7:56On perplexity, on chat GBT, they're going to synthesize both sides. They're going to give me both perspectives. And then the fourth, they're just not cluttered with ads. That may change over time. But putting it all together, it should be no surprise that these AI native services are gaining share. For 10 % of consumers, they're now referring to chat UBT as their search engine of choice. Perplexity queries are growing 25 % to 30 % month over month. And I saw a survey last week that 60 % of consumers for purchase decisions in the last 30 days used a chatbot. All of these services are growing massively.

8:36So if we look at some of the tools that exist today that are coming for that market share, you take a perplexity or you take a chat GPT, those are also broad -based search engines or chatbots. But there are these other players you mentioned consensus or heavier, for example, that are verticalized. Does it surprise you that the last wave of search engines weren't verticalized? It does not surprise me that the last wave or the next wave will result in a winner take most dynamic. It's a pretty monopolistic market. Search is very much a distribution game. Google has an incredible brand. They have incredible direct traffic.

9:14They are the default search engine on most browsers. They are the Kleenex, the Band -Aid of online search. It's important to note that search engines definitely benefit from network effects. More users coming to Google provides more data around preferences for Google. that means the next search has more relevance and brings more users. It also means more users means more advertisers, more profit dollars that you can invest back in the service or into ensuring that Apple puts them as the default provider. Absolutely. And so as we think about this next AI wave, it sounds like you think maybe a similar dynamic might be at play or how do you think about maybe the smaller players actually finding a wedge or differentiating?

10:01One framework for thinking about how search could fragment or verticalize is just looking at why does vertical software beat horizontal software in some cases. And it's typically the vertical requires a specific user interface, proprietary data, features, workflows, compliance, etc. And that's why VIVA in life sciences and pharma can beat Salesforce in horizontal CRM. For the average query on chat, ebt, perplexity, or Google, they're pretty good. You don't need a vertical specific application. But for deep domain research, I can imagine a world in which standalone apps thrive. In the case of our portfolio company, Hebija, they have a unique interface.

10:53It looks like a spreadsheet, which is native to financial services and their customers. It brings in public filings, earnings, transcripts, but also private data. It can bring in research, it can bring in survey results, you can query, and then the output can also be specific to that industry. Not just look like a spreadsheet, but populate a meeting agenda so you can go in immediately prepared. And so when I think about vertical search or vertical apps, it's not just about the search, it's about everything around the search. That's a good distinction. And as we think about how this maybe continues a progression of this industry, if we go back to the last wave of search, we did see a bunch of search engines get traction to start maybe altivista or ask jeeps.

11:40We all pick on ask jeeps. You know, all the gen Zers will not know what the hell, any of that is, but they had some traction and then Google exploded and took over. Do you expect that same kind of consolidation or is this time different? I think consolidation should be expected in general purpose search because of distribution because of network effects. Google won the last time around in part because of their page rank algorithm. It produced superior results. It had a minimalist UI. It was really simple. Ask Jeaves, Altevista. They had tons of ads. It was cluttered. Links on the homepage. No one wanted to use it.

12:20The irony is that today, what we're all complaining about with Google is that their pages are cluttered with ads on the results and that creates the opportunity for these new search engines. I would expect consolidation again. And I think something else that's interesting is what you're pointing at with some of these verticalized solutions is that it's not just for the everyday consumer, but also for the lawyer or for the academic researcher. Do you think that is also part of the fragmentation? Do you expect that to continue? Search historically has been thought of as a consumer product. And for good reason, I make a lot of searches that are related to my personal lives.

13:00But I use as do all the professionals you named search at work. I probably make more Google searches at work than I do around personal matters. But there is a category of enterprise search in a market that has existed. And you can think of that as querying box, drop box, sales force, all in one. But I think those two worlds are going to blend together. Consumer search should not just be limited to what's on the web, and enterprise search should not be limited to just proprietary data. It should have both. And a lot of these AI native services are working on that. Yeah, that's a great point. And as we think about those two models, maybe blending together, we think of consumer search for sure.

13:41it's always been an ad -base model, or at least today. Google is this massive economic engine, but no one's paying a subscription for that. The new entrance do seem to have a more subscription -base model. Is this a temporary thing, or how do you see those two dynamics playing? So the subscriptions could stick around, but I think this is going to continue to be a digital advertising -focused market. And I think digital advertising is going to grow because of these AI services. So these AI services today, they don't have ads. I imagine that's going to change. It's been in the news that Perplexity is already talking to advertisers.

14:16They have a high income, highly educated user base. That should be attractive to these advertisers. But as we discussed earlier, the queries on these services, they're longer. They're more complex. They're more detailed. They're more personalized. And because of that, there's greater intent, which should be more helpful to advertisers and producing the best results on their end and creating an even larger market. But in the meantime, these subscription business models, they make a lot of sense. They bootstrap the business, they cover costs, and from a personal perspective, I'm very happy to pay $20 a month for ChatchyBT, Proplexity, and Po.

14:55I mean, they provide so much value. A lot more than $20 a month. Yes, yes. So obviously this was your 2025 big idea, and I know this is gonna be a many year, maybe even decade -long progression, but what are you paying attention to in the next year? What opportunities are still on deck? As I looked at 2020 -25, Google it may be on decline, but I don't expect them to go down without a fight. Google and Meta, I think they can be big players. Google AI overviews already has a billion monthly active users. Meta's not far behind. They have 500 million monthly active users. Again, this shows the power of distribution for search.

15:33And maybe to that end, Apple, if you could call them a dark horse, is a dark horse. They are not investing the cat -backs like Meta and Google and Microsoft and Amazon, but they control a central node to the consumer. And if they wanted to build a search application, the next day they could have a billion users. We just heard how quintessential winning AI is to multi -trillion dollar companies like Google and Meta. But what about nation states? In the race for AI dominance, compute has become critical national infrastructure, but not every country is equipped to compete in that race. That was...

16:13Anjane Mida, I'm a general partner here at A16Z where I focus on AI infrastructure. And here's his big idea. My big idea is infrastructure independence. And it's the idea that a lot of countries and regions are starting to realize that modern AI, deep learning based AI, generative models, are a form of what have been called general -purpose technologies. In the history of humanity, we've only had maybe 20 or 22 or so general -purpose technologies. And these are usually types of technologies like electricity, the printing press that have very broad -based applications in society. They end up being largely horizontal economic multipliers and progress multipliers across a whole set of pillars and domains and society.

17:00There are usually two moments in the adoption of a general purpose technology where first, countries, nation states start asking, are we going to welcome this technology or are we going to be hostile to its development? Okay. And that's the first step that becomes pretty important in a country's progression or in a nation states progression, a region's progression. Do we want to adopt it? Do we allow this? Regardless of whether we own it. Do we embrace it or not? Yeah. And then the second is, do we build or buy? Right. Which is, can we trust somebody else to provide it for us? Where well past the stage of, do we embrace it or not?

17:34We're already well into billions of people around the world now having already embraced it. So the governments don't really have a choice, so to speak. So in a sense, AI is already percolated throughout society at one of the fastest diffusion rates of any general purpose technology, because it's piggybacked off of years and years of digital infrastructure. And so now the question everybody's asking is do we build or buy? Yeah. It's the single largest probably purchasing decision that's going to happen in the next 24 months is donation states start buying... Do they build or buy? Do they build or buy?

18:05Yeah. I love the parallel of companies because there are many companies that do choose to build, but also companies that choose to rent or buy. Right. And so as you think about that, there's large nations around the world like the United States, which are clearly building. But talk about the argument for smaller nations or the 190 plus that should be thinking about buying. The good news here is that we've got hundreds of years of human history to look at for clues about what happens next. If you're a small country and you were in the early 1900s, you were watching the modern electrification of the developed world, the United States or Europe.

18:45If you chart what happened with many of those countries, many of them decided to actually enter into what we're called joint venture agreements. It starts with a joint venture with a country that's at the frontier. These countries at the frontier via what I call hyper centers. These are countries that have the ability to develop, train, build, and host their own frontier models. I call them hyper centers mostly as an homage to the word hyperscaler, which is that they've been a handful of companies that have had the compute and the talent to actually build frontier AI. And now I think what we're seeing is it's shift from just those companies driving a bunch of frontier AI to countries and regions driving it.

19:20And so if you're a small country and you're going, well, we certainly believe that it's important to have our own AI infrastructure. We want to be independent, but we don't have all the compute required to train these models. Or we don't have all the talent locally. Then what you enter into is a joint venture with a country or an overseas partner that matches your values. And this is the really important thing about AI models and how they're different from infrastructure like electricity is there's a fundamental encoding of human values in AI models because they're trained on data. And the data has these local norms and cultural values embedded.

19:59And so if you happen to train a model on a bunch of internet data collected in the US, the models are just generally American. American. They're encoded with that. And if you train the models on data in France, they actually subtly have a bunch of different values encoded in the models that reflect those cultural norms. And so I think step number one, if your small country is actually being a little bit crystal clear about which value systems you align with most out of the hyper centers. Now it's not lost in people that the way the internet worked out was they essentially ended up being two internet, right?

20:35The Chinese internet and the rest of the world. AI may not end up looking that different. And if you're a small country, what you really have to figure out is whose values align more with yours. A good historical precedent to look at here is the technology of money. Right, money is a pretty general -purpose technology. And what happened in the early 1900s with the modernization of finance, is a number of countries started to ask the same question is, do we build or buy our own currency? Do we rely on the dollar? Right. Or do we have our own currency? And that led to the modern day currency regime where the dollars are single global reserve currency.

21:11And that happened through a bunch of allied cooperation where a number of countries realized they did not have the local resources required to hold the peg of gold right to the dollar. And so I think what we're going to end up seeing is the emergence of very similar to what happened with currency flows where you have a couple of large countries that control their own sovereign currencies. Yeah. You have the US, you have China, you have India, and then you have a number of smaller countries that decided they wanted to be Flow points. So you have Singapore and Ireland and you have Luxembourg and Zurich that become massive global leaders in modern finance because they decide they want to ally with one of those power centers.

21:55Yeah. Right. So if you think about in the AI world, let's call regions at the front to your hyper centers and then we have compute deserts and these are places that have literally no install base of compute capacity to even be relevant. All the smaller folks have to figure out which of the hyper centers they want to align with and how do you become a modern day, Singapore, Ireland, Luxembourg, etc. for the world of AI infrastructure. And so it starts with deciding whether you want to be a compute desert or not and if you're not and you're going to actually embrace AI infrastructure as a government, then I think you've got to figure out which hypercenter you want to align with most.

22:28And then it becomes actually quite easy to reason about how to be a valuable ally. That's such a good parallel because a lot of people think about resources in terms of the farmland that you have, the people who are working in that economy. But what you're pointing out is that countries for a long time have offered value or offered a resource in other ways. And as we think about AI, there's a few things that you've pointed out that countries can invest in whether it's the compute capacity that they have, the energy resources to power AI and forward thinking policy. So maybe we can break down each of those.

22:59How do you think about each of those blocks and how countries should be maybe maneuvering or investing in those things? The good news is that there's only three or four ingredients here that really matter. The first is compute, which we've talked about. The second is abundant and low cost energy, which powers the data centers. The third is data, just the availability of really high quality tokens for these models to learn on. and the fourth is regulation. So that's the good news. Now, the bad news is that world is pretty unevenly split up, right? Some countries just have dramatically more compute than others.

23:28Yeah. Others have dramatically more energy than others because of their natural reserves. Yeah. So if you are in the Middle East, you may not have massive data centers yet, but what you do have is vast reserves of oil. And how you translate that into becoming a hyper center is quite simple. It's the law of comparative advantage, right? You've got energy. You should use that to attract the world's best teams and companies and foundation model labs and so on by trading what you have with what they have. And so I'm quite bullish on allied ties between countries that recognize what their strengths are and then partner with other countries to fill that gap.

24:12And by countries, I mean private companies too from other countries. One of the things we may end up seeing in the coming years is jointly trained models between countries. Basically, think for most countries, it's impossible to have total infrastructure independence at all parts of the stack. What is much more feasible is to be great at one part of the stack and then collaborate with another sovereign or another country or region to achieve joint independence from a value system that you don't subscribe to, like the CCP. And so I actually think what's more important is for countries, regions, and frankly, I'm some of the world's largest companies that operate at nation scale to assess which parts of the stack are critical to them that they must have independence from.

24:56And the answer there is the function of what assets they already have, right? They're strengths. And then if they've got a critical gap to fill, it's to go and buy that. Now, in the long term, you might be able to build things out, but with infrastructure, of this kind, it can often take years, if not a decade long scale. So as an example, lower down in the stack from the model layer, you have the chip layer. And even below that, you have the lithography layer, right? There's a company in Holland called the SML that builds literally the world's most important machines. How many machines do they make per year?

25:26It's some very small number. Each machine costs about $200 million. And I think they did $23 billion in revenue this year. 40 % of which came by the way from China, because China was stockpiling ASML machines before a bunch of export restrictions kicked in. Yeah, and they're the only company that can actually make it easier. They can do EV lithography of this precision. Now, is it feasible for the US to say we're gonna build our own ASML like tomorrow? No, I mean, it's just gonna take 10 plus years. Right? EV lithography just takes a really long time. On the other hand, is it feasible for a smaller country to say we're gonna train our own local models at the frontier?

26:01that's a little bit easier to do over the quarters of time scale. If you've got a leading research team, if, and that's a big if, right? There's only a handful really of research teams globally that are capable of this. And so to answer your question, yes, I don't think sovereign AI or infrastructure independence means you have a hundred percent ownership over every part of the stack. That's infeasible over the short term. It means that you don't rely on somebody for a critical part that you don't trust. Right. Can we talk about private companies for a second? Because you've brought them up a few times.