Inside the collapse of the internet economy (and what comes next)

0:00Today, I am thrilled to welcome Matthew Prince, the CEO of Cloudflare. It's a company that sits at the very heart of the internet. You might not know of it, but I'm pretty certain that at some point in the last 24 hours, Cloudflare has helped you get something done on the web. The company funnels about a fifth of all internet traffic to us. Now, as we move into the age of AI, Matthew is once again at the heart of thinking about how that technology is going to affect the internet at scale. And that's what we're going to talk about today. Viewers know I love the pipes of the internet, and that means I have a soft spot for this company.

0:38I've even got the t-shirt. But that doesn't mean I'm going to give our guest an easy ride. Welcome to the show, Matthew. Thanks for having me. We're at an inflection point in the way the internet gets used. How broken is the content model that has funded it for the last 25 years? Yeah, you know, I think that actually it's been amazing how well that has worked for the last 25 years. The basic interface of the web for the last 25 years has been search. And Google obviously is the dominant search engine out there. And the way that Google works that you all know is if you type something into Google, especially this sort of Google of 10 years ago, you got back what was effectively a treasure map, a bunch of links pointing you to search.

1:22The search wasn't the act of Google returning the links. You then had to continue the search to find what it was that you were looking for. And in the process of doing that, you generated traffic to a bunch of sites on the web. And Google actually, and some other companies, provided the tools that then allowed those content creators, those site owners, turn that traffic into revenue or turn that traffic into ego, just knowing that someone was reading it. But what's happened, and it really started about eight to 10 years ago, it's picked up speed over the course of the last year, is that we're switching from a web where the primary interface is search and search engines, one where the primary interface is AI and what I would call answer engines.

2:06go to chat gpt and you type a question it doesn't give you a list of links to go to it gives you the answer and in fact even google today if you go to it and ask a question increasingly it's putting on the top of the page what they call ai overviews and those ai overviews are essentially answers to whatever question you have and the problem with that is you're not sending traffic then to the rest of the web and so if the if the currency that the web was built on. The currency that funded the web was traffic. The problem is that as we switch from search engines to answer engines, that currency is going away.

2:41And so we've got to figure out some other way to compensate those content creators to make the content, which is what fuels fundamentally the web and even what fuels these AI engines. As you say, it's something that's been going on for a few years. I think Google was early with this idea of the one box, you know, the one box at the top that constructed an answer and delivered what we wanted, which is we wanted the temperature in X or we wanted to know the answer to a simple recipe. And this seemed to be, as you say, the evolution. But there is this change because as people move towards ChatGPT, and I think OpenAI claim 700 to 800 million weekly users of ChatGPT, and I think there's some evidence that for every year that you use ChatGPT, you do 8 % fewer Google searches.

3:31That starts to, as you say, break this relationship. But I'm just curious about whether what we're seeing is the amber warning signal or is it the red light? It's a pretty significant warning signal. The first thing is that the answer isn't to fight progress. Answer engines are a better experience for users. If you think about sci-fi films, which predict a lot of our future, when you ask the helpful, you know, robot, what's the temperature outside? It didn't say, well, here are 10 links that you could follow to figure out what the temperature is outside. It told you what the temperature was, right?

4:03If you ask it for what's the recipe, it told you what the recipe was. That's the interface that makes the most sense for the future. But what that does is it changes the underlying business model of the web. And so what we're seeing from people who are selling media as their primary business is that they're seeing massive drop-offs in the amount of ad revenue that they're getting. They're seeing massive drop-offs in the amount of subscription revenue that they're getting. And as that revenue dries up, it's going to put increasing pressure on content creators. Fewer and fewer people are going to be willing to actually create content if there's not a business model for it.

4:38And so what I think is, you know, there's sort of the nihilistic outcome where we just say, you know, anyone who's creating content like, you know, starts to death and dies like that. That's a horrible, you know, potential outcome. There's sort of the black mirror outcome, which is that, you know, we don't go back to the sort of media of the 1980s, but we go back to the media of the 1400s, which is the Medici's, where there are five powerful families that unless you kind of are under the patronage of one of them, you don't get to do any research. But instead of families, it'd be five big AI companies, where you've got one that's conservative and one that's liberal, and there'll be a Chinese one, and there'll be an Indian one.

5:13Those are incredibly regressive outcomes. I think the better outcome is one in which we say fundamentally some amount that you're paying for chat gpt some amount that you're paying for whoever your ai agent is that's out there as it is giving you value from the content that is on the web some amount of that that fee or that or that revenue that the ai companies are generating that should go back to the content creators and and to me that just seems like common sense. I mean, it's a very appealing notion. It's similar to what we've seen with recorded music. You get the rights agencies collect revenue from all over the shop and they figure out how to apportion it.

5:56I guess one of the things that struck me about what's been happening with this shift to what you call answer engines is that it's unmasked, a fundamental leakiness about the internet anyway. The internet was such that collectively all of the content amounted to much more than adding you know linearly arithmetically any single piece of content and collectively the value that we got from it was greater than the sum of all the individual bits of value that we as users has got from it and from all of that enormous value GDP struggled to to count it we ended up with it being funneled a very small portion of it to Google.

6:39So there was this issue that there's this enormous wealth of richness out there, and maybe 10 % of that value showed up and nearly all of that went to Google. So are we just unmasking a longer standing problem here? If you think about it, though, Google has been really one of the real good guys in the web ecosystem. If Google hadn't created all of the monetization tools that we have, I think the web would be significantly smaller than it is today. And so, yes, they have reaped significant rewards, but nobody has actually given back to the open web at the scale that Google has. The irony, though, is that it's sort of like, you know, it's like a Marvel movie.

7:20Whoever was, you know, the hero before becomes the villain, you know, in the next story. And I think right now, Google is at a real inflection point where they are the ones who are actually saying, we are unwilling to pay for content in this new AI world because we've always gotten it for free. But what's changed, what's changed is that in the past, when Google got the content for free, they gave something back, which was traffic. Now Google is saying we want to get the content still for free, but we're unwilling to actually give traffic back because the you and again, it's not of willingness.

7:53It's actually the interface doesn't inherently give content back. And so the force that really has to change in order to get this to happen is Google has to say the business model of yesterday isn't going to be the business model of the future. Us being the patron of the web is still incredibly important. And we and everyone else who are getting benefit from this for these AI engines that we're creating, we should actually be compensating content creators for that. I'm actually optimistic that one, Google will come around to that perspective, either, you know, willingly or they'll be forced to.

8:26But two, that when they do, that that will actually kind of, you know, be the thing that breaks through the dam and that we will see across the board, AI companies say, yes, we want to do it. Because almost every single AI company that I talk to says, of course we should be paying for content, but it needs to be a level playing field. It needs to be something where if we're paying, Google has to pay too. And so once we get Google to say, yes, we're willing to pay, I actually think that we might actually unlock a huge new business model for the web and for content creators and potentially unlock really a new golden age of content creation.

9:03I mean, Google, of course, is one of the players. And I think Google is probably a better player than some of the other AI companies. I think there's some amazing data about the number of times some of these AI models will crawl a website for the number of humans they actually send, and it's sometimes thousands of crawls to send a single person, which might then be monetized. One of the things that strikes me is that the internet is full of a lot of very mid-content. I mean, it's got some specialist, expert content. It's got some deeply researched material. It's got some very narrow trade information, some very personal information.

9:41But a lot of the traditional media industry, I mean, it's in the word media. It's coming from, you know, medium in the middle. And an answer engine doesn't mean 27 mediocre sort of similar takes of what you or I or the president might have said. It just needs one. So even if this proposal works, it's going to require some changes to the way media owners and content creators think. But I think that what's optimistic to me, if you sort of understand what the incentives are, I think that the incentives of this new model may lead us to an internet that all of us would like better than the internet that we have today.

10:21Those of us who have, you know, a little bit of gray hair, been around around a while, you know, kind of sometimes long for the internet of old, which was more local, more unique, more creative, more quirky. And that, you know, you would stumble across things that were that were, you know, super, super interesting. What gives me hope is it turns out that what AI companies value is more unique, more local, more original, more quirky. And the best example of this is who is it that has struck the best deal with the AI companies? And the answer is Reddit. Reddit and the New York Times, if you add up their total catalogs, have about the same number of tokens in them.

11:00The New York Times has obviously been around a lot longer. Reddit produces more on a daily basis, but it's roughly the same. And yet the publicly available deal that we know that Reddit did versus the publicly available deals that we know that the New York Times has done, Reddit got about seven times more per token than the New York Times. And the question is why? And I think the answer is exactly what you said, which is I love the New York Times. I think they're a great media organization. I love the Wall Street Journal. I think they're a great media organization. And so is the FT. And so is the Washington Post.

11:30And so is, you know, a lot of different newspapers that are out there. but the differences between them are actually relatively de minimis. And you might say, well, you know, the New York Times is more liberal and the Wall Street Journal is more conservative. But it turns out that if I take, you know, OpenAI, ChatGPT and train it on the Wall Street Journal and then say, okay, give me these stories, but from the perspective of a slightly more liberal publication, they'll basically recreate what is largely the New York Times. And so I think you're right that in the future, being just sort of samesies with a little bit of a different perspective is not going to be nearly as strong of a business model as being truly unique.

12:11But that's what I think is really interesting. Like who has struggled the most in the time of sort of traffic being the currency of media? The answer is something like a small local newspaper in a little town. And and yet I think in the future, like my wife and I actually own the local newspaper in Park City, Utah, which is a which is a resort destination. And we know from the newspaper, we do restaurant reviews on what the best restaurants are. Like there's nowhere else that you can go in the world that is going to get you that data on what really is the hot new restaurant in Park City, Utah.

12:47And yet, if you're someone who likes to ski or likes to travel or likes the outdoors and you're thinking about going to Park City, Utah, if you know that the one answer engine out there has the information from the park record, our little local newspaper, whereas another one doesn't, you're gonna be slightly more likely to subscribe to the one that has that information. And so it's that local information that starts to be the real gold that differentiates one answer engine from another. And if you look going forward, I actually think that the AI companies will look more like a Netflix or a YouTube where they're competing for original content, which really matters, than they will like kind of the science projects that we sort of all imagine them to be today.

13:32And I actually think, yes, that will change the media landscape, but I think most people, consumers, most journalists will actually say, wow, I wish that I could do more of those local kind of interesting stories and less of the me too, you know, what's happening in the White House today. I mean, that's a really interesting dynamic that you have painted. And of course, you're right. Your local news and niche news, unless it's industry focused, has really struggled to find a business model since the mid-90s. And I guess that's the story behind Reddit, because if you really want a review of how well a washing machine works after five years, however good wire cutter in the New York Times is going to be, the seven discussions on Reddit will be more real, more visceral, more intimate to people's real experiences.

14:18I suppose what that then also does is it changes the nature of many of the intermediaries on the web who've evolved to take advantage of SEO. A large part of the web has now become, how do we get around Google's algorithms in order to appear on the front page, as opposed to how do we really deliver that insight to a Park City restaurant to the person who's looking for it? My even simpler version of that is that, you know, there are publications out there, sort of the BuzzFeeds and Huffington Post of the world, who saw their job not necessarily as, what do we write in the contents of the article that we write?

15:00It was largely derivative. But they saw their job as how do we write a headline which generates so much rage or generates so much, you know, fear that it will actually get somebody to click on it, like rage bait headlines that were out there. I risk getting on the soapbox. But I think a lot of what's wrong with the world today is that we've just been rage baited into being kind of extreme in a lot of different ways. And I think that that is at some level because Google taught us that traffic was the ultimate deity that we should all be chasing after. And again, Google, I think, is a massive force of good on the Internet.

15:42But Google begets Facebook, which begets TikTok, which feels like we're sort of just spiraling down this attention economy hole that's out there chasing traffic. And yet traffic has never been a good proxy for value. If there's a car accident on the road, you know, it's not like, you know, just because everyone stopped and looked at it means that that's actually a good thing. We don't want car accidents on the road. What we want are actually things that are making humanity better. And what I'm encouraged by is for the first time in human history, we effectively have a mathematical model for the representation of all of human knowledge.

16:24Like you smash Anthropic and chat GPT, you know, OpenAI and Gemini and everything together, and you get a pretty good model for what human knowledge is. Not perfect, but again, pretty darn good. You also then inherently get a pretty good model for where the holes in human knowledge are. And so again, I picture it like a block of Swiss cheese, where there's a lot of cheese that's there, but there's a lot of different holes. Because the business model of the web is going to change. No matter what, it's going to change, because answer engines are coming, and they're better. And as a result, the business model is going to change.

16:59What I hope it changes too, is one in which we reward content creators that create content that fills in the holes in that cheese, where they're actually doing things where they're making things better. And my favorite example of this is, like, I flipped a Stockholm, Sweden, in order to meet with Daniel Eck, the founder of Spotify. Almost no one in the history of humanity has compensated creators that scale the way Daniel has. And you might argue that, you know, it's unfair in some ways or that, but the simple facts are that last year, Spotify sent out$10 billion to people who are creating music, which is more than the entire music industry was worth just 20 years ago.

17:46And so like you go up there and I was sitting with him and we had this really interesting conversation. And one of the things he told me is, you know, when somebody searches for something on Spotify, If you search for like Taylor Swift, shake it off. Like they're really certain that they can give you a result. But if you search for something that they may not have anything about, like I want a song to a disco beat about how fun it is to dance with my cat. Doesn't exist, right? And they know that it doesn't exist. So they try and give you results, but they also know that those results aren't very good.

18:14And so they take those searches that have very bad results and they actually publish it back to music creators. and there are music creators that are making tens of millions of dollars a year creating music for those things that people were searching for. And you might think that that's silly, but when you search for something on Spotify, what you're really searching for is an emotion. You're searching for something that you want some way to connect with. And if the answer is that it doesn't exist, and yet some music creator can take that as signal for something that is created and then be compensated for it, I actually think that that's beautiful.

18:46I think that's filling in the holes in the cheese. And I think that as we think about what the future business model of the web is, we should be thinking about how can we do more of that? Because, again, the world is going to be a much better place if there's less rage bait and if there's more actual content, which is local, unique and actually filling in human knowledge. There's a really interesting idea in there, which starts with the fact that LLMs don't get addicted to dopamine. And so you can you can put this material in there. And one of my experiences of using the different chatbots is that they are incredibly reasonable.

19:22I mean, you can get them to be unreasonable. You can push them in different directions, but they are this aggregate of human knowledge and their training and the reinforcement learning to get them to behave in particular ways. but it's quite hard to get them to be extreme by the time they reach us. Now, there's a question as to whether that will always be the case and whether as the companies get more and more powerful, whether the owners won't start to tune them in one direction or another. But there is also this really important point that you've made, which is we can start to identify more clearly where there are gaps and opportunities.

19:59And one of the things that struck me when I do deep research queries is that Gemini or ChatGPT or Claude will go out and look at 500, 600 websites. And I will have asked a really simple query that needs deep research. And I'm thinking an academic, a human wouldn't have gone to 600 websites to figure out when the bicycle boom in Peoria was and what the impacts were. You'd go to two or three good sources. So there's also this this question that these machines are rather greedy and they're a bit a bit dumb frankly about how they look at the content that's out there so you described the swiss cheese model the sense of filling in the gaps but one of the things to make markets work including information markets is to put in a little bit of scarcity a little bit of the price mechanism that says you have to pay more for this because it's better and this is free because it's it's slop so how do we how do you think about that?

20:54There are no markets without some level of scarcity. There has to be scarcity in order for, we don't have a market for like breathable air, you know, unless you're underwater because there's plenty of it. And there is a market for when you're underwater because there's not plenty of it. And that's the only time that you have markets is there has to be some level of scarcity. And so step one, no matter what the business model ends up being, but step one has to be the people who are content creators say enough is enough. We're not going to let you take our content without you compensating it for us.

21:26And the crazy stat is that in the next few years, the amount of machine traffic on the web will far exceed the amount of human traffic on the web. And there's a real cost to that. Like our customers have to pay Cloudflare in order to do that. You've got to pay AWS in order to serve that traffic. If content creators are not only getting no value, but also like having to pay more to service these machines, like just brass tacks, that's not fair. Someone who is actually benefiting from this should be paying the costs of both creating the content, but then also serving that content back out. And that increasingly is going to be these AI companies that are getting the benefit back to it.

22:07What I think is important is that the answer should not be, you know, that Cloudflare or any single entity determines what is good and what is bad online. What you want is every AI company to have their own algorithm that sort of suggests what it is that is going to be valuable that's out there. And the way that I generally think about it is, you know, how unique is the content and how reputable is the content? And it's a sort of two by two on those things. And the more unique and the more reputable it is based on each different AI company's algorithm, the more valuable it is going to be to them.

22:41And so what I can picture is you could imagine a world in which the AI companies essentially pay for we want to be able to ingest this much content per day. And there's a price that they pay in order to do that. And then you can imagine almost a discoverability service where it takes the algorithm from OpenAI and other things and says, OK, we've now triaged across all the content that was created in the last 24 hours. And here's the the thing you have to have no matter what, and that they will then basically burn down whatever their daily content budget is in order to get that content back and into into their systems.

23:17That's incredibly simplistic. I think that it's going to be some variation of exactly that that defines how this market takes place. You have a scarce resource, you charge for it, and then you help them discover what of that content that was created is the most valuable. And undoubtedly, there will be kind of the SEO of the future that is creating content, which, you know, is exactly what OpenAI wants to think. But if OpenAI has designed their algorithms to say, what we really want is to figure out, you know, what the true carrying capacity of, you know, an unladen swallow is, that's going to be the sort of thing that people will then go out and do research on.

23:59And again, that's filling in the holes in the cheese, and that's advancing in a different way than the article that's like, you know, this one simple trick to, you know, eliminate your belly fat or you can't believe what Donald Trump did today, like whatever it is that's just entirely designed to get people pissed off. If instead we're doing something that actually is answering, you know, queries, whether that's actually factual or creative or humorous or whatever it is, like that is a better world. And that's the one that when the business model changes, which it has to, it will for sure change.

24:35We should all be thinking, how would we like that business model to look going forward? And then how can we design the incentives to get there? There are some models. I mean, financial information is a very good example. You can't get real-time data from the CBOE or the New York Stock Exchange without paying a lot. You can't get Platts oil data. I mean, Michael Bloomberg made an enormous amount of money just charging for 15 minutes of scarcity. Right. He's done pretty well out of all of that. I think the thing that matters for a market like that to work though, is that the seller and the buyer need to make offers and they have to come and agree.

25:12So that mechanism needs to exist for content creators to say, listen, I'm willing to sell this for a dollar or a cent. And they have to probably do that in real time. Now we do have some infrastructure to do that. I think, you know, real-time bidding across ad exchanges, you know, was something that's running at trillions of transactions every minute. We've seen these things in the financial markets as well, but it would be a whole new set of skills and capabilities for content companies, especially smaller ones. You can imagine the New York Times hiring some economists and being able to do this, but you've talked about the local newspaper in Park City, Utah and presumably, you know, Beanbossom, Indiana and everywhere else has got these businesses.