The problem with Altman’s “gentle singularity,” Apple’s AI missteps, and Google’s fading ad model | Live with Azeem Azhar

0:00So the three things I wanted to talk about today. Number one, how do we make sense of Apple's announcements to the Worldwide Developer Conference earlier this week? The second, how should we make sense of increasing data that is showing that Google is referring less traffic to websites, that there's a different consumer behavior emerging as people use, in particular, chat GPT? And the third thing is to reflect on Sam Altman's essay, the gentle singularity.

0:39Now, I just want to give you a sense of how seismic and titanic each of those issues are. Apple is the biggest consumer hardware company in the world. It's incredibly profitable. It is renowned for stunning products. And people are really confused with what happened at Worldwide Developer Conference. Google's ad model has funded the web for 20 years, and there are questions about what happens next. And finally, Sam Altman, who is the CEO of the most consequential company of the moment, OpenAI, has gently said, hey, we're heading towards a singularity of self-replicating machines. And I use the word gently deliberately, he calls it the gentle singularity.

1:25So three massive things. Each one of them would be a huge topic. We're going to do them in about four minutes each. Please be ready for questions. So let's talk about Apple and the Worldwide Developer Conference. So we know what's happened here. Apple doesn't seem to have anything going for it in terms of AI. I have been an Apple customer since 1982. We are looking at a long period of time. You know, when I first got an Apple, Ronald Reagan was in the White House and the Soviet Union was the Soviet Union. And I pretty much owned computers from that company for four decades. And it was incredibly underwhelming to see what came out of Worldwide Developer Conference.

2:09It was underwhelming to see that Apple is really struggling with making sense of AI. And many people quite understandably looked at the recent Apple paper on intelligence and reasoning models where Apple's researchers essentially said, hey, these AI models can't really reason without the PS. By the way, Apple has completely flubbed its efforts in AI. I'm sure at some point they'll get it right. And what came out of the Apple Developer Conference was this new interface called the Liquid Interface, which looks quite odd. Looks quite odd because it looks very unsteve jobs. And I encourage you to do some Googling if you still do that and say, what does the Apple liquid interface look like?

2:52So I was thinking about this and, you know, in a sense, I think what Apple has done is try to think about what's coming next. Of course, they have these deep, deep issues with being wildly behind in AI. You know, the big AI companies say they're months away from artificial general intelligence and Apple's AI amounts to being able to merge a sloth emoji with a light bulb emoji. But what Apple showed in the liquid interface was a vision for what interfaces might look like in a post-iPhone world, in a world where we might have more ambient computing, we might be looking at the world through something mediated by an AI system or a software system.

3:36And that recognition of ambient computing starting to become the modality of use is a really, really important recognition. For those of us who've been using the AI tools, we're probably realizing that there are limits to the phone interface. I have AIs essentially running ambiently all the time through my meetings, through as I go on the drive, I will now increasingly use Claude in its voice mode or chat GPT. And I'm getting really used to having this wonderful assistant just to help me understand what's going on, what I just said, what the other person said, how I bring it all together. it's driving forward my quality of thinking and my host phone experience because i'm barely using any of the phone features at all and often the phone is in my pocket and i'm just got my headphones in that move towards ambience is something that we've already started to see it underpins sam altman acquiring johnny ives company io without for six billion dollars it doesn't have a product And there is a recognition, we can see it coming from two different areas, that computing's factor or interaction paradigms are about to change.

4:59So that paradigm is the following. It's no longer about phones. It's about ambient, invisible computing. And I think one way to read what came out about this WWDC is this recognition that we are moving into a post-phone environment and what would that look like? the next interface you will use will be an intelligent one that listens to you. And we're already starting to see this. I mean, I've been using ChatGPT's voice mode recently, and I actually had tons of fun with it. I recently was discussing something to do with product management, and I said to it, look, I speak a smattering of French and a little bit of Russian, and I understand a bit of Urdu.

5:41could you please respond to me in a melange of about 60 % English and 40 % the other languages in such a way that I would understand what's going on. And I had this fun conversation, kind of serious work conversation with it while I was on the drive. So I think that that's a way to think about the Apple announcements and what looks slightly freakish, which is the liquid interfaces. Let's go to my second point. The second point is the decline of the ad click model. and what that actually means. Now, as a reminder, the internet has been funded by the Google advertising model, which emerged in early 2001, 2002.

6:23Actually, at the time, I was working for an AI company based, in fact, in France, not Paris, but Montpellier, and we were competing with Google. We didn't win that battle, as you may have realized. But across in California, my friend Bill Gross had innovated on an advertising model, the pay-for performance ad click model, which has become the backbone of the internet. What has happened as we start to move to chatbots, for example, the 800 to a billion of us who use ChatGPT is we're going to Google less often. And we are not going through what is now a broken user experience. You go to the Google search.

6:58I mean, let's be honest. I hope people from Google are listening. It's a horrible, horrible, horrible, I don't know if I can fit enough horribles here, horrible experience. And it's really hard to find what you want. And in fact, it's ChatGPT in particular, to some extent, perplexity that is delivering on the Google mission, which is giving you the world's information at your fingertips. And so one of the ways that I look at this is about how AI will become a superpower for the individual. It'll become a superpower for the individual because essentially the AI will sit between us and all of the ad nonsense that exists out there.

7:35I'm not saying that all the advertising is bad, but frankly, most of it, especially think about retargeting, think about AdWord content networks is awful and terrible for the user experience. And I can imagine that the AIs will essentially mediate for us and protect us from that. So what does that actually mean? It means that if you're a brand and you've thought you're going to get customers through buying clicks on the Google's service or elsewhere, it's less likely to happen on the Google web frontier than it is on closed networks like Instagram. And I can imagine consumers being able to access different types of tools that effectively filter out that noise.

8:21And we've seen it already with spam filters. These are built into Gmail and there are applications like Shortwave, which is a Gen AI powered AI client that I use to do an even better job of that, keeping the noise away from me so I can get to the information that I want. And the thing that's fascinating is that already today, 20 plus million people are paying a monthly subscription to ChatGPT, which is basically more people than have ever paid for an internet subscription for information of that type kind of ever in history. So when your screen or your post-screen experience becomes polite, it silences the loudest voices and the loudest voices have been the advertisers paying to pollute us.

9:06Now, you've got to remember that this trend is colliding headfirst with the idea of the ambient computing that we discussed earlier, the WWDC announcements from Apple and Sam Altman buying Johnny Ives' company. And that, I think, does start to pose a major question about what that future model for the internet looks like. I'm not sure we'll get rid of advertising in that sense. I'm sure it will re-emerge, but it will have to reinvent. You're starting to see the emergence of GAEOs, Generative AI Engine Optimization. So I had dinner a few weeks ago in Palo Alto with a couple of founders from an agency that was not an SEO agency, but an agency designed to optimize your content so that it would be well surfaced by generative AI systems.

10:02So you're starting to see the hints of that starting to emerge, and no doubt there will be some kind of commercial model that will emerge there. The third thing I wanted to talk about was the essay that Sam Altman published earlier this week called The Gentle Singularity. What Sam says is that four really key bullet points, and I think we have to take some of this honestly at face value. we can disagree with certain aspects of it for the sake of argument that's taken at face value. So what he says, four things. Number one, AI takeoff has begun, that we have crossed the event horizon where things like GPT-4 and O3 are in many ways smarter than we are ourselves.

10:43Now, I think that is absolutely reasonable. I'd had this discussion with Tyler Cowen a couple of weeks ago. He says, you know, O3 is a smarter, more engaging economist than he is. And I feel that the same, even in my core work. It's one of the reasons I love working with O3, which is that O3 really, really helps drive my thinking, pull me through, because it is just such a great cognitive assistant. And we're starting to see significant advances in discovery, innovation, research type of questions. That idea of takeoff, which is we have tools that are better than us at some of the key things that help us make scientific or technical progress is what Sam's first contention.

11:26The second is that rapid capability progression is what lies ahead. And again, this is something you can see if you measure the benchmark shift. It's something that you hear when you talk to the foundation labs and when you talk to people using AI in more and more sophisticated ways. When I talk to the foundation labs and the developers and I talk to some of my startup founders who are building foundation models or fine-tuning technologies, they all tell me the same sorts of things, which is things feel faster. They don't feel the curve is bending down. It's still bending up. And so Sam's contention is that this notion of, in inverted commas, intelligence is going to be wildly abundant by 2030.

12:08I largely also agree with that. We can argue by what we mean by intelligence, but let's be Potter Stewart for a moment and say, we'll know it when we recognize it. The third contention he makes is that the recursive loops of acceleration are always starting. So if you think back to the idea of the intelligence explosion, Ian Good and all of that from the 50s, it was that, you know, at some point, a system gets smarter and smarter and smarter because you have positive feedback loops. I mean, prosaically, that's what education, university, and institutionalization is, right? So you get a few smart people, you teach them, they build educational institutions, you can teach more people more skills who develop new things push more frontiers and that cycle speeds up and feeds into itself well we're doing that digitally now and you know ai scientists are really really making progress we've seen deep minds alpha evolve sakana out of japan and a whole load of other teams are working on the idea of ai scientists closed-loop systems hypothesis generation, testing, validation, learning from the results, improving the hypothesis and going through that cycle at silicon speed where that's possible.

13:20That level of closed loop, fast virtuous circle can accelerate provided capital flows into that market. So those were the claims that Sam says. And he said that in order to do this, It requires solving the alignment problem and dealing with a kind of collective action issue. So that's a kind of, you know, in a nutshell what he said. I think you should go out and read that essay yourself. Now, let me give you some thoughts on why he says it is gradual or gentle. He says, look, the singularity happens bit by bit, and the merge with the non-singularity world happens slowly. And exponential progress looks vertical when you look up at it.

14:05but it looks extremely flat when you look backwards. And look, I looked at exponential curves for many years, many decades, in fact. He also argues that we adapt very quickly to wonders. I think that's also true. How many of us would go back to GPT-4? And when I'm flying, I have a bunch of GPT-4 level LLMs running locally on my laptop. So if I don't have internet access, I can still continue to co-work with them. It's terrible. I mean, it now feels like you're working with a remedial class because we've got used to things that are so much better than gpt4 and he ends up by claiming that living through this transition will be impressive but manageable so i just want to reflect on the claims that he he makes there and the points where i would agree and where i would depart from it so of course i agree that the exponential progress is real the of course is not because the title of the substack is exponential view or i wrote a book called Exponential.

15:02I mean, the title emerged from the empirical evidence I was looking at, and then from the processes that I discovered and was able to articulate, and that they are real. I mean, and we are at a point where we can feel that exponential progress because it is happening within the loop of human lifespans, and then within the loop of decades, and then years. And that was naturally going to be an emergent property of some of the underlying processes. The other thing I would agree with Sam is that as we go through this process, distribution, availability, absence of concentration of this technology or its capabilities is really, really important.

15:39And I would also agree, it's the back half of my first book, that governance and institutions are really, really important. But places where I would slightly disagree is that the gentleness of this transition is not something that falls out of the bag the way tasty Doritos fall out of a bag of Doritos cool tortilla chips, right? It's not a gift. It's not a natural outcome. In fact, the work needs to be done in order to make that transition feel gentle. and that work involves individual behaviours changing. We can talk about the institutional challenges first. So the first set of institutional challenges is going to be about the way in which governments are able to respond to this in policy terms and in capability terms.

16:32We will dig into in Sunday's newsletter what I think is quite a good policy paper from a right-leaning think tank in the UK around how government ought to think about governance in a world of artificial general intelligence. But a lot of us interact with companies as well. And for those of you who know me quite well, you'll know that I'm incredibly unsympathetic about companies that fail to adapt. At some level, I'm a Schumpeterian at heart, right? I believe in creative destruction. I do think it's an incredibly important part of a business ecosystem and an economy to continue to come up with new ideas, to slough off those things that are now atrophied and unhelpful.

17:14So I don't have a lot of sympathy for the sort of Blackbreeze and blockbusters of the next eras because they have their opportunity and maybe they had their day in the sun. I mean, Bjorn Borg won five Wimbledon titles. He didn't win 55, even though he's a great tennis player. At some point, your moment passes. So I'm not particularly sympathetic to this notion that established firms will fail. It'd be great if they didn't, but only if they don't fail by innovating, by leaning into the new. But so many people are dependent on the success of those established firms and they're ultimately dependent on the capabilities of managers and the directors and the managing directors and the board directors that there is an issue of those types of institutions don't respond well, which I think they might not.

18:01I think the other issue is about how people individually prepare for this. So I've been through my oh shit moment with powerful AI systems. That happened a year or two ago when I was working on the second book. And essentially, the machine could almost read my mind in terms of the ideas I was coming out with. And I was thinking, God, are they actually good ideas or not? And in fact, what happened once I'd sort of slept on it after two sleepless nights was it just pushed my thinking much, much further. And I think people have to get through that and get to the stage where they do feel that these tools have potential to be helpful and to take an optimistic reading of what their possibilities could be.

18:39And I don't think that that is a natural stance. There are cultural differences that we're already seeing, that people in East Asia and South Asia are incredibly optimistic about the potential of AI, and those in Europe in particular, but also in the US, are rather pessimistic about it. And I think the other thing that Sam's timelines may not work, I think they will be very quick, these timelines, and quicker than other things like electricity's rollout, but they won't be instantaneous, because there is a materiality to all of this, because institutions do still play a role because there is a transmission mechanism that takes time.

19:11And that was the thesis of my first book, and I think it's pretty robust and holds here. And I think the other thing that he downplays is the risk that there is some kind of deep, deep backlash. I don't think the deep backlash prevents this wheel spinning in anything but really catastrophic scenarios. But I do think it is unlikely that you can put knowledge back in the bottle once it's out there. So I'll take some questions. And I want to just jump into Jeff. He says he's not sympathetic to Altman because he says some things that I can't say. I actually fundamentally disagree with the notion that this is really about Sam Altman.

19:51Technology systems, and by which I don't mean digital technology systems, I mean technology systems going back to cultivation of wheat 14 ,000 years ago, have a habit of finding themselves in feedback loops. And those feedback loops, by their own necessity, end up being faster and faster. A great example of explanation of this, although it comes from a Marxist political frame, is the idea of time compression from the sociologist and geographer David Harvey. And in there, he talks about the interaction between financial capital, communication, and information technologies, and how that logic is a constant and repeated acceleration.

20:37Because that's effectively what we've done in the world. We've compressed the world as we've increased the bandwidth in fiber optic cables, and we've increased the processing power in chips. And the reason we've increased the processing power in chips has been for very simple reasons connected to, on the one hand, physicists scratching an niche, and on the other hand, the financial incentives. Now, there is a countervailing view that I've heard from an academic in Stanford whose name I've now forgotten, but I had lunch with him a few weeks ago. And Demis Hassabis said something similar in an interview a couple of days ago, which is the really exciting thing about AI could have been AI conceived through really, really narrow tools that tackle first and foremost, the really hard scientific problems of the day.

21:22but Demis's point was that didn't happen and it didn't happen out of design I think it didn't happen by accident essentially we got the transformer model to work really really well on the text that was available which was the text of the internet and those things intersected and provided highly highly performant models in protein folding in chess in video games and other things. But ultimately, this intersection with internet text is the piece that has driven the capabilities of these chatbots. And they happen to be there where they are. People love using these products, and they work really well.

22:04And they work really well even in situations which are kind of non-manipulative. So someone asked a question around Mistral and Macau's recent announcement. So this is the question, right? Is France carrying Europe's AI dreams? Europe is an enormous continent. Its economic weight is really hefty, like, you know, there's a similar scale to the US and China. Of course, there have been announcements this week. Jensen has made some announcements. Mistral is releasing new products. But by the scale of Europe, I don't think we can say that this is being taken as seriously as it's being taken by the US.

22:43Either the economy, well I mean it is the economy now or the Chinese sort of state private sector fusion or the ambition of say the Gulf states and now my take on the Gulf states I think I put it in the newsletter a couple of weeks ago and I gave a speech about it in Brussels a couple of weeks ago is that those Gulf deals with the UAE and with Saudi to put big data centers in were incredible master strokes of geostrategic thinking, both for the US, but in particular for those regions. So they will effectively power a large part of this AI infrastructure, build capabilities, build know-how, build relationships.

23:22And those systems that they power are within a 100 millisecond ping of essentially the whole of Europe, barring the northern part of Scotland, the Faroe Islands, and Iceland, and a little bit of Portugal. And 100 milliseconds is more than good enough to run a chatbot or a large volume of AI applications. So in other words, if the capability and the capacities are there, that could become a data center zone and a place where essentially a lot of AI processing and services could be delivered for the whole of Europe. And that, I think, puts a lot of challenge on European ambitions for digital sovereignty, for European values.

24:02And I think the fundamental issue that Europe faces is if Europeans want whatever that means, but let's say it's some sort of Berlin, Paris, Brussels, and a bit of extra, is if they want to play in this, and if they want European values to persist, they have to stay in the game. They simply have to stay in the game. There is no point dying on your sword in round three of 19 rounds or 27 rounds. You have to be able to compete. And I think the challenge here is that these are all good moves. And there were some good moves in the UK as well with AI supercomputing and some bioinformatics announcements, many of which had been sponsored by a friend of EV, Matt Clifford, who's the Prime Minister's AI advisor.

24:48These are all good and positive and meaningful moves, but I don't think they yet speak of the foundational importance of this technology. So are you seeing any promising hardware for consumers to interact with AI in new ways? And I thought quite a lot about this question, and I'll talk you through a few things that I've seen and I've tried. So I saw a fantastic product actually three years ago, very, very early days, which was a doll called Moxie that had been designed by psychiatrists and psychologists who'd worked with kids with various disorders on the autism spectrum. and Moxie was designed to help them develop empathy skills.

25:35And of course, I entered into this absolutely horrified by this idea because that's what you do, right? And when I had a chance to meet with the team at Moxie and talk to them and look at videos of kids using this device, I was really, really impressed. And then the evidence of impact on the children who were sort of typically five to seven years old was really, really impressive. So that to me was something that opened my eyes and perhaps got me to think a little bit less presumptively. It's very easy to kind of click the down arrow button when you hear something like that. A second thing that I tried that I really thought would be a truly, truly horrible idea, you can see where I start with all of these things, was an EV reader, and forgive me for forgetting who you are, sent me their prototype wearable ambient AI device.

26:25It was a little chain and you'd have it around you and you could get it to listen to your conversations most of the day and then it would help you play things back. And I thought, gosh, this is a kind of horrible idea. And I played around with it. And then I was wrong there. And actually, now that I've got used to having an ambient AI around me and, you know, often walking across the garden or going for my walk, I will be leaving notes and sort of emptying my head into something like this. I realized that you know if you can get around the interface problems and most importantly you can get around some of the the sense of what is private and what isn't private issues it's actually a really nice interface mechanism and I think the way that it would need to work is it would need to work in such a way that whatever's being recorded ambiently is in a way sovereign to me as opposed to being dispatched into some kind of public space I'm able to acknowledge clearly to you that that is the interaction that we're having.

27:24And you're able to say, I don't want this interaction to go into your AI system, which will become an extension of your experience. So I look at things, something like that, and I say, actually, I probably jumped to a conclusion too quickly. So apologies to whoever said that. So let's just go to the last question before I wrap up. So will AI change the dominant model of online services from free with ads to API style paper requests and what would the implications of this be? So this is a big observation, right, that the pay via ads model seems to be coming under pressure. I imagine that there will be some kind of ad-based model that will emerge.

28:02It's just that these models have always emerged in a way because it's kind of useful. There are times where we are happy to be reached by somebody in some way. And I think the thing to remember is that when Google launched pay-per-click, it was a product that was far better for the advertiser and it was far better for the consumer. It was much better than banner ads and interstitials, which slowed down the user experience, were often completely irrelevant. I sold and ran my first banner ads in 1996, six and ad agencies used to fax over the ad for me to put into the website, which then in hilarity ensued as I tried to reconstruct it in Photoshop.

28:48But Google cost for click advertising was absolutely superb as an interaction mechanism. It just worked for the consumer. And obviously, it worked for advertisers because people flooded to it. But at some point in the past decade, maybe it was a staff change. Google broke that system. You know, they just have made the experience really difficult to work with. And of course, ultimately, people gained the system. That's where SEO became so important and search engine marketing became so important. So I think we are ripe for something new. And I'm sure there will be an advertising model that will start to emerge.

29:25But I certainly, and I'm sure that's the case with many of you, have enjoyed having an experience through perplexity or chat GPT or cord that has less advertising. And I'm very happy for the, well, three,$400 a month now that I spend across these products. So we're coming to the top of the hour. Thank you so much, 212 of you who joined. Have a great weekend and we will be back.