Episode 4 - Ian Hogarth: How Powerful Could Artificial Intelligence Become?

0:00Even in 2021, when Anthropic raised their first round of funding, people whose job it is to invest in technology startups all basically didn't want to invest in that company. Technology surprises us. Ian Hogarth, welcome to the Amol Rajan Connection. Thanks for having me. Is there an AI bubble? There's a real chance of a sort of a very extreme market crash, like deeper than 2008. Can you just give a sense of the rate at which things are coming out of these AI labs? As fast as people can mobilize talent and capital and compute to move faster.

0:29Amol Rajan: You very nobly gave up shares in Anthropoc, I'm guessing, would cost you quite a lot of money. Is that just part of what comes with public service? It was a very expensive decision. I've got a mortgage, right? You can't really have someone in charge of potentially with influence over regulation also holding stakes in one of the biggest companies that could be regulated, right? It just doesn't add up. I don't think the British public would be okay with that and I wouldn't be either. Okay, I promise you this is going to be one of the most useful conversations about technology that you've heard this year.

0:59Amol Rajan: I've just had it with Ian Hogarth. We've just spent a couple of hours together and I learned more from it than I think I learned from any conversation I've had in the year 2026. We cover everything. There's artificial intelligence, there's other revolutionary technologies like robotics, quantum computing, and nuclear fusion. He is the guy, remember, who a British Prime Minister turned to when he wanted Britain to be a world leader in AI safety. So we talk about the big subjects, the existential risks, and we talk about the exciting future that technology could deliver us as well, if we get it right.

1:28Amol Rajan: I learned so much from this, and I think you're going to love it. Just before we get into the conversation with Ian, I want to say a huge and heartfelt thanks to our launch partners. They make all of this possible, and we could not do it without them. So thank you to Waitrose, thank you to Amazon, thank you to Google Cloud, and thank you to NatWest for your support for the Amol Rajan Connection. Ian Hogarth, welcome to the Amol Rajan Connection. Thanks for having me. We're going to get straight into it. We have had so much interest in you and in this subject from our amazing community of viewers and listeners.

2:00Amol Rajan: And we're not going to do any niceties. We're going to get straight into the fact that people are concerned about technology. You've got a couple of very different professional hats on. You think a lot about AI safety. You're chair of the AI SI in the UK, but you're also a hugely influential investor in tech. We're going to talk about both those things. AI first. Do we need to slow down the race for ever better AI? I think what's challenging is that there's such different paths forward for the technology. I think there's a scenario where we have systems that are able to sort of recursively self-improve themselves.

2:34I personally, in that situation, think it'd be incredibly warranted to figure out some way to develop the technology more carefully because we'll go through such a hyper acceleration of the capabilities. I think there are other scenarios where you know we end up on a more a trajectory where the technology looks a bit more like sort of a souped up version of electricity very very important but at the same time you know a bit more manageable in terms of the sort of the pace at which it's developed so I think it depends a little bit on which of these branches we turn out to be on and there's uncertainty around that.

3:11Amol Rajan: So just to go with the mega existential as a species there is a sense in which we are at this kind of fork of the road isn't there there is a kind of we have unleashed something in the world as Demis Hassabis who you know extremely well puts it we have made sand think and we need to sort of decide what's going to happen next we've got some big decisions to make right yeah I think so and I think that the biggest challenge is is the level of uncertainty you know like this idea of recursive self-improvement there are people building the technology who don't think it's possible there are people building the technology think it's almost certain and that's a very extreme place to be in where we're talking about such completely different worlds depending on which of those people is correct um and so i i think my view on it is i mean first of all i think it's we should have a much larger public conversation about it what's challenging is the sort of rate of um technological progress i think is kind of outpacing the conversation a bit and i think you can that i think that is driving a lot of the like fairly deep unease you see out there about the pace of development you've used a couple of times the phrase recursive self-improvement.

4:14Amol Rajan: What does that mean? So there's sort of different degrees of it, but you can think of a kind of very simple version of it as in the same way that AI is getting good at writing software and AI is good at developing mathematical, kind of proving mathematical conjectures. If you put those sort of two skills together, you could have an AI that is good at discovering algorithmic improvements to allow itself to be smarter. but where it's contributing to itself becoming smarter. So there's a sort of, it's sort of the classic sort of sorcerer's apprentice, the AI sort of... Genie out of the bottle stuff, right?

4:51Amol Rajan: So you kind of unleash this thing and then it gets better and better and better and you can't control it. Yeah, but we have a level of that that's already been happening. So for example, AIs are now being used to design faster and smarter chips, which are the chips that the AI itself is running on. So it makes the whole system more efficient. So there is this kind of, there is already some degree of a kind of recursive self-improvement just embedded in capitalism, embedded in the development technology. Everything builds on what came before. But the thing that I think people really talk about when they talk about cursive self-improvement is this idea that you have an AI in a data center where humans can kind of step back from the data center and the intelligence in the data center just grows exponentially.

5:30That's the thing that people really worry about

5:33Amol Rajan: when they use that phrase. Okay, that's incredibly useful. I feel no shame at all in asking simple questions about sort of what might seem to people obvious things. One of the things I want to do with this world is make complex ideas accessible. What is AI? AI in many ways is kind of like a sort of an approach to engineering where you are teaching a computer to recognize patterns and learn concepts. And it's sort of somewhat inspired by neuroscience and understanding how the brain works. Neural networks. Neural networks is, you know, sort of deep learning and sort of the ability to recognize patterns by showing lots of examples.

6:13And then reinforcement learning, which is sort of a little bit like the brain's sort of dopamine system where you kind of have rewards for achieving various things where you sort of, if you do make the right decision, then, you know, you get a reward and the system learns to sort of operate better in an environment like a computer game, for example.

6:32Amol Rajan: Okay, so AI is the sort of the science of teaching machines to learn for themselves. how does that make AI fundamentally different to what we've just known as computing which is driven in part by code but also by algorithms yeah algorithms are computerized decisions where does computing become AI so I think that in many ways they're both software right they're you know the ultimate people are sitting down and writing and writing software I think the word you used in there was sort of science and what's interesting about AI is it's almost slightly pre-scientific in the sense that we don't fully understand why it works.

7:09So, you know, a phrase that people in the industry often use is that rather than, you know, designing these systems, we're growing them, you know, we're sort of showing the systems, you know, ever larger amounts of data, giving them ever larger amounts of computing resource, more complex environments to explore. And in the process, they become more capable, but in ways that like, you know, prior to the experiment you couldn't predict. And so there's an odd thing here where it's largely a sort of figure out what works as we go, rather than having kind of a sort of solid scientific foundation to be able to predict exactly what is going to happen and exactly how it works.

7:45And part of the challenge there is that we don't really understand how intelligence works. So it's a large, complex problem. So AI, I've heard Demis Hassabas talk about this.

7:55Amol Rajan: He says that AI used to be most of the domain of what we would call expert or logic systems, which were things that people might have heard of something called Deep Blue, which was this chess computer that beat Garry Kasparov in the mid-90s. Something very different has happened over the last 10 to 15 years, which is machine learning. Can you just give us a sense of how machine learning of artificial intelligence has radically accelerated over the last 10 to 15 years? Yeah, I mean, if I put it in the context of my own kind of experience of it, I studied in machine learning at university. And you know, back then - This is early noughties, right?

8:30Early nought, 2004. And my professor, this wonderful guy called Roberto Chipola, he said, look, you know, we're next to a really great hospital in Addenbrookes. And, you know, I've gotten hold of 200 labeled images of breast cancer biopsy scans. See if you can figure out how to make a machine that can learn to sort of identify patterns in those scans and figure out what is cancerous and what is not cancerous. and so I built a little system that essentially you'd show these labeled examples and it would figure out how to discriminate between the two different kinds of things sort of by itself and it's a very it's a very sort of magical feeling when you first do that because you've seen this computer sort of figure out things and figure out structures in the images that you maybe hadn't realized were useful and you know I'm doing all of this as not as a radiologist right and so what's happened since then is just the there's been an astronomic increase in the amount of computing resource that we give the systems.

9:25It's just gone up like a truly staggering amount the last two decades. And the other thing that's happened is we've found a way to use a huge amount more data. And the big thing we figured out was how to use the internet. So we figured out how to use the internet. That's sort of the fundamental insight behind the large language models scaling and starting to really work, things like ChatGPT, because they were trained on the internet. And then the thing we've gone on to do since then is figure out how to sort of efficiently let them learn other disciplines. You know, like, for example, progress in mathematics, progress in writing software, progress in various other scientific domains.

10:02Amol Rajan: And for people to understand the speed of what's happening, the rate of acceleration here, in the 20th century, we had this idea of Moore's Law. Moore's Law comes from a guy called Gordon Moore. He was one of the founders of Intel. Mid-20th century, Moore's Law was that the number of transistors on a microchip, which sort of translates to computing power, doubles every two years while the cost per transistor decreases. That was a kind of exponential change. It was sort of Moore's Law was invented to make people get excited about kind of in the 70s, 80s and 90s, the rate of change. Moore's Law is completely redundant now, isn't it?

10:34Amol Rajan: We're not talking about doubling every few years. We're talking about, you know, extraordinary exponential curves. Yeah, I think that like some of the people who made the best predictions about when AI was going to become really capable basically did it by looking at Moore's Law. So people like Shane Legg, who was one of the co-founders of DeepMind, or Ray Kurzweil, they basically looked at the amount of computing resource that would be in the world every year and sort of started to look at when we would have the kind of amount of compute to be able to, for example, be comparable to the amount of computing resource we believe to be in the brain.

11:08And they used that to sort of infer when AI systems could be kind of comparable to humans in some way. And so in some ways, it's a sort of deep continuation of a trend that's been running for many, many decades. You know, if you look at the total amount of kind of computing resource in the world, it sort of had a pretty predictable and constant trend for decades now. The big difference is now we've sort of introduced a new kind of computer in the GPU or computing resource, and that is growing faster. But the total amount of compute available to humans is still probably the most important thing going on.

11:51Amol Rajan: Okay. Just to strip back to the other fundamental questions which people will have, and you're the man to explain it. Why is AI different to every other technology hitherto invented? We don't know if it is, right? So I think this gets this uncertainty about what exactly we are dealing with here. um you know i think there are technologies that we whether it's the wheel or fire or electricity or writing agriculture like it could be that you know that kind of a technology what's sort of potentially different for it is the idea that we can we we've we create something that is kind of significantly smarter than humans in all dimensions as you've got something that's sort of strictly more capable than humans on all dimensions in the same way that there are you know there we are strictly more capable than some animals on different dimensions.

12:37Amol Rajan: When other people have been asked that question, the way they've answered in the affirmative, i.e. AI is different to every other technology, I've heard and looked closely, they've said it's kind of three things. One is it makes its own decisions, which, you know, every other technology from the fork to nuclear power doesn't. And secondly, it comes up with new ideas. It can come up with new ideas. And the third thing, which is kind of related really, is that it's generative. And that one of the big things that's changed over the last 10 to 15 years is that artificial intelligence or machine learning has gone from being very good at classification, video, text, you know, what is this thing before me to generating ideas and potentially generating new knowledge.

13:15Amol Rajan: Isn't that an important distinction with other technologies? I think it's an distinction, but there's always a distinction between technologies. And so I think it's something about the sort of the magnitude of the impact on us. And I think that mostly can be reduced to how intelligent does the system end up being relative to humans. Is it smarter than humans in some ways, but not in other ways? Is it sort of roughly the same as us, kind of very similar to us? Or is it like a whole new class of intelligence that's, you know, there's this phrase super intelligence, which is the idea you have a system that is smarter than us at basically everything from science to engineering to, you know, literature, whatever it chooses to set its attention to.

13:57Amol Rajan: The future of food is an issue very close to my heart. If we get it right, it could unlock so many benefits for us as a society. That is one of the reasons why I wanted Waitrose to be a launch partner. Waitrose was the first UK supermarket to commit to regenerative practices. Since then, they've been working to support over 1 ,500 of their UK farms to transition to nature-friendly farming. They're also the only UK supermarket with their own farm, the Leckford Estate, where they've been farming regeneratively since 2020. More in the link below and keep an eye out in store for their new label on products that champion these practices.

14:36Amol Rajan: You wrote a very influential piece a couple of years ago in the Financial Times, which kind of went viral, which not many behind paywall pieces do. And it said, I think the headline was, and I read it closely, we must halt the march to God-like AI, artificial general intelligence. What is artificial general intelligence? um so you know the there are like varying definitions but i think probably the one that the leaders of the sort of people developing this would would probably use this sort of the idea of a system that can do anything that a human can do at human level so you know it can do physics as well as alb einstein it can um you know write sonnets as well as shakespeare like that's the sort of the idea it matches human performance across all domains in a sort of general way And this is cognitive performance, which is the capacity to solve problems.

15:25Amol Rajan: That's what intelligence is. Yeah, but it's also, you know, I think it probably ultimately extends, you know, you could have a sort of purely, purely kind of cognitive version of it, but you could have a version of it, which is, you know, says, you know, you've got to better take, you know, free kicks as well as the best footballer. right so there's a version of it which expands to like you know obviously you know the world's best sportsmen are incredibly people incredibly intelligent right the the things that their brain is computing when they're thinking about like a you know landing a free kick from however far away with the wind blowing and this person moving it's a similar like extraordinarily complex calculation happening that person's head how does an ai take a free kick well it can't right now uh but you know if you had a the question is like when when could you get to the point where like a robot could bend it like Beckham, right?

16:10That would be the sort of the equivalent comparable there. And that's when robots get good enough and the AI system is good enough to like do exquisitely precise physical actions.

16:21Amol Rajan: Why did you argue before others did, you know, because lots of other people have come to your position in a very public way, including people like Dario Amadei, who's the boss of Anthropic. Why did you argue a couple of years ago that we must halt this march? Well, I didn't say halt, I said slow down. Slow down. And I said race, because I think that to me, the structural challenge is that we've got a potentially sort of catastrophically significant technology that we are developing in a kind of all-out race dynamic and the races between companies and the races between countries and that creates big challenges in terms of doing it safely and so I think this kind of this deep question of where exactly does the technology cap out and the fact that we don't really have a scientific foundation for the technology made me argue that there would be a more careful way to do it that reduced the risk of a big mishap.

17:11And the thing that, you know, for example, could happen is we have a kind of Chernobyl-like event that could have been avoided if we had done it in less of a sort of breakneck race dynamic between these different players.

17:21Amol Rajan: To be clear, people from Demis Hassabis to Dario Amadei to Elon Musk, Sam Altman, there's been this kind of outbreak recently when Dario Amadei wrote an essay, which he often does, saying we must pace the frontier, in other words, slow down, basically making a similar argument. to what you've just made and did make in the Financial Times, there was this kind of unlikely outbreak of consensus where lots of other people got behind him and said, we should do this. So there's people at the frontier who are saying, slow it down. And I just want to turn from that to what is actually going on. Because there are lots of people who have a kind of, you know, weak relationship.

17:55Amol Rajan: I don't mean weak, it's immorally weak, but kind of an occasional relationship with the news, and they just can't keep up. And you know, there's other people who have a deep relationship with the news who are trying to keep up, and they struggle, people like me. Derek Thompson is one of my favorite writers on Substack. He wrote a piece a few weeks ago where he described what happened in two weeks in July in the field of AI. A Chinese company called Moonshot AI released a model called Kimi 3, which is an incredible open weight model. A few days later, OpenAI acknowledged that one of its AI agents had escaped a testing sandbox hacked into another company called Hugging Face.

18:23Amol Rajan: The next day, Alphabet disclosed that its quarterly free cash flow had turned negative for the first time in the public company's history. A few days later meta announces earnings its stock plunged 10 a few days later one of the most famous ai funds in the world crashed out and sold off the bulk of its portfolio after south korea's ai fueled stock boom abruptly imploded can you just can you put with your lens and your experience give our audience a sense of the rate at which things are coming out of these ai labs the rate of development i mean i think the best model for it is it's it's being developed at the fastest pace possible like how fast is that like as fast as people can mobilize talent and come and capital and compute to move faster uh and you know like the leading companies are all working on this concept of recursive self-improvement where they're trying to devote larger a percentage of their resources to making the system able to improve itself so like when i wrote that essay i mean it's like we've literally had zero deceleration it's been about as aggressively accelerated as it possibly could have been since I wrote that essay.

19:30You need to write another essay. But my point is that there's a very, very, very deep, powerful force is driving this. I mean, it's basically, you know, AI has the potential to create extraordinary wealth. And I don't just mean that in the sense of like wealthy individuals, but wealth for society, you know, like huge scientific progress can be unlocked from this. And at the same time, you know, it can be transformational in terms of military. I wrote an essay in 2018 called AI nationalism that basically explained how this was going to have big implications for nation states. Really? Yeah. You know, if you look at what's happening in Ukraine right now, it's basically an AI enabled war, right?

20:03Like people are using AI and drones and swarms of drones to kind of, you know, try to outwit their adversary. The way that AI can be used in signal processing to understand, to be able to read signals that you can't read, that we haven't been able to read previously. Cyber attacks, you know, the mythos stuff, the ability to perpetrate more complex cyber attacks or have more sophisticated cyber defences. That's all kind of being enabled. And so you have a very, very complex dynamic where it's sort of in the interests of governments, especially large rival powers, to develop this as quickly as possible to kind of get ahead of the other.

20:40And then it's also, there's a huge kind of capitalist incentive to develop this as quickly as possible. So those forces together are producing like sort of max acceleration for the past few years.

20:52Amol Rajan: So those forces and the geopolitical implications and particularly how Europe and Britain doesn't get left behind. It's one of your great interests, actually. And I know that informs how you think about technology broadly and also how you invest. We're going to come to that. Can I just ask you about AI and great wealth? One of the concerns that people have is that AI would create a kind of new oligarchy. If you look at a few very, very wealthy people, a lot of these companies aren't yet public, what you're creating is a radically accelerated inequality. And there are a few different components to that.

21:21Amol Rajan: And I just wonder what you think. The first is that while these companies aren't public, you create a huge amount of wealth for a few people. So you, if you like, privatize the gains. But if there is a kind of crash of any kind, the implications will be nationalized and be felt by the public. How concerned are you about that? About A, an extreme concentration of wealth and B, the fact that the risk will lie not with those rich people, but with all of us? I think that like concentration of wealth is kind of embedded a little bit in in startups and technology in that it gives great leverage to individuals.

21:57And so there's something about individuals being able to use technology that just I think is kind of amplifying inequality. And we have a big challenge about what to do with that. But there's a really big positive, which is it's also, you know, if somebody invents an extraordinary vaccine that cures cancer and they become a billionaire on the back of that. but we all get cancer cured, it seems like a pretty good trade to me. It's worthwhile. People would argue that you wouldn't have the extraordinary technologies we have

22:22Amol Rajan: if you didn't have billionaires. That billionaires are, in a sense, almost, it's one argument, they're almost a price worth paying in order to create the incentives for people to pursue things. I think that the system we're currently operating in, that's sort of the trade. And, you know, like, I'm very, like, clear-eyed about the fact that, like, when I'm helping a startup founder working on a new technology, Like if I'm successful at helping them build the most important company in their category, you know, sort of baked into that as they're going to become a billionaire in the process. Right.

22:49So it's a it's a kind of it is a I think it. So I think this kind of fundamentally there is a like a there is a level of wealth inequality kind of baked into how technology has been developed and startups in general. With AI, I think that the thing that is sort of more pernicious is the fact that there's potentially a more serious rich get richer, the powerful get more powerful effect that can kick in here. Where, you know, if you have the resources to train an incredibly powerful and useful model, and you generate huge amounts of revenue from that, you then have the ability, you then have the sort of the financial position to create even larger computing cluster, to train an even more powerful model, to make even more money, to make an even more powerful computing cluster.

23:32and it's sort of harder for like a small company to ever catch up to that. So I think if there's a sort of an embedded risk in the way we're doing this, where you almost get these kind of black hole-like companies that just start sucking everything in the economy towards them. But that's very much baked into a version of the world where you have something closer to recursive self-improvement being possible. And there's so many other things that we should be worried about with that beyond wealth inequality. But I think that, you know, concentration of power is another way to look at this. And I think there are huge concentration of power risks embedded in how we're currently developing AI.

24:07And so I think there's a very complex policy challenge, which is, you know, if you're the, you know, let's say the British Prime Minister or the German Chancellor, you've got this kind of simultaneous challenge of these incredibly powerful systems are being built in primarily in California and in China, you know, that could significantly impact. impact uh you know life in your country but at the same time and you know you there's there's some big governance challenges of how we navigate that and what's you know what should the sort of international cooperation be around managing these risks and at the same time you're kind of sucked into the race yourself because if you don't um participate then you know you could be left behind in the scenario where this is like a sort of a bit closer to an electricity like technology yeah so that that's why i think that like almost the the the key question is what is what is the ultimate trajectory of the technology, is this recursive self-improvement or is it something that sort of stabilizes in a way that is closer to something like electricity where the benefits can be much more easily diffused?

25:07Amol Rajan: But if we don't know, the question is how do we proceed? And by the way, it's worth saying, I want to say at this early stage, you've given over 20 years of your life to the good that technology can do. And there's a lot of other technologies, including quantum, which I want to get onto, in which you've invested, in which you, I think, hold out great hope for. And, you know, I'm very keen, not least with this show, to promote the idea that technology can and often is a force for good. But just on the concerns before we move to the positive case and the other tech, you talked about the fact there's a race dynamic.

25:37Amol Rajan: What does a better dynamic look like? CERN, global cooperation, multilateral institutions? I think it's sort of, it depends a lot, again, on what branch we're on. Are we on the sort of recursive self-improvement branch? Some people like to call the other branch like normal technology or like electricity. I'm using that as a sort of shorthand. I think if we're on an electricity-like path, then I think it's going to be in some ways a lot more straightforward. There will be less extreme concentration of power to navigate probably in that version of the world. I think that there'll be exceedingly large risks to manage, like around, for example, cyber risk or uplift to people wanting to do bad things with biology, for example.

26:22But I think it will look a bit like managing power grids and managing the risk of electrocution. And there will be a lot of things that we will innovate and invent as a society to navigate doing it well and safely.

26:38Amol Rajan: A word about Amazon, one of the proud partners of this show. You know, around a million young people in Britain are not in work or education, and solving that is a real passion of mine. Amazon do a lot here, but there is one programme that really inspires me. Supported internships, training for young people with learning disabilities and autism, where 95 % are unemployed despite most wanting to work. Amazon give them real jobs with a coach beside them the whole way and more than 8 in 10 go on to find permanent work. Search online for more. How do you fit into this? Because it is a remarkable thing that when a British Prime Minister decided that Britain should be a leader in global AI safety, they set up this thing called the AI Safety Institute, and you are the chair.

27:30Amol Rajan: that's a big weight it says something about your not just intellectual caliber but your experience was it rishi sunak that asked you what is your role there well i think i mean my role is really to to kind of to to kind of set something up so i guess i i kind of ultimately i think my role is like co-founder of of the thing um and the thing that i believe um not everyone in technology believes this but like i just believe the state has a really important role to play um not just the nation state but also like the international community of states and i believe that to do that job the state needs to have capability of its own to evaluate these systems and so i helped to recruit kind of a hundred or so you know fantastic ai researchers into government paired them with experts in national security and things like you know cyber chemical and biological weapons and we have basically taken responsibility for trying to understand the risks embedded in these systems.

28:30So not just relying on companies to tell us what the risks are but trying to assess them on our own behalf and then briefing senior decision makers in the UK government and beyond on what we find and working with other governments to figure out how to sort of collectively work to manage those risks and working with the companies. A number of the things we found, the companies have then changed things about their systems to make them safer as a result of what we've discovered. So it's a sort of prototype, I would say, for states who are ultimately responsible for national security, contributing to managing the risks and the technology.

29:05And so it's a slightly unusual thing and there's not something that similar to it out there. And I'm really proud that we've done it quickly. Like we've built this in less than three years from a standing start. And I think it is having a pretty big impact. Like we've tested something like a thousand evaluations, over 50 of these most cutting edge models across a whole range of capabilities. And I think there have been a few specific things where I think we've really moved the needle in terms of understanding.

29:31Amol Rajan: Let me say two other things, which I know you're too humble to say. The first is that Britain is held out to be a world leader in AI safety. And that's because of your institute you're a chair of. And I know that not just talking to people in this country, but elsewhere. There's a feeling that this is something that Britain's done well. and the thing that Britain's done well is the thing that you founded. So congratulations on that. The second is that you very nobly gave up shares in Anthropica, I think it was, which I'm guessing would have cost you quite a lot of money. Many, many, many millions, perhaps more than that.

30:03Amol Rajan: And you did that for sort of noble reasons. Was it necessary? I mean, do you have to do that sort of thing if you're going to do your job? I mean, is that just part of what comes with public service? Yeah, I mean, I think that like... It's very decent of you. Yeah. I mean, look, I think it's a very expensive decision. Like, you know, I've got a mortgage, right? So it's like, it's not like, you know, it's sort of... To be clear, for people who don't know about investing, what this means is you, because you're ahead of the game on these things, you were an early-ish investor in Anthropic. Anthropic is soon to have an IPO, initial public offering, which will value it maybe around$2 trillion.

30:41Amol Rajan: So the value of your shares, which you had to forfeit at a particular point, have grown very, very exponentially. And therefore you've potentially given up millions and millions, huge amounts. It's a very expensive decision. Was it the right and necessary one? Yeah, I think that like, if you rewind back, like I invested in the first round of Anthropic as an angel, as an individual, because I was really worried about these safety challenges that I could see coming down the pipe. And I thought that both Anthropic could be an extraordinarily large company, but also if they were to become a really large company, we'd have really big safety challenges to navigate.

31:14And so, it is an extraordinary amount of money that I gave up. But I think that you were a British person during the pandemic, when you saw some of these kind of examples of graft, it's just kind of, it's a bit grotesque, right? And I think if I'm gonna be in a position where I have influence over, at that time there was a question of whether AC would be a regulator. It's not a regulator, it's just a technical body, but that was uncertain what it was going to become. And if you can't really have someone in charge of, potentially with influence over regulation also holding stakes in one of the biggest companies that could be regulated right just doesn't add up I don't think that like the British public would be would be okay with that and I wouldn't be either and so I don't know if there was another option like you know there is a there's a thing people do where they put the shares in a blind trust but like you sort of know what's in the blind trust a bit you know what I mean it's not necessarily I don't know I it felt like the right thing to do to me and I think what's hard is that like if the if if the cost of this kind of public service is like 10 million dollars then not that many people would would make that choice and so there is a fundamental challenge in in i think figuring out how to straddle expertise that comes from the private sector and wanting to contribute to the private public sector but i just i really believe in our country and i wanted to contribute and i felt this really important and it felt it it felt worth worth the the sacrifice and And I also kind of knew that if I didn't do it, it wasn't going to happen, you know?

32:42So I just, I felt like there've been so many incredible people who've built AC and have contributed huge amounts. I think that like someone's got to be the first bit of snow to start rolling down the mountain, you know? And I decided to be the first bit of snow.

32:55Amol Rajan: So let me just say it. I think that was noble. I think that was a genuinely noble thing to do. And I think that was enormously admirable because I think you're being humble about how much it's cost you as well. Two other questions about AI and we'll talk about the broader picture. The single biggest thing that I want from this show, your Moorajan connection, which you've very kindly been an early guest on, is I want to answer some people's anxiety about what's coming. There is this extraordinary, and you're sticking around for our listeners episode. We've had unbelievable amounts of questions for you, which is partly about your stature, partly about the anxiety people feel and their hopes.

33:27Amol Rajan: People feel overwhelmed by the speed and scale of change, and they're not in the world of recursive self-improvement. They want to know how this could work for them. And I want to help them with that. And let me be completely transparent. Two of our launch partners are tech companies, Amazon and Google Cloud. I use their products enormously, did before I was in business with them. And those products have made my life a lot better in the case of business, Google Cloud in particular. How should people listening to this now take advantage of AI to improve their lives? I think there are things you can use AI for that, you know, when I want to, like I love investing and helping people working on, helping to build companies at the frontiers of technology.

34:07And one thing that is sort of staggering to me is how quickly I can learn about a new topic. You know, the ability to basically almost have like an army of researchers that will go away and do research for you on a topic is just, it feels like a superpower, right? And I think that superpower is coming to like many professions. And so the idea that kind of, you know, just as one example, research kind of collapses in cost to the point where I can just sit there all day long trying to get smarter about long duration batteries, for example. It's an amazing, amazing thing to have available to you.

34:40And I think that in the same way that like chess players have gotten better because they've had access to chess computers, right? There's this kind of amazing opportunity to kind of learn faster than I think we had before. But I also, you know, like to me, again, if you sort of go back to the example of where I started out in 2004 working on these breast cancer biopsy image processing, like it should be possible to significantly reduce the cost of healthcare um it should be possible to reduce um because or kind of increase the productivity at which we understand and treat and diagnose and cure diseases so i think that that's the kind of the that's the reason one should be really one should preserve optimism and i i think that although i'm there are aspects of what we're doing right now which which frankly terrify me i also can see all this upside and So there's just this extremely complex thing in navigating through and finding our way through to kind of getting the benefits of the technology without mitigating the sort of the worst consequences.

35:39Amol Rajan: Just before we get to the upside, AI and beyond, other people here who I'm just trying to tune into kind of their relationship with the news will have heard a lot of talk about an AI bubble. They may or may not be invested in the AI bubble. You are a serial and very successful investor. You're also one of the foremost brains in this country and beyond on AI. I've heard a thousand different answers to this. I might as well just ask you, Baldy, is there an AI bubble? Yeah, quite possibly. But it's sort of like a strange, there's different reasons why there might be a bubble. There's different types of bubbles.

36:12Different types of bubbles. But I think there's a real chance of a sort of a very extreme market crash, like deeper than 2008 sort of scale of crash. Really? What do you mean? as in just like the sort of the scale of the financial crisis that could be triggered by something going, you know, something by AI kind of having a sort of significant financial shock. And I'll give you a few different examples of how this could happen. So the first and most kind of the most well-discussed one is that sort of the profits in AI don't sort of shake out in the way that people predict. So, for example, the capabilities that we really need from these models at the moment turn out to not be kind of Einstein level capabilities, but we just need a sort of, you know, smart accountant level capability or smart lawyer level capability or smart programmer level capability.

37:08And if that is available from a sort of, you know, an ever larger set of companies, ever larger set of open source providers, you could have a sort of commodification of the technology that makes it harder for some of the very large companies to make a lot of money. And in that world, you know, those companies, if they've taken out, if they've made extreme commitments to the future in terms of the size of data centers they're going to, they've committed to, they could find themselves in a position where they're unable to pay their bills because the revenue they're generating hasn't kept pace with the financial obligations they've taken on.

37:42Amol Rajan: Huge debts. So, you know, tens of billions in debts, often in bonds, which I mean, we're talking like hundreds of billions. Right. And so for specific companies, overall, it's actually in the trillions. Yeah, exactly. Exactly. So there's just this kind of, that's the technology sort of plays out differently to how people are making very large leverage decisions on. Another way it could play out badly is we could have a really large catastrophic event that involves, you know, involves the world collectively deciding to pull a handbrake on the technology. Right. It's sort of Chernobyl like event where there's just a kind of recognition.

38:16Hang on a second. We can't keep doing it like we're doing it. In which case, again, you can have a kind of a crash, but it's different. It's not a crash because the technology is not powerful. It's because it's almost too powerful, right? So that could be something to do with the financial system, something to do with a big cybersecurity incident. In some ways, it's all priced only for upside right now and only for a particular version of upside. So I don't think it's at all naive to be worrying about that. And this is what is so extreme about the sort of times we're in. in the, you know, if you're thinking about you're a policymaker trying to decide what to do, you've got to think about the recursive self-improvement risk.

38:51You know, it might be small, but I don't think it's zero. You've got to think about the sort of normal technology electricity version of the world where you want to really maximize the benefit of this thing. And then you've got to worry about a massive financial crash on some trajectories. So I think the only way to manage as a head of state is to basically have a portfolio of strategies that map the different scenarios playing out, right? But that's really difficult. It requires, I think like a very high level of sophistication to get the right strategy facing this thing.

39:19Amol Rajan: Well, can I put to you as someone who's obsessed with the question of debt is going to talk about it a lot on this show and takes a close interest in British debt. And in fact, what you might call rich world debt, that the big difference between the great financial crash of 2007 and eight and now is how indebted governments are. So back then governments would come to the rescue, you know, they could sort of send in the fiscal firepower and they could have quantitative easing but we've had huge debts from then then we had the pandemic and so going into this if there is a huge crash it's not actually clear that governments who are already not only massively indebted but paying debt interest at very high rates it's not very obvious that there is that support from governments so this is a, it's not a sort of black swan because it's not an event that comes out of nowhere this is a foreseeable I think catastrophic exposure of the geopolitical system and the global economy, which I think people aren't really thinking about.

40:14Amol Rajan: How can we prepare for these catastrophic scenarios, if at all? I mean, I think what's so horrendous about financial crashes is just the impact they have on people's lives. And if you have something, like 2008, people had their homes foreclosed on, people lost their jobs. It can be really catastrophic. And so I think that like it sort of we've just got to do our best job to build the technology in a way that that avoids these horrific scenarios we've been talking about. And I think that that's the like that's the crux challenge here, right, is how do you do that? What is the what are the what's the governance mechanisms that let let let us navigate this better than we've ever navigated probably a technology before?

40:59And we've just got to rise to the occasion. I mean, you know, before we started, we're talking about the people who sort of helped us, you know, get through the kind of creation of nuclear weapons without a nuclear war. You know, they're like individuals made a difference. And there are individuals from the Cuban Missile Crisis to other situations that sort of ultimately played like a, you know, a role in avoiding horror. and I think that we've got to just really hope that Donald Trump, Xi Jinping, the people close to them can really play an incredible role of leading us through this era because ultimately they are the most important actors right now.

41:41Amol Rajan: What are the other technologies that are going to shape tomorrow that we're not talking about enough? It's worth saying, I think you've invested in over 150 companies and it'd be wrong for me to cast you, I mean not that we will, but to cast you as the kind of AI guy, right? We might get into Songkick and your early forays in being a very successful entrepreneur, but you're someone who invests in a huge range of technologies. What are the technologies that we're just not talking about enough but which are going to change the world? Yeah, I mean, there are so many exciting things in development.

42:10I mean, what was really interesting for me about AI, actually, is even in 2021, when Anthropic raised their first round of funding, it was actually very difficult for them to raise funding. It wasn't like a popular company to invest in. And, you know, the sort of, you know, the people whose job it is to invest in technology startups all basically didn't want to invest in that company. Right. And so, you know, technology surprises us. And just like everyone woke up to AI with ChatGPT, I think there are some other technologies coming that are going to be, you know, I think potentially as significant.

42:42So, I mean, one that's very dear to my heart is fusion, nuclear fusion, because I think it's just it's I mean, we've been talking about a lot of like, really hard to balance positives and negatives with AI. And I think what's so wonderful about Fusion is the negatives are pretty slim. It's basically cheap, clean, sustainable, baseload energy. It reduces our dependence on externally imported resources. I've been talking about it for a long time, Fusion. I remember getting excited about it when I did A-level physics, which is 25 years ago.

43:17Amol Rajan: But why has it not turned up yet? But I mean, it's really getting close. You know, we've had this amazing experiment in the US demonstrated more energy out than energy in, you know, at the National Ignition Facility. And I believe that we will see, you know, companies, startups demonstrate net energy gain in sort of power pop, in things that could be turned into a power plant in the next, you know, five years. I mean, it's so exciting. I think when people realize that fusion really might happen in the next decade, we might have power plants connect to the grid in the next decade. It's like insanely exciting because we will have a way to scale the amount of cheap energy we have access to without putting carbon in the atmosphere, without, you know, you can put one next to the city.

44:06Like one of my personal bugbears, I really feel like we shouldn't have turned Battersea Power Station into a shopping center. We should have saved it for a fusion reactor. like how wicked would that be if you had a fusion reactor on the bank of the thames but you know i don't know maybe i mean it's hard these facilities are pretty big but it would have been some international property developer but it would be i think what i love what i love about that idea is that like fusion is inherently so safe you don't have this kind of meltdown risk and the idea you could build it in the middle in the center of a big if one of the biggest cities on the planet is kind of like an inspiring idea

44:42Amol Rajan: AI is changing the world, but the dilemma we face in business is making AI work for us and not everyone else. That's where Google Cloud is delivering results for big UK customers and why I am so pleased that they are launch partners of the Amol Rajan Connection. Their work is based on 20 years of research and development. And as part of that, Google Cloud's AI assistant, Gemini, helps each company design a unique platform, bringing together fragmented data and workflows. Discover more about making AI work for your business, cloud.google.com forward slash AI. Let's talk about quantum. So I did this series.

45:22Amol Rajan: It's a quantum computing. I did a series called the Amal Rajan interviews for the BBC. the first guest was Sundar Pichai. And I went to Silicon Valley. I was sat in the Google campus, hot sunny day. And Sundar Pichai, who, transparency, Google Cloud, a large partner of ours, I don't mind admitting this, he's one of the most impressive people you could ever meet. He was saying that quantum is kind of basically up there with AI in terms of the potential implications for the good of humanity. What is quantum computing? Well, so in some ways, this stuff is all quite like mind bending. but if you've got a sort of what we think of as like a classical computer, a normal computer like a CPU, GPU, they think they can kind of compute in terms of bits, ones and zeros.

46:04And that's the state the system can be in. Whereas a quantum computer can kind of exist in a range of states between one and zero. So it can do calculations that a classical computer can't do.

46:18Amol Rajan: The toying cost is the obvious one, right? So people say you toss a coin, it could be heads or tails. Computing, conventional computing is heads or tails. But think of that coin as it goes through the air, going round. It's all the states between heads and tails. And therefore, it's all those different possibilities. So it's a computer that can do exponentially more. Yeah, exactly. But the thing that is like much deeper than that, and that like, it's... I'll take that as an insult. I'm happy to be... No, no, no. My shabby base kind of metaphor didn't quite catch. No, no, but it's not that the metaphor's bad.

46:50It's that Newton developed sort of an understanding of how the universe works that was ultimately sort of succeeded by Einstein's understanding of how the universe works, which is that we need kind of quantum effects to explain how the universe works. Right. And so you can sort of think about the distinction between a classical computer and a quantum computer is kind of one is sort of operating the world of Newton and the other is operating in the world of kind of Einstein. And so what's sort of amazing about that is that, you know, implicit to that is that a quantum computer is fundamentally a more powerful general purpose technology than a classical computer in the same way that like Einstein's insights explained the world more accurately than just the Newtonian laws of physics.

47:34physics, right? So it's sort of like, you can think of it as being sort of a more powerful technology than the computers we have today. And what I think is exciting about that for like an everyday person is that in order to simulate materials, understand drugs, you have these quantum effects. And so if we actually want to be able to understand how materials work, how biology works, we need a tool that understands how these systems with electrons work. And so, there are lots and lots of things that classical computers will never be able to do that quantum computers can do. And they can do it trivially compared to the computers we have of today.

48:18So, it's incredibly exciting. And what I think is great is that the UK is actually, you know, this is kind of a, it's a race, similar to the races in AI or the race in fusion, but it's a race where I think there's a lot more to play for. And I've been helping one company which is called phase craft which is a company for transport you invested in invested yeah yeah exactly and um i was their chair from kind of pretty early on you know we we have there's another company called quantum motion i'm not involved i think it's fantastic um there was a company called r &q there's a bunch of companies sort of working on this technology that are based in the uk that are that are great so when it comes to the technologies of tomorrow how does britain win i think we sort of need to be ahead of the curve is like my overall view i think we have incredible scientists who are ahead of the curve.

49:01You know, there's so many profound things that have been invented in the UK and discovered in the UK. And then I think we do actually have amazing entrepreneurs, founders willing to take those ideas and sort of take them out of the lab. You know, think about someone like Demis Asabis, you know, he studied at the Gatsby Institute at UCL and then found the DeepMind. And then we've got to basically protect them as they try to get bigger. And the thing we ultimately, I think, have to protect them from is much, much, much larger companies, predominantly in California, because if they see something that a small British startup has invented that could be world altering, the first thing they try to do is buy the company.

49:40Amol Rajan: So DeepMind is now Google DeepMind, Facebook, Instagram and WhatsApp. Yeah. And so if we want to have these kind of brilliant ideas that turn into brilliant young companies we sort of have to support them through this very very uh challenging phase where they've got a brilliant idea they've validated they've done something amazing like Demis did with with early deep mind um and we've got to kind of help them cross the chasm where they they can actually become a sort of you know we want our own nividias our own googles our own microsoft how do they cross that chasm and how do we help them i mean it's i would say it's more sophisticated uh investment that keeps pace with the kind of investment available in silicon valley or in in in china Is that to do with the size of number or the type of money?

50:19A bit of both. Yeah, I think you need basically people who actually understand what it takes to build these companies. Most of these things we're talking about require a lot of resource. And so, I mean, as a positive example, this company, this fusion company I've been working with in Europe, you know, they are now the best funded fusion company in Europe, period. And in the type of fusion they're going after, which is called QI Stellarator Fusion, they're now the best funded in the whole world. Better funded than any QI Stellarator company in America or in China. And we've done that.

50:44Amol Rajan: So what went right there that you could learn from to make sure we have the NVIDIAs and the Googles and the Amazons of the future? I think it's very valuable to have people like me working with these kind of often first time founders to just help them learn quicker and go faster and access capital that they might not be at access otherwise. So kind of someone in your corner who's just fighting for you, you know, like helping you get there. And that's sort of what I try to do for these brilliant founders. I think there's also like, there's a lot of things around the creating incentives to pull capital into these kind of companies in the UK.

51:18Like you want to make it, ideally, you want to make it more attractive for these companies to raise investment in the UK or Europe than it is to raise this equivalent investment in California, because we're already coming from behind a bit.

51:32Amol Rajan: But the story that you read and hear and see, I don't know if it's true, you'd actually know the truth. And I merely read it, is that Europe lags. is that in Europe, generally speaking, you hear people generalize things. You know, Europe, they prefer bureaucracy to investment. There's a kind of planning mentality. There's not a kind of approach to risk. And obviously, Europe's a big place. Europe is not the same thing as European Union. West Europe is not Eastern Europe. You know, Europe is expanding. But can I just ask you to address this generalized sentiment that, you know, our American listeners might have heard, and I've certainly heard, which is that Europe is not where you go to do business.

52:02I think there are ways in which Europe is definitely harder. Like I think the lack of, you know, a single language, things like that make, you know, diffusion harder. I think there's also like maybe a little bit less of a sort of cultural sensibility of business owners to sort of adopt the latest thing as quickly as possible that you see a bit more of sometimes in America or China. There's just a hunger to kind of adopt. Let's use Fusion as an example, right? Germany decided to spend a billion euros to build the most advanced stellar age in the world as with public money. And they built this thing and they basically built it pretty much on time and on budget.

52:38And it achieved records for this kind of fusion that had never been seen before. And it basically validated this was pretty much ready to turn into a power plant concept with some changes that commercial fusion could bring. And the German state did that. And then you've got brilliant founders. This company is founded by two Italians, two Germans, and a Brit. And they all came together to build this company together. and I don't see any difference between the caliber of those people and the caliber of the equivalent team in Silicon Valley. Like I think it's literally neck and neck.

53:11Amol Rajan: This is this idea which people at Mariana Mazzucato have had about the entrepreneurial state. It's the idea that the state can actually, it's not kind of state or private, but the state can facilitate the private. Yeah, I think that like in China and America, the state has played a consequential role in some of the largest technology companies. I mean like the classic example would be SpaceX, right? SpaceX wouldn't have got to where it is today without contracts from NASA. And in China, you've got significant state support for their fusion projects, have significant state capital in them as well.

53:43So I think there is something to learn from... We should be humble about the fact that the largest technology companies in the world come largely from a very narrow radius around the San Francisco Bay Area and a small number of places in China. And, you know, we have some examples like ASML in the Netherlands, but we don't have many. And we should try to learn from what has worked historically in those environments. But mostly when I look at these companies in Europe versus the same company in Silicon Valley or China, my observation is that the founders, you have incredibly talented founders. You have amazing foundations.

54:15They're standing on like this example in Germany. And we just need to get them the risk capital to go for it. So is that true of Britain as well?

54:22Amol Rajan: If you had to say, what's the one thing we need to do to make sure Britain is the world leader it ought to be? what's the thing we need to do here? I mean, I think this kind of thing about making it easier for these companies to raise the capital to compete on a global stage is probably the biggest thing. How do you actually do that? So what does that actually mean? Is it access to VC funds? What are you actually talking about? Yeah, I mean, basically, I think having a larger venture capital industry would be, you know, like I think that the, I may not get that easy numbers exactly right, but I think that very roughly, you know, 800 billion was invested in you know um american startups last year and like 80 billion was invested in european startups there's like a 700 billion capital gap so you've got to figure out how to close that gap right and what i'm claiming is i don't think the problem is the talent of the founders at the scientist engineers coming up in europe i think we've got comparable numbers and we've got amazing universities amazing you know like i mean the sum of the stuff being spun out i mean like the these are all like basically spin outs from universities in often case or from you know government funded projects we've got this incredible base research foundation and now we've got to figure out how to transform it into these extraordinarily important and influential companies okay you talked about the spirit of learning we have had so many questions for you and we're gonna in monday's episode we're gonna get into that i do want just briefly towards the end of this to talk a little bit about your uh amazing career and what you've learned and actually to get some advice because I'm a startup myself and you might, well, I think it's pretty evident that you've got a lot to teach me.

55:50Amol Rajan: Just before we even do that, I do want to get excited about technologies and another one which I know you've invested in which has extraordinary potential in ways that people don't necessarily understand is robotics. And I just want to, you know, we excited people about fusion, we excited people about quantum, we excited people about Britain and Europe. Can we just excite people for a moment about what robotics is about to achieve? Why should we be talking about this in a positive way in which, is sometimes lost in general coverage. Yeah. I mean, I'll give you a couple of examples of companies I find inspiring.

56:19Amol Rajan: Let's be transparent about whether you've invested in them. Yeah, sure. So there's a company called Monumental in the Netherlands that has developed bricklaying robots that can basically lay bricks faster than human bricklayers and build the walls, et cetera. And what's really exciting about that is that we've got a big building problem, right like we know we need more houses right and actually the sort of the the skilled builders are you know we don't have enough of them to build everything we want to build in this country or in europe or in america and so if we can automate aspects of construction you know we can significantly reduce the cost of home ownership and increase their supply of housing right so to me that's really exciting it sort of doesn't have any sort of you know big risks attached to it it's massive positive thing if we can do it.

57:07Another example would be a company called Robutei, where we're invested. It's a French company that has developed the first ever surgical micro robot. So it's a robot about the size of a grain of rice. It can drive through the brain, causing significantly less damage than sort of traditional biopsy methods, gather data, and then eventually deliver drugs. And so, you know, you're bringing this insane benefit to the brain that just doesn't, you know, the ways we currently treat the brain, if you have a horrible disease, you know, to gather the biopsy sample, you have to sort of stick a needle through the brain.

57:42Do brain damage. Do brain damage. Or you have to, and if it delivered drugs to the brain, you typically have to drug the body and hope that a decent amount of it makes it through the blood-brain barrier. Wow. So it's sort of, it's such a step change we could have in the ability to cure these horrific, you know, these horrific diseases.

58:00Amol Rajan: NatWest partners the Amol Rajan Connection. It's the bank that I've used for 25 years since that first day I walked into a South London branch with my dear dad. What sets NatWest apart is its mission to help people turn ambition into action. From taking a first financial step to building a business or planning what comes next. It is next level banking with practical support, ideas and expertise behind it. I'm very grateful to them for their support. As they say, let's do this. you mentioned bricklayers which means i've got to ask you about one thing which people worry about we're getting wrong with technology which is jobs yeah and the big concern that lots of people have is we don't know about what ai is going to do for jobs etc but the kind of feeling is that if ai is a kind of cognitive um technology that it could do for the knowledge economy white collar jobs what automation did for blue collar jobs we had this i mean i know you've got an interest in social mobility as have i we had this story in the 20th century which is you know go to university, join the credentialed elite, join the knowledge economy, rise up the ladder.

59:02Amol Rajan: How worried are you about the implications of these frontier technologies for the jobs that we will do? I think, you know, I know I kind of keep coming back to it, but I think so much depends on which of these paths we're on. I think in a world where you have a sort of recursive self-improvement loop that takes these systems to being superhuman at everything that we can do and more, then I think you know we have a pretty you know we have such a big thing to handle that like jobs is kind of one big thing there but it's obviously just it's a it's a sort of it's a really good life I mean it's sort of it's it's going to be so consequential and it will have it'll be very extreme in terms of what it does for um you know the way that humans find meaning and humans ability to participate in the economy etc then there's this other path I described which is kind of closer to sort of something a bit like electricity or, you know, where we kind of have something that is, you know, very, very, very capable, but kind of there's still space for smart humans to figure out what to do kind of in complement to it.

1:00:05And I think we're kind of currently in that phase at the moment in terms of what the system can do. There's obviously ways in which like you can add huge value on top of the systems as they are if you freeze it today. And so I think we, you know, we need to know which of those branches we're on. And I think if we're on the kind of the, you know, closer to electricity path, I think it's going to be, you know, kind of amazing. I mean, think about some of these technologies coming and what people will be able to do with them if you want to work on new materials or you want to work on new kinds of planes or rockets or ships.

1:00:37Or, you know, there's going to be so much capacity to take innovation and run it faster. and so I think it could be again amazing in terms of unleashing talented people on to have a greater impact on the world faster than they could before and I think that you know think about doing your podcast you know 20 years ago versus today and how much more you can do with a smaller team and yet still have huge impact like that's a that's a change that sort of you're getting leverage from technology I think in the happy path this gives us extraordinary leverage and we will we will see our young people be able to do even more than that we could have imagined just a final thought on that

1:01:10Amol Rajan: because we've got a lot of young people that follow this program and a lot of parents of young people, including me and you. How do you think about the changing imperatives of education in this world? Should we be studying things differently? Should we be studying the same things? Should we be kind of complementing what we do? How do you think about how education and the purpose of education changes in this revolution? I think I do. I think I'm like probably a little bit skeptical about trying to sort of be too smart about predicting exactly where the jobs of the future are going to be i think a general education is probably a yeah a safer thing to do i mean think about all the people who said okay like looks like software's where all the money is let's focus on software and now the software's probably been the easiest thing for ais to automate right what does that yeah what does that mean just sorry this because i literally were talking a few hours after there's a story this morning about how the value of uk software companies could be massively impacted by this right because this idea of reverse engineering right suddenly you can sort of work out how they've created all this software.

1:02:08Amol Rajan: I mean, is that an example of how you could actually have the destruction of a large amount of economic value? Because software, which used to be something that people got paid for, is that something that you can automate? Well, yes, in some ways, but also the productivity of anyone working in software will go through the roof so they can do more with less. So it's, you know, you'll become a software company in the sense that you'll have apps and things that people are using that you wouldn't have been able to afford before because of the cost of making software collapsing so i think it i i i don't know whether what we'll see with the uk software industry i don't i'm not i'm not like i don't think it's clear yet how it how it shakes out um i've noticed that as a theme of what this incredibly enjoyable time we've had together of your kind of humility about your kind of epistemological humility you don't know how the future is going to be i'm going to get told off asking this question by my um close colleagues because they they'll just think it's inappropriate I'm going to ask you anyway.

1:03:02Amol Rajan: I've just started a company. You know a bit about growing companies. What do I need to do? I'm not asking you to join the board. This is not some weird sort of strange thing. What do I need to do to make this a huge success? I think that the deepest lesson I've taken from building companies are two lessons. The first one is that just like the importance of a really, really, really compelling and exciting vision combined with very practical steps to start moving towards it to tomorrow. And I think that people just are, you know, it's so when you meet someone who's working on something, it's incredible vision about how the world's going to change if they're successful.

1:03:39You know, it compels people to want to help them and join them and support them. But then that has to be combined with this sort of hyper pragmatic sense of, okay, practically, what do we do tomorrow? Then if we want to get towards this thing, that's going to happen in 10 years time. and then the other one that I think is it's slightly harder to learn is just like how kind of people are everything you know like when I when I when I um uh set up AC it was my second the sort of second organization I'd kind of founded and I was really curious where I was going to take my own advice uh did you and I did and I basically what all I did is I just sat in a room in you know um in Westminster and I just called people I was trying to recruit for like the entire first three months.

1:04:18And I got given a list of all these meetings I was supposed to do and all these things. And I just said, I'm going to basically focus on hiring the people I think we need to build this thing. And so I think there's a total obsession on getting amazing people into your organization is the single biggest thing. But it's hard. Most people still shortchange it. They kind of don't realize quite how obsessive you have to be about talent.

1:04:40Amol Rajan: Okay, I'm going to make sure I don't shortchange it. Final quickfire questions. maximum 10 words, minimum one, which I ask everyone with no warning. When did you last cry? Probably on a plane. I get like really emo watching movies on planes. The last film I cried out was All of Us Strangers, which I thought was amazing. If you could relive just one moment in your life, what would it be? That's a great question. The thing that comes to mind is, you know, I had like a very happy childhood and my mum and I spent a huge amount of time together and I loved her very dearly. And I think that whatever it must have been like to be five years old, just hanging out with your mum all day and having fun.

1:05:22Yeah. It fades over time, but I think that's probably something like that. Something really far in the past where I can't really remember it. So that's what I'd be most kind of curious about is what it was really like then. Because I've got kids and, you know, yeah. So have I, it makes me think about it. What do you worry about the most? A mixture of the future of the UK and like AI risk. What are you most excited about? Future of technology, I suppose, like fusion, things like that.

1:05:49Amol Rajan: Our slogan is win the future. How can we win the future? I think we've got to be really determined. I think we've got to just try our absolute best. And I think if we do that, we'll have a pretty good shot. I do believe that there just is an extraordinary amount of talent and ambition in this country to do wonderful things and make make things that the world finds valuable and that as a result make our country kind of rich and and um able to afford all the things we want to spend money on and so i i think we just have to be really hardcore just just try really hard ian it is such a pleasure and a privilege thank you mate thank you thank you so much you know it's really hard when you have a conversation of that caliber to distill it down to a few things but here are three quick takeaways number one we're at a fork in the road as a species we have unleashed a rival intelligence and we're about to take that fork in the road and we're going to find out in the next 12 months whether it turns out that ai is a good thing broadly a bit like electricity unleashes great productivity or it's a terrible thing because it recursively self-improves that is happening and it's happening right now point number two i think we're underpricing the chances of total economic catastrophe?

1:07:04Amol Rajan: Ian made very, very clear as someone who's both an expert in AI and a very successful investor, that the chances of either a single catastrophic event or the entire model of these AI companies unwinding is really quite high. And as we discussed, the chances of people coming to the rescue, like governments, as they did with the great financial crash, it's just not going to happen this time. That could be very, very bad. But the flip side, to end on a positive note, is technology is neither good or bad. It's what we make of it. And there are other technologies which are maybe not getting the attention they deserve that could completely transform the prospects for humanity over the next decade.

1:07:38Amol Rajan: Robotics, nuclear fusion, quantum computing. These are incredibly exciting. And what we've got to try and do is master them and use them to unleash human flourishing, eudaimonia, instead of allowing AI to kill us all. Thank you so much for your time. Thank you for watching. Please, please, please do come back on monday where you'll get our questions and answers episode quark where ian answers your brilliant questions and please also do subscribe or follow for free wherever you're listening to or watching this podcast just before we go i want to say thank you again from my heart to waitrose amazon google cloud and natwest they are our launch partners for the amal raj and connection and we couldn't do this without them we also couldn't do it without you see you monday

1:08:25you