Q&A with Ian Hogarth: Is it too late for the UK to control AI?

0:00I think it's something we should take seriously as a risk. The crux question is, do we have systems that are able to recursively self-improve themselves to a level of intelligence that goes beyond humans on all areas? And then if we do have that, are we able to kind of design those systems such that they are kind of, they care about humans flourishing?

0:17Amol Rajan: Hello and welcome to this Monday edition of the Amol Rajan Connection. Remember, if it's Monday, that means it's over to you. This is the questions episode. Questions plus ARK makes Quark. We call it the Quark episode. we've got loads of brilliant questions from you for ian hogarth and he's about to answer them just before he does that if you want to submit questions to our future guests if you want to find out who's coming on please do me a tiny favor head to amol rajan connection.com search for join the club and then that way you can sign up for the free weekly newsletter find out who's coming and submit your questions as well just before we get your questions and ian's fantastic answers let me say again a Huge and heartfelt thanks to our launch partners.

0:57Amol Rajan: So thank you to Waitrose. Thank you to Amazon. Thank you to Google Cloud. And thank you to NatWest. They are our launch partners. They make all of this possible. And I'm very grateful to them, as I am grateful to you and to Ian for answering your brilliant questions.

1:13Amol Rajan: Ian, are you feeling ready for some of our audience questions? Sure. We've had a lot, partly because of your stature, partly because people really care about this technology. Not just AI, but beyond. Okay, this is the first question, and this is from Angela. Hello, my name is Angela from East London. And my question on AI is this. A few weeks back, a news item said it's possible AI might wipe out the human race. So how would AI achieve this? By strangulation? Poisoning? Firearms? Something else? Angela, thanks for going straight to the heart of the matter. If AI was going to cause existential wipeout, what's the mechanism?

1:50Amol Rajan: How would it happen? The scenarios people describe usually are things like the way in which humanity, without really intending to, has ended up destroying various species by changing the environment to suit us in a way that doesn't suit the other species. So that's the classic way it could happen. It's just you sort of effectively have created a competitor species that uses the environment in a way that no longer makes it possible for humans to flourish. What sort of probability do you attach to that? I don't really attach probabilities. I think it's something we should take seriously as a risk.

2:32And basically the crux question is, do we have systems that are able to recursively self-improve themselves to a level of intelligence that goes beyond humans in all areas? And then if we do have that, Are we able to kind of design those systems such that they care about human flourishing? Right, those are the deep questions there.

2:52Amol Rajan: So you talked in Thursday's episode about kind of two paths, this fork in the road. Is it like electricity or is it something very, very different? When and how do we find out? Yeah, I mean, in many ways there are people working really hard right now to try to really understand this question of whether an AI system can recursively self-improve itself. and they're trying to gather evidence by looking at things like how much can ai improve its own algorithms how much can ai improve um the chips that it's running on and and how quickly can those um can it can it kind of close part of those loops so it's a bit like you know when epidemiologists were trying to understand you know how quickly a um a a like a virus can grow it's sort of you know there's this kind of idea of the like at some point you have something that just grows exponentially can we get to a place where ai has a sort of similar dynamic is the big question those people are wrestling with and i think that you know the reason i think we should take it seriously is that there are there are people who seem like you know people like yoshua bengio jeff hinton researchers these organizations the leaders these organizations who've all sort of said we think it's possible like that's i think that means we should take it seriously i think we also should you know try to gather as whatever empirical evidence we can do as to exactly what is going on so we can have a more informed conversation about it okay well we're trying to have that conversation now which is why it's so fantastic to speak to you this is another question uh god we've got so many this is from dwight dwight thank you for sending in this video hello amal hello ian i'm dwight here in bristol home of the uh isambard ai supercomputer and where you will know i'm all drumble music became drum and mace so my question for you ian is do you think britain has become over dependent on foreign nations for say technology and should we should we seek to address that and if so what role does the state have to play in making that happen or have we been asleep at the wheel too long at least too great wow yeah that's a great question i mean just to go in with the musical theme i like that um that dizzy rascal line uk stand-up tool and i think there's like I think we owe it to our kind of country to reduce dependency where we can.

5:15I think that just like we have kind of, we have a concept of energy security and food security, I think we should kind of have a compute security mindset around some of this stuff where we have, you know, we do have data centers, we do have our own models, we do have our own robotics companies so that we can play a meaningful role in both influencing the trajectory of the technology, but also like having a higher degree of sovereignty and I don't believe it's too late. I think fundamentally there is scope for us to lead in lots of ways, but I think we need to get really serious about what that means and mobilize the appropriate attention and resources to do it.

5:51Amol Rajan: Okay. Dwight, thank you very much indeed. Maybe see you in Bristol and if I do come down there, I'll explain in detail the difference we've done with the drum and bass, which seems to be the thing. Until my dying day, I'll be most famous for it. Dwight, appreciate your video. Thank you, sir. This is another question. This is from Gail. Let's hear Gail's question. My name is Gail from London. Much of AI's content has been scraped, aka stolen, from copyright holders without their permission or payment. When will AI companies negotiate proper terms with the copyright holders and give the copyright holders the right to opt into the agreement instead of opt out?

6:28Amol Rajan: Gail, thank you very much indeed. I don't know if Gail's got a particular interest as an artist or not, but this question of copyright is a big one. Yeah, I think it's a really big one. I don't understand this as well as some other people who kind of specialize in it. But what I think is quite tricky at the moment is that it's legally less risky, it seems, to train an AI model on the Internet, for example, in America or in Japan or even in the EU than it is in the UK. Because we actually do have, I think, stronger laws around copyright. Right. And what's what's challenging about that is that it doesn't actually seem to be affording British creatives and rights holders any more significant protection from what I can tell, because the models are still being trained and we're still using them.

7:18Right. They're being trained in China. You know, we're using Chinese models. I've been trained in America and we're using American models. and so it's not obvious to me that we are we have got a strategy yet around around copyright that really works for the interests of the the British people but I think that the question raises is incredibly valid like how you know in many ways if AI is being built upon you know this enormous edifice of all the knowledge and work of humans to this point and how does like you know if it's being trained on all of our contributions in some ways for example through the internet then how to kind of um how is that recognized right uh and i think that's a very deep and hard question that we need to work through but i think what's a bit odd is i don't think the approach we're currently taking seems to be working either for people who want to build ai companies in the uk or british creatives so it's not really i i feel like we we need to have a better conversation about this topic and not sort of just polarize it and figure out how to win.

8:20Amol Rajan: Okay. Gail, thank you so much. The only reason I'm not coming back to some of Ian's answers is we have got so many questions. And I, yeah, by the way, thank you so much to everyone watching and listening who is sending questions in. It's incredibly important to me that this Monday episode is a chance for you to put your questions to our incredible guests. So thank you. This is from Kieran, who's emailed in Ian saying, Kieran says, what are the implications, both positive and negative for when AI and quantum computing truly begin to converge? Well, that's a great question. Thank you, Kieran. AI systems are able to get smarter much faster when they can verify that they've got a correct answer.

8:58And so, you know, a simple example of that would be like if an AI knows the rules of chess, for example, and it can get really fast because it can kind of verify, did I win or not? Was it a checkmate position, right? I've seen you've got the little chess set. You're a big chess man, aren't you? And so there are some of the domains where we're seeing the fastest progress right now, for example, programming and mathematics, part of the reason for that is we can kind of verify correctness. And there's this thing called lean, which is basically a way to verify mathematical solutions. And if we had something similar for other aspects of science, where we could verify correctness in, for example, material science or biology, then progress in those areas would speed up very dramatically.

9:42And the thing that could be that kind of verifier for, say, material science would be a quantum computer itself. So there's a really powerful feedback loop between the two that could kick in when quantum computers are mature enough to be useful verifiers for AI systems.

9:58Amol Rajan: And for people who don't know much about quantum, obviously, they should go back and listen to Thursday's episode because we talk about the potential of quantum computing in it. But kind of is quantum computing developing at a similar rate to AI? There's also this race dynamic there. how far are we from sort of you know having quantum computers in our pockets or whatever it might you know i think we're i would guess we're at a minimum a decade away from having them in our pockets because you've got we've barely started down the process of understanding how to miniaturize the technology it's like sort of quite bulky um but what is amazing is there are like lots of different approaches to building quantum computers uh lots of different companies within within each approach and they're all making progress at the same time and above that we have software that's figuring out how to do more with the hardware that we have.

10:40So I think we're going to see pretty staggering things start to happen with quantum computers around, you know, call it 2030. That feels like a kind of a bit of a watershed moment where the hardware will, there'll be a powerful enough computer, smart enough algorithms that we'll start to see it be very useful in things like material science or optimization. Wow. It's exciting. It's really exciting, yeah.

11:01Amol Rajan: Kieran, thank you very much at E for that. God, lots of questions. Brenda Bates has emailed in Ian to say, my question is, why do we seem to accept that those who own and operate AI businesses are not responsible or accountable for their activities when most other industries like aviation, pharmaceuticals and food are? Do you accept the premise of that? I'm not sure they are not accountable. I think that like, I don't, there are some sort of liability shields that are being created around certain industries. I think social media has a kind of liability shield around it. As far as I know, that has not been created for AI companies.

11:39So I think they are legally liable in a way that, for now, has been preserved.

11:45Amol Rajan: There is this broader question there, isn't there, about if not legal liability, the kind of moral hazard of if they've built these business models where they've taken on huge debts and they can't actually fund them because they can't get the revenues. If they crash, then it affects all of us. Yeah, but at the same time, they could deliver huge benefits to all of us. So I don't think any company should be too big to fail, personally. I believe in you have to let things fail to have a feedback loop that doesn't just encourage people to take risk that ultimately other people bail out. That seems sensible.

12:25Amol Rajan: Right, another audio question. This is from Guido. Thank you, Guido, for sending this in. Hello Amal, Guido Liguori from Bedford here. I use AI frequently to support my research, but I do worry about the environmental impacts of data centres, given all of the media commentary around water and energy usage. First of all, what is the real state of affairs regarding the environmental impacts of data centres? Secondly, are the companies behind AI actively working on solutions to make data centres more environmentally acceptable? And finally, are data centres dedicated solely to AI or do they also serve a wider purpose such as general data storage?

13:11Amol Rajan: Guido, thank you so much for that question and well done for squeezing three questions into one. I'm going to put all of them to the great Ian Hogarth. State of affairs, what's going on? I think there's lots of different approaches to data centers right now. You know, I think that, for example, one of the ways people have been building data centers very quickly is to build completely brand new gas power stations just for the data center, right? And so they're like, you know, there can be some local pollution challenges to manage there. And then there can also be like global pollution challenges in terms of the carbon emissions of data centers.

13:42There are also data centers being built next to very abundant hydroelectric resources, for example, where the energy doesn't have any carbon emissions associated with it, isn't necessarily in competition for other industries to use it. So there's a lot of variety. And ultimately, a data center is not just consuming energy. It's also helping us do other things more efficiently. So it has knock-on effects in how we use energy more efficiently elsewhere. I think that, for example, let's say a data center develops a new superconductor we can use to transmit energy far less wastefully, could be a big net systemic benefit versus one that doesn't exist.

14:20So I think it's a complex set of trade-offs. I think personally that we should be building data centers kind of as quickly as we can in this country. I think it's a source of hard power to have these assets. But there's a couple of caveats, which is I think we should really try to innovate on the form factor of a data center so that the way we build the data center is it squeezes the absolute maximum amount of intelligence out of the energy footprint. and as much as we can figure out how to have a kind of sustainable data center in the sense of the way it uses energy, the way it uses resources, is something we can scale without it being kind of in contention with what humans need the energy and resources for.

15:03So I think there's a set of trade-offs. And the last thing I'd say about the UK is that we've brought a huge amount of new renewable assets onto the grid. We've invested massively in upgrading our grid. We've done all these really important things and we have incredible, for example, wind resource in this country. But what we lack actually is demand because the cost of energy has been so high that the industry has been leaving the UK, which means there's less people using the energy we do have, which means that the cost of energy is ultimately going up for everyone. So if we can get some very large, well-paying data centers onto the grid, it could actually help finance upgrading the grid for everybody.

15:44So there are some really big benefits here where I think we kind of need to find a sort of smart, pragmatic way through. And, you know, in reality, like, you know, I think that China is projected to add something like 50 gigawatts of data center capacity next year. Like the US will be tens of gigawatts of capacity next year. In the UK, I don't think we're going to even add gigawatt next year. Right. So we should be clear that we are not building a large amount of data centers presently relative to our GDP or our population size.

16:09Amol Rajan: What does that mean for the question, which I know you think about a lot, which is how do we not get left behind? China, but we often people understand this in terms of airports, right? Because they can see airports or cities. You know, we've not built a new city in this country since Milton Keynes 60 odd years ago. We've not built a new reservoir for 35 years. When you say that China's adding 50 gigawatts of data center capacity and America in the tens of gigawatts and we're not getting to one. what does that actually mean for us as a society what's it what's it what are the implications of that for people watching and listening now i think it means lowered sovereignty you're like reliant on some a data center somewhere else and the cables then you know connect you to that data center in in times of war being you know maybe severed so you're just like you have fundamentally less sovereignty and you know unlike energy sovereignty or um you know energy security or food security there's kind of compute security do you have enough compute to do the things you'd want to do in your country so i think it just makes us vulnerable if we don't have the you know the equivalent kind of capabilities just like you'd want power stations you want data centers i think uh in the scenario where ai looks a bit more like electricity um because it's it's it's ultimately the the means to the sort of the means of production and how is just i i don't i don't know how is britain doing on compute sovereignty i think we've got a number of uh we've got a lot of startups actually innovating on the underlying nature of computing and how we build data centers.

17:37And we've got an amazing project as part of this thing called ARIA working on kind of better data, smarter designs for data centers, I think is really compelling. But the thing that we are not doing is building data centers at a great pace. And I think that has sort of been a kind of a choice. You know, it's been a choice because I think there's been a perception that data centers are in conflict with an agenda to decarbonize. And I'm not sure they are in conflict. Personally, I think there's a way to harmonize those goals and see data centers as essentially adding demand to the grid that can help us modernize the grid, add new generation assets to the grid faster.

18:18Like, I think there's a way through it where you can kind of win on both fronts. Okay.

18:21Amol Rajan: So this is a question, Ian, from Joe Braidwood. Joe Braidwood's company, Glassis, G-L-A-C-I-S, is building a flight recorder for AI. Ian, AI is already making decisions in hospitals, banks, and insurance. When something goes wrong, there's often no reliable record of what the system actually did, only what its makers say it did. Planes have flight recorders. Should high-stakes AI be required to keep independent, tamper-proof records of its actions so that regulators, insurers, and the public can check for themselves? I would put that in the category of questions that sound quite technical and have huge implications for many industries.

18:59Yeah, I mean, I think that like, I think we need to figure out kind of, for a better word, like monitoring for these AI systems. So you'll be familiar with this thing where the OpenAI agents kind of, you know, hacked another website as part of kind of collaborating together to try to achieve a goal. And so, you know, part of the reason that took a while to discover is because the monitoring capabilities that had been set up to monitor what the agents were doing weren't adequate. And so I do think that monitoring is important, but it's only one aspect of security. I don't think it's even vaguely, it's sort of necessary but not sufficient.

19:41So I think it's useful, but not the whole solution.

19:45Amol Rajan: Okay, two final questions. Let's see if we can get them both in. This is from Tom. Here we go. Hi, Amal. My name's Tom Shuller. I'm an MA student in AI and ethics at Birkbeck. And my question for Ian Hogarth is, has he read Anthropics Constitution? And does he think it's an adequate safeguard for the future? Thanks very much. Thank you, Tom. I don't know if Tom's got a particular research interest in Anthropics. For people who missed this episode, which I should go back and listen to, So you were an early investor in Anthropic, but are no longer an investor because you forfeited at great personal cost your shares in Anthropic to do a public service job.

20:27Amol Rajan: Have you read the Constitution and is it sufficient? I haven't read the most recent draft, but I'm very familiar with kind of what it is and what it's intended to do. I think it's one hypothesis for how we kind of steer these systems to be more aligned with kind of human goals. There are other ideas for how we could achieve the same thing. I mean, for example, some people would say rather than having one sort of centralized AI with a constitution, every individual should have their own AI and their own constitution, right? So, you know, or every country. There's lots of different hypotheses for how you should do this.

21:03And it's very much a work in progress figuring out the best mechanisms for doing that. So I'd say like it feels like an important and interesting thing that they are trying to do. But it's very contended what the best way to solve that problem is. It's a contribution to the space.

21:20Amol Rajan: All right. Final question from Annabelle. Annabelle, thank you. Hi, this is Annabelle from Sussex. I have this question for you. Are we forgetting to think about the people in the AI frenzy for productivity and efficiency gains? I'm not referring to sort of reskilling or upskilling. I mean, what does the future of work actually look like when we deeply consider sort of the huge advantages of humans and what we can do, which AI will never be able to replace? Annabelle, thank you very much indeed. It's worth saying we had absolutely tons of questions on that theme, which is what does AI and technology more broadly mean for work?

22:01Amol Rajan: and I should say that one of the things I know that you've talked about this episode and on Thursday is that you do kind of want people to come first and it's about making technology work for people rather than the other way around yeah and that's and that's why some of your interests beyond AI and again people should listen to Thursday's episode whether it comes to nuclear fusion or robotics or quantum computing it's really how can technology serve human interests and human flourishing eudaimonia as Aristotle called it in my favorite word Annabelle's question is about what AI specifically means for the future of work and the implications thereof for humanity.

22:35Yeah, I mean, I think it's a great question. And I think we're still figuring out what does it look like to work with AI? I think the version of it that feels kind of most optimistic to me is kind of helping people do less of the bits of their job they don't like and more of the bits of their job they do like. You know, that feels like a sort of positive place to head towards where, you know, you know when you're at your work and there's things you get, you know, you get, like I get incredibly energized talking with people building new things about how to like bring their dreams to life faster.

23:09And I get like more de-energized by say, I don't know, like writing up meeting notes, for example, right? So there might be things where AI can just take over some of those steps and save you time in a way that then lets you focus more on the things where you maybe have more to contribute. so i i think i think we're just figuring it out and i but i think annabel's um it was annabel right like her question is really on point because sometimes it feels like the people building these things don't have that strong an intuition for how like you know kind of people want to work like and i think we need to be really careful to sort of have like some sort of like you know dehumanizing version of work where you've got all these billions of agents and you can barely keep up and you're you know working even harder than before but not necessarily even you know there's a not seeing the not seeing big productivity gains for yourself with it and spending even less time with your kids like you know there's there's versions that are pretty dystopian and assuming that we can be we can work very in a high we can be very complementary to these ais um the companies that really figure out how to like harmonize human and machine intelligence the best will i think be

24:19Amol Rajan: some of the most successful companies it's so interesting you say you get excited and turned on by people building new things um so do i which is why i'm doing it ian you've been so generous with your time and i build this when we started talking thursday's episode as i wanted it to be the most useful conversation people hear about technology and i really i mean it's been the most useful i've had about technology this year so i'm so so grateful for your time um you're a busy man and the fact that you've made time to come in and be an early episode on the immoral roger connection means a world to me thank you so much man thanks a lot cheers man wow i think i think that was one of the most useful conversations I've ever had.

24:55Amol Rajan: Thank you so much to Ian Hogarth, who was so generous with his time. And we really appreciate it. Thank you also to our launch partners. So thank you to Waitrose. Thank you to Amazon. Thank you to Google Cloud. And thank you to NatWest. They make all of this possible. We could not do this without them. And we could not do it without you as well and your brilliant questions. If you want to find out who our upcoming guests are, and if you want to submit a question, please do me a tiny favor. Head to amolrajanconnection.com. just hit the button that says join the club and sign up for our free weekly newsletter we'll be back at the same time on thursday i'll see you then