AI's Mythos Moment: Rishi Sunak on preparing governments for AI

26:50And that's not unreasonable, by the way, so much so I think the EU over time has realized that that approach was not right. Mario Draghi, who's written, Mario is a friend of mine, has written a very comprehensive report on European competitiveness and growth, has spoken about that act and actually where it needs to be either watered down or paused. And that has gained a lot more acceptance now. But it goes back to what you were saying. You've got to get the balance of these things right. And if you kind of broaden them out, tip too far in one direction, you're going to lose people. And it's a good example of that in this case.

27:25Well, I think there's a couple of interesting threads one then can pull from Mythos itself. The first just relates to the fact that this is a general purpose model that happens to be extraordinarily good. at finding vulnerabilities in software. Anything that it can get in a binary form and then can decompile into source code or anything that it can get into source code, it can find bugs that no one has fixed. And hence, it can become a cyber weapon, so to speak, for nation states or for ransomware actors. The second thing that I think is interesting is in the national security community, whether you're talking about the UK or the US or anywhere else, the culture is very different from, say, a tech company.

28:11In a tech company, you get up and you sort of ask, what can go right today? What can we build and what will it create that will go right today? And in the security community, I think people get up and go, what is going to go wrong today and how do we prepare for it? And in the parade of what could go wrong the last few years, I think people have often started, not inappropriately, by focusing on bioweapons. How could somebody create the next virus that would become the next pandemic. Once in a century is more than enough. Thank you. They could try to use it to create a nuclear weapon or a dirty bomb.

28:49Here we are. The real issue is cyber attacks. And to me, I think maybe there's an insight. There are far more people in the world trying to create cyber attacks every day than create viruses or nuclear weapons. So as we just look to the future, it's always helpful to then take this learning and understand how do we think about different risks. And the risks of cyber attacks is going to be with us, both because of the nation state incentive, but also just the financial criminal activity that results from it. And I think, Brad, the fact that you mentioned both of those actors is critical. And it was probably something that I didn't fully appreciate until I became prime minister.

29:33And then obviously, I'm spending more of my time on national security. And then it really hit home to me just the kind of sheer scale of the hostile cyber activity that was happening. I think the UK is the country that after the US experiences the largest number of cyber attacks. And I think you see that in your Microsoft data as well. And I saw that as Prime Minister in government. And obviously, there are criminal ransomware actors. And you know what, that's economically disruptive and all sorts of other things. But I'm particularly worried about the nation state actors. And, you know, there are a few countries that are particularly responsible for a large volume of some of the hostile cyber activity.

30:19And it's state backed, state supported. And again, you see this at Microsoft and countries like North Korea and elements in China, especially, and in Russia. And it's a form of what we call hybrid warfare, right? In this world where if you're looking for things that are asymmetric, that don't cost you very much if you're the attacker, but can have big disruptive impact for your adversaries, then that's the asymmetry of it. Cyber ticks that box. but it's also quite hard to, well, it's easy to hide yourself, right? You can, it's why it's, you know, someone launches a missile from place A, you can see who's launched it, you can track it, you know exactly who's responsible.

31:04You know, cyber is a little bit harder. It's in the shadows. Obviously, we know where it's roughly coming from, but it's a lot harder to very clearly demonstrate that. So it works because it gives the, you know, from an adversary perspective, a pretty useful tool. And to your point, we've kind of focused bio and nuclear and all these other things. This is a risk that is already happening today that we're dealing with. And what AI does is just considerably lower the barriers to entry for this existing thing that is happening already that we're dealing with on a pretty industrial scale. We've just suddenly got a tool that makes it far easier to automate cyber attacks, to have them happen autonomously, et cetera, et cetera.

31:53And that is something that, as a mythos, has really brought home for all of us. The other aspect, I think it really, frankly, speaks to the vision that you had. But here we are. After people in the United States concluded they didn't want any regulation that, say, would require that a company submit a model to the government before releasing it if it was at the frontier, therefore has no law requiring it that it do so. All on the theory that this is what will accelerate innovation has led to a dynamic where Anthropic has, call it the world's best AI model, but is not able to release it. It could.

32:43I mean, it could say, you know, Everybody be damned. We'll get it out there. And if it causes more problems than we think, we'll ask for forgiveness later. But in no small measure because of the power of persuasion from the White House, and I'll just say good people stopping to think. It'll never go faster than the speed of thought. I think that's one of the principles here. You know, we're sitting here without clarity, and it may work, clarity about what rules should exist. It may work for the month of May. Anthropic is, I think, letting certain people test it. It's certainly letting the U.S.

33:24government test it. But I don't know too many people who would say this is an approach that will scale, that will be sustainable, that will just rely on the good nature of people and the ability of one government to catch something in time and ask someone to stop. Yeah. Does it sort of feel like, wait, maybe we should go back to Bletchley Park a second time and remember what we all were talking about? Yeah, I think that is right. Just actually picking on our last little conversation on the cyber side and why this is so important is because, you know, as a country, You know, you have lots of bits of critical infrastructure where, you know, if they are the targets of states, not just criminals who are trying to extort money from a private company.

34:13But if they start to become targets for state-based activity, you know, that obviously is a big problem for you as a leader of a country. and the worry I often had in government is I'm relatively relaxed, not relaxed, but I had confidence that our biggest, most sophisticated banks or whatever it might be, like huge IT departments, big budgets, very sophisticated, they're probably going to be fine. And then you said something similar as well, right? You have to worry about those people. But it's all the, whether it's the smaller financial institutions or the utilities or other bits of critical infrastructure, which if they were targeted and impacted, would have huge disruptive effects in your country and communities.

34:56But those organizations are probably not ones that are used to being at the cutting edge of having to invest in these defenses. And that's another area where governments do need to, I think, lean in and make sure that the private sector, because a lot of these bits of critical infrastructure have been privatized, are operating with this appropriate level of cyber defense that they need to, because if something happens, it will be the government that's carrying the can for the disruption, which kind of brings us to this mythos situation. What do you do? And look, I have to be honest, we are a little bit, all of us learning as we are going.

35:30Right. That's a very good point. And I think we have to be, we should just be open about that. But it does start at a bare minimum, I think, with what we talked about at Bletchley, which is, you know, we've got to have a kind of threshold for who this applies to, which, you know, it's never going to be perfect, but is you can do and have those organizations, if they're developing these frontier capabilities, submit them to some technical authorized expert body, which is credible and independent, to provide an assessment of the risks that this model will pose on all the domains, whether it's cyber bio that we care about.

36:12My instinct is that organization is better off being at least linked to the government because it gives it democratic legitimacy, but also it then has better access to the security services and defense network, which will know things that private actors would not. And that seems to me to be the bare minimum. And we've kind of ended up in a place like that little bit with Mythos because of, to your point, the decisions that individual people made. And obviously, I'm an advisor at Anthropic and I'm, you know, I would say this, wouldn't I, but the company itself was willing to have that conversation with the administration.

36:52And then what's happened is you've had a kind of gated release for this thing where it's like, hang on, there's some risks here. We can't just let this thing be released into the world. We need to give defense a time to implement everything and get ahead before it's released. Otherwise, you know, we're giving the bad guys a head start, essentially. And we've kind of ended up in this position. But to your point, we haven't codified it. We haven't quite specified who does that apply to, what threshold needs to be met, and then who exactly gets access to it or not, who's making that decision. Those are all conversations that need to be had and need to be worked through in the usual way in the political process.

37:33And that's what you would expect. And as all of these things, there'll be trade-offs and tensions. But it does slightly focus the mind that I hope people realize that we have got to the point where we probably can't just have an ad hoc approach to this anymore. We do probably need to sit down and have a set of guidelines, guardrails, acknowledging that they are not going to be set in stone forever because things are moving too quickly. But we've got to at least have a kind of framework for how we're going to deal with this, that people have a little bit of certainty and predictability and reassurance that we have a plan.

38:06And ultimately, citizens will need that. They'll need to know, need to be reassured that we have a way to make sure we're managing the risks and this is being were deployed responsibly. The other dimension that you clearly pulled out and put before people quite rightly, I thought, at Bletchley, was there needs to be some mechanism for international coordination. In the absence of that, you put the tech companies and the AI model builders in a, I think, a very difficult position of having to go from capital to capital, go through the same process repeatedly, in all likelihood end up with outcomes that require them to change their products.

38:48So you can be Barclays Bank and you have one model that you're using in the U.K. and potentially a different one in the U.S. and yet you have a common workforce and customers. That also slows innovation and slows productivity growth. And one of the things that we started to study back in 2023 and 24 was what are the models out there for regulatory collaboration? And the one that in some ways spoke to us the most clearly and certainly spoke the most clearly to me is for commercial aviation, a system that was created towards the end of World War II. too. And the fundamental tenet was you had to go have your new airplane tested and certified somewhere.

39:34But there would be a reciprocity regime so you wouldn't have to get it tested everywhere. And thank goodness, because there's one thing we know for certain, the plane that you get on in Seattle and it's the same plane. It's landing in London. You're not going to change one thing while it's in the air. And so that also started to take root. But if you look at the state of international relations, 2026 looks very different from 2023, especially on these digital issues, digital sovereignty. You have been in the middle of a lot of these conversations. How do you take stock of these changes and what they mean?

40:15So firstly, you did a lovely job, Brad, of explaining why the cooperation is so important. And my hope was that these AI security institutes would get developed in different countries, which is, to be fair, happening. They're developing in different speeds with different resources. But the idea I always thought was, well, if you had a network of these bodies and they were cooperating and collaborating with each other, that would allow for this reciprocity where we trust the US Security Institute in the same way that they would trust the UK Security Institute. If they say it's fine, it's fine. And they have that equivalence.

40:57That was my hope, because otherwise you do end up with this completely fragmented system, which is no good for anybody and will mean that we slow down all the benefits. So I hope that conversation is happening. But it does play into this broader issue of tech sovereignty. And the reality of the situation is all these frontier models are all developed in the US. So a lot of the hyperscaler capacity of which Microsoft is one are US companies. and at a time when technology has become so much more important to not just our economic lives but national security issues it has caused lots of countries around the world particularly in Europe to examine some of the assumptions and dependencies that they have taken for granted over decades really and you know that is well documented over the last you know one to two years of certain things that have happened, whether it's in the transatlantic relationship or more broadly, which has led people to question whether that dependence on a small set of US companies for technology is really a sensible or appropriate approach for their country.

42:09And that has posed some really challenging questions, candidly. What it has meant is political leaders, countries have had to really confront across the range of activities that they use technology for. Who are they reliant on? Is that appropriate? Are they going to have agency over the things that are important to them? If in the absolute worst case scenario, they can't rely on a country or a company that is located in a country that might want to leverage that dependency for some reason. And that is the simple question of where we are. And it's a strange question to have, because most of us have grown up in an international geopolitical environment where you wouldn't have had to think about things like that, and particularly not of countries that you do so much with and cooperate with, the US being an obvious case in point.

43:05But a very live discussion in most European capital certainly is how comfortable or sensible is it for us to be dependent on such a small range of US technology companies? Not because we don't trust those companies, but ultimately, they are subject to the US government. And if a government is willing to use, as I said, those dependencies for leverage in some purpose, that is not a situation that sits very comfortably with you if you are running a country. So that is, that is in a nutshell kind of opening up the question of what does it mean to have digital sovereignty in today's age. And, you know, as you know, at Microsoft, we are a voice for international collaboration, certainly among the broad number of countries that have long been friends or allies, share common values.

44:00It's why, and you were there when we created it, at the Munich Security Conference in February, the Trusted Tech Alliance. So companies from different sectors of the tech sector, different parts of it, different countries that said, here's common principles that we adhere to, including respect for the rule of law and protection of privacy, but just good governance, transparency and the like. But we're having to fight for that principle. Yes. To do at least a little bit to hold a little bit of the world together. And try to remind people, especially, say, across the Atlantic, but also across the Pacific with Korea and with Japan and other countries, that this ability to use technology from multiple places in a trusted way is, frankly, the bedrock for ensuring we all have access to the world's best technology.

44:56Yeah. How do we get it? How do we get there? Yeah. At some level, I don't know whether it's optimistic. We should be optimistic or pessimistic. All that matters is that we should be determined, and we are. But what's our best path for success here? Success in that regard. And I think actually what you said, you would not meet any disagreement in most of those European or Pacific capitals. what they would say is, that's great that you say that, but ultimately, you know, you're accountable to a government in your own country. And what you can't guarantee is how they are going to behave in any given situation.

45:38And some of the things that have happened over the past 12 to 18 months, give us a little bit more concern that they might seek to weaponize or leverage these relationships and dependencies that we have. So thank you for saying all the right things and more than that for putting in writing lots of the commitments you will make to us to guarantee access and that we can rely on you as a trusted partner but you know you're kind of not our problem in a sense and and you know what you know what does that mean i look very very practically i think we've gone on a little bit of a journey on this and and in in the wake of and you and i were you know when we were at davos together obviously greenland was was very fresh in everyone's imagination and what was happening you know there was this kind of almost knee-jerk response was, say in Europe, well, the US has a tech stack that's sovereign, China has a tech stack that's sovereign, we need to have a tech stack that's sovereign in Europe.

46:33My view has always been that, arguably not completely possible even for the US and China, but let's just accept that it just about is possible for those two places to have a sovereign tech stack. I just don't think it is realistic or sensible for any other country or entity to think that they can achieve that or to make that an object of policy. So this idea that you can be completely independent of some reliance on US technology, I just don't think is a practical way to go about addressing this issue of sovereignty. But clearly things have changed, and you do need to ensure that you have agency.

47:13So what does that mean? Well, I think first of all, it, it means kind of recognizing that, sure, there are things we can do on sovereignty, but we need to differentiate between kind of data sovereignty and operational sovereignty. This is something obviously at Microsoft we deal with a lot on the hyperscaler on the Azure side. It's like clearly there's bits of data that we don't want to leave our country and be subject to other countries having access to. That is a different set of challenges than the operational sovereignty or the kind of famed kill switch is, are we going to be able to rely on these services that we use every single day and use them in the way that we want to?

47:52And that is a different type of sovereignty. So we need to approach those two questions differently. The next thing is to accept, okay, probably on both of those, there are things we can do to enhance our sovereignty, to reduce our dependence on a small group of US companies. But if we're being honest, most of those alternatives come with a cost, right? There's no free lunch. So if we're going to kind of dial up the sovereignty side of the scorecard, something else probably is going to dial down. Either it's going to be a more expensive solution or a slower solution or a solution that's not operating at the cutting edge of technology or a solution that's more vulnerable to cyber attacks because we've lost all the benefit of the global knowledge that a global hyperscaler has of cyber hostile activity.

48:41So there's no free lunch here. So if I'm going to dial up sovereignty, something else has got to give. And then given that, let me look at all my technological activities that I have, whether I'm a country or a company actually, and be pragmatic about where is it worth paying that extra price, whatever that price is, not just a financial cost. And there'll be some activities where actually having as much sovereignty as possible is the overriding priority. And we're willing to trade off that against other things. And in other areas, we just, it's not worth it. And there'll be some things in the middle where the data we want to protect, but actually if we're using US models, that's maybe fine.

49:26We just need to build our applications in a way that mean we can switch out the models and have some leverage that way. So I think kind of looking at it as a portfolio of activities and then calibrating where you want to dial up sovereignty and dial up or down the other things is how people are now thinking about it. And the other kind of little bit of the coder I'd put on it, which is, I guess, maybe less welcome development, but a reality nonetheless, is the recognition around the world that the geopolitical environment has changed. This rules-based order that you spent most of your career in, that I've spent most of my career in, is gone.

50:05We don't quite know what's replacing it. But what we do know is that hard power is back and that we're in a world that is more contested. And if you're a mid-sized country, you may not be able to have a completely sovereign tech stack or supply chain. But what you can credibly do is make yourself not independent, but indispensable in particular areas. And that is a strategy that makes more sense for a middle power is to identify those bits of key supply chains, technology or otherwise, where they can occupy a really important position, a choke point, so to speak, which gives you some leverage.

50:44If we're at a time when someone is saying, well, I'm going to withhold this from you, you at least have something that you can bring to the table that says, well, if you do that, I can withhold this and this from you over here. And look, as I said, is that a welcome development? Probably not, but it's a reality of where we are. And as I said, hard power is back, leverage matters. And if you're a mid-sized country, that is where policymakers in mid-sized countries are now thinking about sovereignty as well. Let me build for myself some hard power leverage. And that is an approach to sovereignty that I think you will hear and see more of going forward.

51:19And I think to some degree, this speaks to all of these issues. Some ideas will stand the test of time. Yeah. Like the return to Bletchley Park to think about what was discussed there. We may find that some ideas run out of steam, let's just say, and will be brought back together. It is a challenging time. I think we're going to have to get together again to talk about some more of this, but I don't want to let you go without asking you a question about one other aspect of what transpired in the UK those two days in November 2023. Because I'll always remember, we all left Fletchley Park, you returned to London, and that evening of the 2nd, you hosted an anticipated fireside chat with Elon Musk.

52:07But there was a moment during that conversation when Elon turned to you and said, well, you know, with AI in the future, there will be no jobs. There will be no need for people to work unless they want to, but that's not how the economy will work. And you sort of famously said, well, hold on a minute. Work is about purpose and all kinds of things. What was your reaction at that moment? Yeah, I remember that was one of the more stressful jobs I had as prime minister, because that whole discussion was live streamed, I think, on X. And actually, credit to Elon on that whole topic of AI security, he was very engaged.

52:47I think you remember, he came to the summit, he kind of wore his lanyard and participated like everyone else and was very thoughtful and constructive about these topics that you and I have been discussing over the podcast. And so I was grateful that he participated. And this jobs question, I think since then, has probably taken even more of a front seat in people's minds. And look, we have to do the the quick version, because I know we're close to time. History would tell us that these things work out okay, right? Every previous technology cycle has meant that actually over time, some jobs go, but lots more new ones get created.

53:27MIT, as you and I have discussed, have done lots of work on this. Most of the jobs that we know today didn't exist 50 years ago. So I think someone said it, actually, predictions that technology is going to lead to job losses have a long history and a poor track record. So you'd start there. And I would want that to be true. Having said that, the most dangerous words in the English language are, this time is different. It does feel like there are plausible reasons that this time might be different. And that's because the speed at which AI is being deployed, how quickly it is getting to bits of our economy and you see how quickly it takes to get to 100 million users, right?

54:08It took whatever electricity and the telephone 50, 70 years. It took the internet seven, PC 15. It took chat GPT two months, right? So this is happening so quickly. And it's affecting so many sectors at the same time, because it's such a horizontal technology that you could plausibly think this might be a little bit different, certainly in the time that we have to adjust. And what should we do about it? I tend to, I'm not a fatalist on this. I think we can shape the future. My sense is, look, we can create an environment where AI is used more to augment how we as humans do our work, rather than just being used to automate and take people's jobs.

54:47I don't think it is predestined, the deployment between those two things, how that happens. And then the question for policymakers is, well, what are the right policies that you can put in place that make it more likely we end in the augmentation scenario and obviously support people through the transition. And you and I have spoken a lot about approaches to education and skilling and AI literacy, building that in your workforce, how it will be easier to create companies and be an entrepreneur. So there's lots of things that we can do. I think the interesting one that I've started spending a little bit more time on is the tax system.

55:25I think most Western countries tax employment quite heavily because we have social security taxes and all the rest of it. So they're taxing the employment of humans. Whereas if you're going to have an agent, you don't pay any of those social security contributions or anything else. And it does make me think that if you think that the way our economic activity is going to evolve because of AI, like is our tax system, the way it's currently constructed, really set up in a way that is going to tip us on the side of augmenting people rather than just replacing them? And I would say at the moment, actually, because we have these big taxes on employment, we should be looking to reduce and ideally eliminate them because that will, you know, you have to find that revenue elsewhere, obviously.

56:08But that feels to me a change that we will need to make. But I do think it's incumbent on policymakers and leaders to get ahead of this. And, you know, we have to recognize AI is, you know, it's obviously it's popular here on the Microsoft campus. Right. And we've had talking to CEOs the last couple of days. We're all excited about what it can do, you know, out there in the public, in the US and the UK and other places. is AI does not have a great reputation, right? It's people's straightforward view is, hang on, it's putting up my energy bills, it's taking my job or my kid's job. I'm hearing about all these new risks that it's bringing along, et cetera, et cetera.

56:41And a small group of companies seem to be making a lot of money out of this. What's in it for me? That's not a great setup. We need to get people in a more positive frame of mind. So policymakers need to get out ahead of this and show people that there is a path through. Sure, it's gonna change things, but actually it's gonna be able to do so many things that make your lives better, and it will empower you to have more purpose in whatever you're doing work-wise. And maybe I can close with taking us back to Bletchley. I really love this kind of story from that time. Alan Turing and his colleagues sent Winston Churchill, who was a then prime minister, a note.

57:19And they said, look, we're making good progress, but we really need more resources. And if you give us more resources, we think we can crack this thing. And Winston Churchill read the kind of note from them and the team. And he sent it back to his office with a big red stamp that he put on that note. And the stamp said, action this day. And it was a real call to arms. And there was action this day. And that led to all the history that was created and the outcome of that conflict. And I feel what we need here is action this day. We need a sense of urgency amongst policymakers and leaders about just how quickly AI is changing all of these things.

58:02And we need to get out ahead of these issues because we all want to see the benefits, but we need to make sure that those benefits are delivered, you know, reasonably, responsibly, equitably across our countries. And we need to mitigate the risks. So we need action this day on this issue. And that's a brighter future that hopefully we can look forward to. Well, unfortunately, we'll have to close this day right now, at least for our conversation. Let's get back on another day and talk about this aspect more, because I do think it is one of the most important questions for, say, the second quarter of the 21st century.

58:35What will AI mean? What can it mean if we do it right for the future of jobs, the future of work? Thank you. I think you are on the right path. and I think it's a good time to remember that conversation at Bletchley Park and more and where we can all take it together. So thank you, Rishi. Well, thanks for the opportunity to be here, Brad. It's been a great conversation. Good. Thank you. Thank you. You've been listening to Tools and Weapons with me, Brad Smith. If you enjoyed today's show, please follow us wherever you like to listen. This episode is also available in video on the Microsoft YouTube channel.

59:11Our executive producers are Carol Ann Brown and Brent Christofferson. This episode of Tools and Weapons was produced by Mike Morrisrow and Lexi Swanson. This podcast is edited and mixed by Trifilm, with production support by Sam Kirkpatrick at Run Studios. Original music by Angular Wave Research. Tools and Weapons is a production of Microsoft.