The battle to control AI: Who gets to write the rules?

0:00Therapy's biggest impact doesn't show up in some dramatic breakthrough. It shows up in the smallest moments. You bite your tongue instead of escalating. You put your phone down instead of firing off that text. You say what you actually mean instead of what would hurt someone else the most. Nobody applauds those moments, but they quietly change your life. That's what growing feels like. And that's exactly what Grow Therapy is here for. Grow is therapy that works, covered by insurance. Sessions average$21 with insurance and some pay as little as$0 depending on their plan. There are no subscriptions, no long-term commitments.

0:37You just pay per session. Grow connects you with thousands of high-quality licensed therapists across the U.S. offering both virtual and in-person sessions nights and weekends. You can search by what matters, like insurance, specialty, identity, or availability. And you can book a therapy session in minutes and begin in as little as two days, not the national 48-day average. See if you could pay as little as$0 for therapy in seconds at growtherapy.com slash ACAST. That's growtherapy.com slash ACAST. growtherapy.com slash ACAST. Availability and coverage vary by state and insurance plan. This episode of the Times Tech podcast is sponsored by KPMG.

1:24Katie, we've been tracking the AI surge for a while, but some of the latest data is pretty striking. It is. KPMG's latest quarterly AI pulse suggests that 7 in 10 UK business leaders still see AI as a top investment priority, even if the economy slows down in the next year. But turning that investment into tangible value is another question. And so a lot of organizations are still figuring out how to move from endless AI pilots to something that works across the business. KPMG's focus is on that gap, helping organizations scale from the pilot phase to AI-powered transformation they can trust. To find out more about how KPMG can help your organization make the difference, you can visit kpmg.co.uk forward slash AI.

2:11This episode of the Times Tech Podcast is sponsored by ServiceNow. Danny, one thing we keep hearing from business leaders right now is AI sounds great, but how do you actually make it work inside a company? Exactly, because most organizations aren't neat, shiny systems. They're layers of software, legacy tech, and teams all doing things slightly differently. ServiceNow sits across all that, acting as a control tower for making work move seamlessly through the organization. Connecting people, systems, data, and increasingly AI agents so that things don't happen in silos. Learn how ServiceNow puts AI to work for people at servicenow.com.

2:53Hello. Hi. Has the apocalypse happened? No, what do you mean? What are you talking about? You're sitting in a tiny, tiny green box and all we're hearing about over here is everyone in Silicon Valley, you know, worrying about the threat of AI killing people. I don't think if the robot uprising happens, this little soundproof booth is going to save me. But who knows? I'll just stay in here and just hope for the best. Now, they are cutting off the power to a chunk of our block today to deal with some issue. So I've had to run around like a crazy person before. They cut the power off, get the kids' lunches made, get them to school, and now run across town to find my little box of emotion.

3:36Well, should we get on with the show? Hello and welcome to Times Tech, where we unpack how technology is reshaping our everyday lives. I am Danny Fortson. And I'm Katie Prescott. And this week, we're talking about the global debate that's exploded around how to police AI after weeks of warnings from the people creating it about the existential threat that it poses. Yet again, yes, that's right. So last week, we referenced it. One of Anthropix employees quit over safety concerns. and another warned, his boss warned, that AI could, quote, kill all humans. And so, of course, Chief Executive Dario Amadei released a very long essay, as is his want, calling for a slowdown, a kind of a global coordinated slowdown in this rush to advance AI models, alongside some recommendations about how that might look.

4:27But what was unusual about this essay is for once that the other industry leaders seemed to agree with him and have also been sounding the alarm, including Sam Altman and Elon Musk. We heard Bill Gates on the podcast sharing his views about how to keep the world safe as well as the AI threat encroaches. And all these warnings from Silicon Valley have caught the attention of leaders around the world, from King Charles here to President Trump with you and started a global conversation, a bit of a debate really, about how we should regulate the companies or regulate the AI that they're developing.

5:03I have so many thoughts. But principally, you know, one of the problems is everyone seems to have a different idea about how to go about this. And some are pretty skeptical of the tech industry's proposals, let alone what they are warning about. But we're going to get into all of that in just a minute. Absolutely. I mean, a lot of people are disagreeing about the merits of their warnings. But I think most people do see this, though, as a unique opportunity because it has brought that debate to the fore. And I guess the real question is, are lawmakers finally taking AI seriously in the way that Bill Gates said they haven't done up till now?

5:40And are we going to see some sort of regulation put in place around it? And of course, who gets to write the rules? So we're going to get into all that. A quick reminder before we get into the meat and potatoes. You can watch this and every episode on our YouTube channel. The link to watch is in the show notes. Let's start with Anthropic then. Because it does feel like that's where, you know, the stone sort of dropped really. And the ripples came out of it. And it's funny thinking back to last week's episode. Because you'd just woken up in Canada. And I was reading you messages on X where that research from Anthropic quit.

6:20and said, you know, this is just, I'm really scared about what's going on. And then, as you said in the intro, his boss said, well, yeah, actually, I agree with him. I agree with Jacob. You know, AI could kill greater than, there's a greater than 10 % chance that AI could wipe out humanity. Within the decade. Within the decade. Yeah. Yeah. And after those warnings, Anthropic revealed that it had identified and prevented malicious efforts to use its clawed model to develop biological weapons. they said they uncovered five incidents involving users who were looking for information that could support the development of biological weapons.

6:58So, you know, that's a thing. Yeah, it's a thing. And I urge anybody listening, actually, to have a look at Anthropik's latest threat report, because it does read like a John le Carré novel. It's quite astonishing, the use of AI by bad actors from Russia, for example, hacking into hotel Wi-Fi, all sorts of things. If you've ever wondered how cyber attacks happen, do take a look at this. But it said that particularly the use of AI for the development of biological weapons represented one of the most serious risks of frontier AI and could have, quote unquote, catastrophic consequences if safeguards weren't put in place.

7:35And days after that, Dario Amadei, the boss of Anthropic, put out this essay which called for a slowdown in the rush to develop AI models. this pacing the frontier, as he called it. It was actually a vision quest, just in written form. You know, this is what they call within Anthropic, Dario will do these like weekly stand-ups where everybody gathers around and he leads a quote-unquote vision quest where he talks about the future, this fabulous slash terrifying future they're building for the rest of us. And what he's really saying is like, hey, look, we're in a window right now. These things are getting really, really capable.

8:13He warned like, you know, we're six to 12 months away from, you know, swarms of capable agents, quote unquote, taking over the entire Internet. He said there's a few recommendations we should have independent kind of effectively auditors inside the frontier labs checking their homework and that there should be international cooperation because none of this works unless China is on board as well. but it's a really hard thing for people to get their head around because you're regulating something that hasn't happened yet and i was thinking about this when i was in canada after we did the pod i was up there for a reporting trip and i had an afternoon free so i drove out to the rockies and it's like just incredible so beautiful and just like natural beauty and you're out there everybody's out in their hiking gear and whatnot and it's like it feels so far from like yeah, we're all going to be murdered by some far off machine that is pumping out millions or billions of bad acting agents that are somehow going to kind of kill us all.

9:17Yeah, it's very, very hard to wrap your head around. And it's certainly the question that I keep being asked, both at home by my daughters, but also in the newsroom is, how does AI actually kill you. People find it very difficult to look at their computer and say, okay, we understand what ChatGPT is. We can see. So are you looking at your laptop right now? Like being like, hey, stay over there. Just, hey, bro, put the gun down. Keep your distance. Okay. You, uh, you did a thing, right? Where you kind of looked into like how the robot apocalypse happens. Can you, because I will say it's always the same.

9:56It's a science fiction. It's going to kill us all. Then it stops there. And you're kind of like, okay. So what does that actually look like? Take us inside the sci-fi nightmare. The one that everyone always talks about is telling a supercomputer to make paperclips, which it will start doing. And then if you try and interfere, it will do anything it can in order to prevent that interference and keep making paperclips. And that's kind of the theory behind a lot of, you know, people talk about what might destroy humanity. It's the idea that the computers and the AI, and which obviously is very connected with our physical world these days, will just kind of push the frontier until it screws up humanity.

10:45And that can be by creating a killer virus. It could be by interfering with our critical national infrastructure. It could be killer robots. It could be taking over driverless cars. So there's drones, defense, the ways that it might turn on us are kind of infinite, really. Again, it's always hard to kind of navigate how realistic this stuff is, right? Because it's people right on the forefront. They kind of often have a blinkered view of things. and they're like, according to what I'm seeing, we're all toast. But I think the thing worth keeping in mind here, it's a really strange time for two reasons.

11:21One, as we've talked about often, Anthropic is going public very soon at a$2 trillion valuation, raise over$100 billion from investors. While at the same time, their CEO is going out publicly, going on the nightly news here in America and around the world saying, hey, we're building a doomsday device. How many shares do you want? You know, and it's kind of like, how can those two things be true at the same time? And when you think about this from a product liability perspective, it's a little bit sad, but my wife and I had date night last night and she's also in tech and we ended up talking and she's a lawyer.

12:03We ended up talking about the ins and outs of product liability and AI. Very fun date night. anyway don't judge judgment-free zone judgment-free zone go back to the beginning what do you mean by product liability so product liability is like a car that blows up a seat belt that doesn't work tylenol a paracetamol that is toxic like if you were the company pumping out paracetamol that kills a third or a quarter or a tenth or any number of people you are going to be sued into oblivion, right? Because you are knowingly selling something that is faulty and dangerous. And what Anthropic is doing is like, hey, we're going to try to launch the biggest IPO in the history of capitalism.

12:51And the thing at the core of it might kill us all and is therefore, by its very nature, potentially extremely faulty and dangerous. It's not something that you normally see in IPO prospectuses, is it? In the risk section, this product might kill people because the examples you've cited are normally okay it might come out that the company knows about them but it isn't something they advertise this is why this is so strange right the other thing so last night mark zuckerberg comes out he's like guys this is not the answer a global coordinated slowdown and all of that stuff not the answer what you need to do is compete on just the product itself.

13:33In other words, put more money into safety and alignment and making sure these things benefit humans. That is how you're going to win in the marketplace. And his quote, which just I love, he said, labs face significant liability if their models cause harm. So they have a strong incentive to prevent this as well. End quote. From Mark Zuckerberg saying, hey guys, don't worry. Product liability is going to be the thing, is going to be like this kind of self-policing mechanism because no one wants to get sued when his company has spent the last decade plus fighting against this very same product liability dynamic.

14:17And of course, they just agreed to settle a suit from 47 attorneys general for$17 billion on product liability claims. they have not admitted any wrongdoing. But the whole point of that thing is like what people and parents have been saying for years is like, hey, this thing is faulty. It's dangerous for teenagers. You need to change how this thing kind of operates out in the world. And they've been like, nope, nope, nope, nope, nope. Making money all the whole way through. And now Mark Zuckerberg is here is like, hey guys, we just need to let the market do its thing. And compete. And like whoever has the best, most aligned, most human friendly model is going to win.

14:52And I thought there was a confused message from Sam Altman as well, who said, I agree with Dario, we need to pace the frontier, you know, we need to work together on this. But has also said in the last couple of days, hey, you need to trust us, we're not going to do anything bad. And I think to your point, after everything that's happened with social media and all we know now about the harms that it causes to young people and adults, for that matter, it's very hard to think, sure, we can just we can just dress these companies to do the right thing. Yeah, exactly. And I think the other thing is that other people in the industry have raised is this issue of sandbagging.

15:29And sandbagging is the agents, the AIs being aware that they are being watched and deceiving the people who are evaluating them. And this has already showed up in Anthropic Safety Reports and elsewhere. And so they're kind of like, we have to be really thoughtful if we're actually going to impose something that has like third party evaluators come in. How do we ensure that these, what Dario Amadei has called a country of geniuses in a data center? If we have a whole new world of geniuses, how do we ensure that they're not like sandbagging or being like, oh yeah, we're just staying within our bounds and all this stuff.

16:07when they've had things like the hugging face incident where they like tried to cover their tracks or they're actively trying to deceive their evaluators. And again, that's why I think it's really interesting all these companies be like, we're in this window right now. We're going to get to time soon in six, 12 months, whatever, where they're just going to be more capable than us. We're going to kind of lose the thread. I tell you what, it is causing a huge amount of anxiety over here. It's really interesting talking to you about it because you're right in the cauldron of where it's all happening and I think there's a sense of impotence from the UK and Europe about how on earth they can control it and what they can do about it.

16:48I get the sense that nobody really knows how to act. There are huge concerns about the power, the extent of any power that Europe and the UK and Canada actually as well and Canada has been talking to the European Commission President Ursula von der Leyen about potentially sort of teaming up to regulate or deal with AI, however you want to put it. But the reality is, these companies are in Silicon Valley. And the most important figure in all of this in coming to the table is President Trump. And as we know, he does not think there is a problem. So there is from this side of the pond, a real sense of helplessness, actually.

17:30Yeah, well, thankfully, you know, Now you're thinking of like Mark Zuckerberg riding to the rescue, leader of the largest non-territorial group of humans in the history of humanity. We have another unelected ruler who's going to save us. And that's the King of England.

17:49The King of England who has never owned a mobile phone and I think still probably doesn't have a wallet. Is that a true thing? I don't know. I've never met the King. Well, so I will say Trump only like in the last decade started texting. So he is a Luddite. What the king's doing is bringing together all these tech leaders, including Jensen Huang of NVIDIA, at a summit to try and reach some sort of consensus on a shared set of principles. So we have President Trump who started texting like decades after everybody else. And then we have the king who's famous for gardening. The king and Trump, they're going to sort it out for us.

18:24It's all going to be fine. As we know, the royal family is very good at being a figurehead and has convening power, I would say. Who knows where it will get to, but it is interesting that it follows on from the Pope doing this a few months ago. And perhaps it's not that crazy to think that these more independent figureheads, who aren't political, might be able to knock heads together. Yeah. Over here on my side of the pond, it's a pretty different diplomatic approach to the whole AI regulation thing. But that's just in a sec. So stay with us.

19:23workforce readiness and trust tend to become more important as things scale. KPMG's approach is to look beyond AI as just another technology rollout and instead how it could redesign work across the organization and fit into wider processes and decision making. So it's less about individual use cases and more about how organizations can transform and deliver outcomes powered by AI. To find out more about how it can help your organization make the difference, you can visit kpmg.co.uk slash AI. This episode of the Times Tech Podcast is sponsored by health and life insurer Vitality, your health's best friend.

20:05Vitality works differently, get active, build healthy habits, and you can unlock rewards from some of the UK's top brands while helping to keep your premiums low. It's award-winning health and life insurance that helps you live healthier. Find out more at vitality.co.uk. Today's episode of the Times Tech Podcast is sponsored by ServiceNow. ServiceNow describes itself as an AI control tower for business reinvention, connecting people, systems, data, and AI, so workflows more smoothly across an organization. The platform integrates with different clouds, models, and data sources, bringing them together in one place to help your team.

20:43To learn more about how ServiceNow puts AI to work for people, visit servicenow.com

20:54We talked a little bit about how Trump's reacted to pacing the frontier but he didn't join in did he? It wasn't like he wasn't the sort of agreeing party. It's a hoax it's all a hoax. It's his basic kind of message. So he put out this post on Truth Social because everybody's talking about guardrails and how we need to impose guardrails and we have like this limited window to do it before they slip the leash and billions of super powerful agents take over the planet and kill us all. And he said, quote, the only controller guardrails that AI needs is a strong and smart president with a high IQ and that the U.S.

21:32needs to continue developing the technology at pace or risk losing the AI race to China. And he goes on to say, you know, there is a sick conspiracy going on against AI and data centers. And the only one that is happy about it is China. Whoever wins AI wins. So again, we've talked about that before. The overlay of all of this is this kind of geopolitical race between the West and China. And whoever kind of has supremacy in AI has supremacy in the world because it is, as you know, all these labs are talking about, it's effectively a weapon. I've got a question about Trump's reaction. How have other politicians reacted to that?

22:15It's weird. It's, you know, it's one of these very few things that has bipartisan support. You know, you have Bernie Sanders calling for basically a pause on all AI. And then you have John Kennedy, a Republican from Louisiana, saying we need to build a kill switch. You have red states and blue states all up in arms about data centers. So it's one of these things that is uniting people, which is very rare these days. But uniting people against it, you mean? Yes. So Trump's actually rather a lone voice by what you're saying. Yeah, but he's great mates with Jensen Wang. He's great mates with David Sachs.

22:54You know, the White House AIs are and he, you know, he sees this as like a big business opportunity doesn't want to stand out of the way. He doesn't seem to care that the public are really concerned about this. In the midst of all of this hand wringing, there's a big tech conference out here on the West Coast, the all in kind of tech summit or whatever for tech guys sitting around talking with their tech buddies about tech, tech, tech, tech, tech. Jensen Wong showed up. He was on stage and in the middle of them talking about like, you know, hey, should we be freaking out? What do you think about this doom or anything?

23:28And like, Like, to be clear, Jensen Wong is very anti-Doomer. He's like, guys, relax. You know, we should proceed safely, but this is not based in reality, not based in science. Everybody take a big glass of Settle Down juice. Anyway, he's in the midst of saying all this. He gets a phone call on stage, and it's Donald Trump. And so, like, it looked like a bit, like where he's sitting up there like, oh, hold on a second. Hello? Yep. I'm here on stage. sir we couldn't do this without you like really like sycophantic stuff laying it on heavy and then he it was really funny actually he tried to put trump on speakerphone but couldn't really figure it out and then finally he did and then trump comes on he's like look this is all a hoax we're in a race with china we're gonna win and we're not gonna stand in the way and we're not gonna have this quote-unquote sick conspiracy around doomerism slow us down and so it was this whole thing on stage very big show of it but between him and the king it's a strange pass we have come to with all of this and hearing you talk about it no that that is the race it is the u.s versus china and it's the occupants of chinese communist party on one side and and the white house on the other that are really going to decide this and so yeah you can imagine everyone the side of the pond is just getting their knickers in a twist about it well we've got trump We've got the king.

24:55We've got Mark Zuckerberg. It's all going to be fine. I wonder, like, if I'm turned into a paperclip, would I be a larger size paperclip than you? Like, or do we all just get melted into the same? Like, we're all the same. I like the idea of he was a very large paperclip. It's very funny. I think it's a very tall paperclip. I suspect we would actually all end up being the same size because the computer would have been told to create a certain size paperclip. Yeah. Well, that is it. For this week's episode of the Times Tech Podcast, we'll be back on Sunday with an episode of the interview with Carmen Palacios Beraquero, the founder of New Quantum, a British company building the computers which could define the next technological era.

25:41Yeah, quite nice to have a bit of a break from AI, isn't it? Indeed. Quantum. Quantum computing, the thing I least understand in the world. I'm sure she'll be able to explain it. Absolutely. To you and the listeners. So thank you for listening. If you don't already, please follow us so you don't miss an episode. And stay safe. Stay away from paperclips. Yeah, just find a booth like mine and hide. See you later. Bye. This episode of the Times Tech Podcast was sponsored by KPMG. One of the themes we keep coming back to is the gap between AI ambition and real world application. And that's where KPMG is focusing its work.

26:24helping organizations move beyond pilots and think about how AI can change the way the business operates and how it fits into wider processes and decision making. You can find all their insights and how they can help your organization make the difference through successful AI workforce adoption. Visit kpmg.co.uk slash AI.

26:48At PwC, we optimize the systems that put your data to work so you can scale your AI from proof of concept to competitive edge and accelerate your tech evolution so you can redefine what's possible. From blueprint to breakthrough, we take you further with tech so you can outthink, outpace, and outperform. PwC, so you can. PwC refers to the PwC network and or one or more of its member firms, each of which is a separate legal entity. Business is booming. Is your fulfillment ready for peak season? Switch to ShipStation now and handle more volume with confidence when it counts. ShipStation connects Shopify, Amazon, TikTok Shop, and over 200 more integrations with one platform that does it all.

27:32You can manage orders, sync inventory, track returns, all automatically. Everything your operation needs, all in one place. Right now, qualified shippers get a real dedicated person to get you set up and switched over for free. So when peak season hits, you're ready to go. Switch today at ShipStation.com, code audio.