Why is Anthropic asking people to fund AI doom?

0:00This episode is brought to you by Capital One. Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called Chat Concierge and it's simplifying car shopping. Using self-reflection and layered reasoning with live API checks, it doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trade-in value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One. This episode of the Times Tech podcast is sponsored by KPMG. Katie, we've been tracking the AI surge for a while, but some of the latest data is pretty striking.

0:42Danny: It is. KPMG's latest quarterly AI pulse suggests that 7 in 10 UK business leaders still see AI as a top investment priority, even if the economy slows down in the next year. But turning that investment into tangible value is another question. And so a lot of organizations are still figuring out how to move from endless AI pilots to something that works across the business. KPMG's focus is on that gap, helping organizations scale from the pilot phase to AI-powered transformation they can trust. To find out more about how KPMG can help your organization make the difference, you can visit kpmg.co.uk forward slash AI.

1:22This episode of the Times Tech Podcast is sponsored by ServiceNow.

1:26Danny: Danny, one thing we keep hearing from business leaders right now is AI sounds great, but how do you actually make it work inside a company? Exactly, because most organizations aren't neat, shiny systems. They're layers of software, legacy tech, and teams all doing things slightly differently. ServiceNow sits across all that, acting as a control tower for making work move seamlessly through the organization. Connecting people, systems, data, and increasingly AI agents so that things don't happen in silos. Learn how ServiceNow puts AI to work for people at servicenow.com. Hello, how are you? So it's eight o 'clock in the morning.

2:07I've just been like scooping out peanut butter, cutting up apples, getting lunches ready. I like look all bedraggled. and then you come on post HMU, as you say, hair and makeup, looking fabulous. I just feel like we've got to do something about this. Is there, do we have budget so that like, we can send some people, we can send some people, or maybe I should just do like, find a good YouTube tutorial where I just kind of put myself together, you know,

2:34Danny: contouring. That's what it's called, Danny. You need to buy a contouring kit to really enhance your cheekbones. And then maybe a little, little, little something. Yeah. Should we get to the pod? Let's do it. Yay! Hello and welcome to Times Tech, where we unpack how technology is reshaping our everyday lives. I am Danny Fortson. And I'm Katie Prescott. And this week, in a very jolly turn of events, we're being told that AI is yet again going to wipe us all out. Huzzah. Yeah, of course. It's just another week. Yes, the whole debate around AI safety has resurfaced again in a different way. and yet for various reasons which we will soon come back to we're a little skeptical about the plans to rein it all in and it has happened though the long-awaited anthropic prospectus got this week the document that we've all wanted to see basically kind of look inside the bonnet of the business except it's a really astonishing piece of work because it's mainly ai doom and warnings about how the tech that they're developing poses an existential risk to humanity.

3:41Yeah, not your typical S1. But that same document is asking investors to help them finance$518 billion of AI infrastructure. So half a trillion dollars, one company that's losing money, which we'll come back to. But it's really interesting to see because this is the first time we've actually started to just get a glimpse, as you say, under the bonnet of this company, looking at the AI boom. So we'll try and square all of that in just a minute.

4:12Danny: One of the most important companies of the world, and clearly its valuation has a massive impact on others as well. And as warnings have been stacking up and up and up, Donald Trump called a meeting with tech bosses to discuss this idea of regulation that he's been pushing back on for such a long time and they didn't actually agree any regulation but they did sign a self-regulation pact which we're going to come back to later in the episode and we also thought we'd look at today what feels like a huge disconnect between the people who control tech this tiny group of men in silicon valley and the rest of us the rest of the world who are listening to these mixed messages I think with alarm, dismay and confusion.

5:02Danny: And frankly, I think people are getting a little bit tired of it all. Yes, indeed. Yeah. So last week, Saturday Night Live, a very famous show out here, obviously released just a perfect sketch mocking Dario Amadei of Anthropic, his safety warnings. And I think it really just like captured it's like three minutes. But it really captures I think how people are feeling and how they view these companies at the moment. I heard a wonderful female tech investor this week who knows them quite well refer to them as misfits with messiah complexes. Yeah, that feels right. Anyway, and a quick reminder, you'll get not just one, but two episodes from us this week.

5:46Danny: And this Sunday, it's an interview that Danny's done with the boss of the Silicon Valley vibe coding company Replit. That's right. Yeah. So please do check that out. And he's got some really fun kind of surprising takes on all the doomrism coming out of AI. And he's been in this world for a very long time. So he knows what he speaks. But anyhow, back to today's episode. It's all about Anthropic, the IPO, the strategic leak, I would call it, of parts of it to Reuters. Yeah, it's an interesting one, isn't it? Because it's obviously been seen by some investors. And some of that investor base, you say strategically pushed it out to the media.

6:23Yeah. So just to kind of step back for a second. So Anthropocon's IPO, they're going to wait until after the November 3rd midterm elections here in the US. And if you look at kind of what has trickled out, there's some kind of confusing or kind of head scratching numbers, which we'll get into. But also they apparently spend 80 pages on what they call quote unquote risk factors and you've looked at you know probably a million s1 documents these you know these documents you have to file with the sac before you go public um so have i you know risk factors are always a thing because they have to be like hey here if you want to buy this stock

7:03Danny: here's every single thing that can go wrong and i remember famously ubers when it said we may never make any money right so they have to kind of like be uber transparent right but this one is a little bit different because Anthropic is, you know, I would argue is trying to pull off not only the largest IPO in the history of capitalism, but also potentially the strangest because the risk factors include potential, quote, catastrophic or existential risks to humanity of the thing they're building and want all of us to fund. Which brings us to the numbers and they're trying to raise$100 billion at a$2 trillion valuation.

7:49Danny: So that's the price tag that they're putting on this technology that could potentially wipe us all out, which as you say, they spend 80 pages detailing. And it's this really bizarre situation, which I think is confusing people. It's something I've been asked about a lot in the last week, actually, is why are these companies, and in particular anthropic pushing the negatives of AI when we've been told so much in the past about the amazing things that AI could do like like we heard from Bill Gates cure all sorts of diseases advance science in really positive and exciting ways erase inequality but what they focus on are all the negatives at the same time turning around not quite with a begging bowl that's the wrong analogy but you know just saying to people hey give us your cash well I think you know Dario friend of the pod.

8:38I was listening. I went back. He came on December-ish, 2024, a million years ago in AI years. He and his founding team, I think, all come from the same kind of almost like cultish belief that AI, you know, it's so much more than software. It is this thing that can change the trajectory of humanity, but it's also wildly powerful. And the existential threat is very, very real and they feel it is on them to kind of build it in the right way because they don't trust other people to do that but when you look at there's two things i think you need to look at one is the philosophy behind it this strange company that's actually warning that their product may be disastrously faulty and two as a business last year they brought in sales 4.6 billion.

9:29They lost on an operating basis over 8 billion with a net loss at 42 billion. But most of that was like an accounting thing. So this is a deeply loss making company. They're growing incredibly fast, but they also plan to spend half a trillion dollars on cloud computing, data centers, etc. And they're just going to keep losing money.

9:50Danny: Although they would say, I mean, they made a small operating profit, didn't they? And again, these are all these like strategic leaks. But even that was like, they had carved out, you know, small details like the cost to train their models, you know, the cost to build the product. And you're like, okay, well, you know, if I don't have any expenses, I am rich. But to that point then, Dani, why not, as I got asked this week by someone who's not in the tech world, why not then push the positives and say, yes, we've got to spend half a trillion dollars on data centers to make this work. But look, hey, our revenue was 12 times what it was the year before last year.

10:33Danny: So as you say, we're growing really rapidly. This is the rosy future. it does seem bizarre and maybe comes back to this point about misfits with a messiah complex or as we've said before the slightly odd group of scientists that are running and working at this company that this is what they choose to focus on well partially i get it insofar as that they're hacking all of these systems open ai anthropic all these kind of all these systems are just breaking out of their quote-unquote sandboxes hacking into companies scraping government data that is happening today and what do we what do we all get is like a really cool chat bot or something that helps us do research so in a way you're kind of like yeah i i get i get it but also it gets to this stranger thing of like i don't know if you saw the new york times did a big piece this week right i was reading right before we got on where anthropic is gathering religious leaders on a regular basis get a nice private room and a shishi san francisco restaurant to talk about morality to talk about how do they imbue their ai with moral like with a kind of human morality and the reporting that came out was like a lot of these rabbis priests people came out and be like, they're talking about this software.

12:03And that's what it is, as like a moral being that might have intentions and feelings. And, you know, that there's some equivalence that they're effectively growing a new species. And so again, I think those are genuinely held beliefs inside the company, which gets you to this weird place where we're like, yeah, we're growing this new species, and it's going to be more powerful than us pretty soon. And we're freaking out.

12:27Danny: But this is what happens when you have got a group of people in a very, very small part of the world living in a bubble with no perspective. And Anthropic has always been very focused, haven't they, on this idea that Claude has a personality and they've been very sort of thoughtful, focused about how they train that personality. so it's not that much of a leap to the moral but also as i've heard from people within the company the idea that it has some sort of consciousness it's fascinating though isn't it do you remember not that long ago within google there was that engineer blake lemoyne that's right exactly who believed that ai had got some sort of conscience yeah and ended up leaving the business yes indeed And now you have like the people are not only inside the tent, they are the kind of the bosses and they are pushing this.

13:26And I think, again, genuinely believe it.

13:28Danny: I would say, though, there is a happy medium to be trodden here. So if you look at someone like Demis Sabis, now the chair of Google DeepMind, but the co-founder of the business, who has founded a company called Isomorphic Labs here in London. And they published a blog this week about building a new path to make medicines with AI. And I feel like he has managed to look at both sides of this very reasonably and say, yes, we know there are risks here. But also, guys, here is the path to a magical future. And Anthropic doesn't seem to be doing that. And I don't quite understand why. Now, you send a lot of joyous clips from the US and things for us to watch and read.

14:11Danny: But I think one of the best things you've actually sent this year is that Saturday Night Live clip that you mentioned of Dario Amadei. So we thought we'd play it for fellow British listeners who may not have stumbled across it because it is very good. I may have been alarming in my recent interviews, but I want to assure you that if we can pressure lawmakers to create guardrails, we will be able to stop me. can you just explain how your company isn't going to kill us so so so so so so so that's a really hard question it shouldn't be so you know ai is the devil and i it's maker what if you're not watching that it's the way he twitches when yeah it's so good it's so good but like I said just like comedy is like just has a way of cutting through all the nonsense but I think it's that point it's like another line is like you know I'm confident that we can save us from me or something like that where you're just like you are the boss you are the boss Sam Altman is the boss but I can't help it we've just gotta keep building this thing but it's terrifying exactly anyway I just I've I love that little bit of levity but after the break we're going to get just a tad bit, just a tad more serious.

15:40Danny: No, we're not.

15:46This episode of the Times Tech Podcast is sponsored by KPMG.

15:50Danny: We often talk about the shiny object problem with AI, where things look impressive in isolation, but are much harder to roll out across a large organization. Issues like governance, workforce readiness and trust tend to become more important as things scale. KPMG's approach is to look beyond AI as just another technology rollout and instead how it could redesign work across the organization and fit into wider processes and decision making. So it's less about individual use cases and more about how organizations can transform and deliver outcomes powered by AI. To find out more about how it can help your organization make the difference, you can visit kpmg.co.uk slash AI.

16:36Today's episode of the Times Tech podcast is sponsored by ServiceNow.

16:40Danny: ServiceNow describes itself as an AI control tower for business reinvention, connecting people, systems, data and AI so workflows more smoothly across an organization. The platform integrates with different clouds, models and data sources, bringing them together in one place to help your team. To learn more about how ServiceNow puts AI to work for people, visit servicenow.com.

17:27and estimate trade in value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One.

17:42Danny: I don't really know how to begin with this. Is any of this going to lead to some sort of regulation from the US? As we heard from Bill Gates a few weeks ago, one of the reasons he thinks there is no global regulation is because there's no leadership from the White House in order to do that. And Donald Trump was like, hold my beer. He's like, all right, watch this, guys. So Donald Trump invited the tech leaders to the White House to talk about AI safety. Mark Zuckerberg, Greg Brockman from OpenAI, Dario Amadei, Elon Musk was there too, although not at the dinner, I think. Anyway. Sundar Pichai from Google.

18:22Danny: And so he brought them all together and they signed a one-pager. And I don't know about you, I'm sort of losing track of how many documents these guys have signed about AI safety, because there was definitely one at Bletchley Park. I'm sure there was one at India in the AI security summit. Anyway, so they arranged some voluntary safeguards. But you sent us another extraordinary clip that many listeners may have also watched, which was the press huddle that Trump held afterwards. Yeah. Which starts with a very awkward Dario Amadei. Yes. Sounding a little bit like his Saturday Night Live self with Trump praising him.

19:01Yeah. You know, there's a press gaggle, they're outside the White House. And then he kind of just puts people on the spot like Dario, why don't you say something? Then it was Mark, why don't you explain what we've done here?

19:11Danny: It's just like a bunch of boys in school and he sort of pulls them forward. And then Dario kind of shuffles back awkwardly to the back line and they're all smirking at each other and it's just like oh my god these men are running the world and it's just like the whole thing was strange but like all of this was about what a one-page thing signed by trump zuckerberg sundar pichai musk amaday to basically agree to a self-policing regime that is completely non-binding. It's not a law. It's not an executive order. Trump said he it would be quote unquote, morally binding. And then we're supposed to kind of come away with this being like, Oh, whew, because you know, the tech industry has a long, long history of doing really well at policing itself.

20:05See social media.

20:06Danny: And when we're talking about this really consequential technology, it's terrifying. The crazy part is that, you know, it all feels very seat of the pants. At the same time, as we were just discussing, these things are already committing felonies, you know, like against the wishes of their creators who are still pushing them out anyway. OpenAI said this past week, like, oh, we're actually not going to push out our latest model because we're worried about safety. But then there's other stories about like they basically have just ignored safety warnings from their own people and pushed out their last model.

20:38So like, it's just, we're all kind of on the train. It's going at a hundred miles an hour. And all the guys in the front driving the train are like looking at each other and be like, are you going to slow us down? No, I'm not, but you should. No, but you should. But oh, we should all slow each other down. And you're like, this is just not going to work. And this agreement, you know, it talks about bringing in third party evaluators and how you know being more transparent and kind of keeping tabs on each other but it's all just feels you know yet another talking shop and this one was a very odd photo op but not much else incredibly awkward super awkward there's there's a really lovely phrase in daniel susskind's new

21:29Danny: book what should my children do which is all about what kids should do in the age of ai where he describes being at an AI conference. This is a British academic who writes a lot about AI. And he describes being at an AI conference and looking around and the sort of feeling of, well, I hope the leaders or the people who are making decisions are in another room down the corridor and realizing that it's actually all on their shoulders. And I had that feeling watching that Trump press conference. You're like, oh, I hope the decisions are being... And by the way, the answer to your question is woodworking, apparently, which is actually a thing I was told by somebody at OpenAI.

22:11It was like, because I think I mentioned it before, I met a bunch of their top team. They're all very young, very nice, very earnest, obviously very smart. Only one of them, I met like six people, only one of them had a child. And I asked them that question of like, well, you know, I've got a seven-year-old, nine-year-old, what should I have? And they're like, well, it's a good question. I mean I just think the future is going to be awesome that was more more than once I was told

22:37Danny: that and then it was maybe they should yeah talk to Anthropic the future is going to be awesome and then the other thing was the one guy was like well my brother's really got into woodworking and I feel like that's a thing that's like you know gonna stand the test of the robot uprising and I was like cool cool like I feel great now you're like what but again also I just think I just think this place is such a bubble inside a bubble inside a bubble the world is big and messy and slow moving indeed i can tell you the power of inertia is incredible you know it's just like dario said you know a couple years ago that we'd lose 50 of all white collar work and it's like nothing even remotely close has happened to that and he was practicing that like happening imminently because they're like well i've built the machine it is clearly better at thing xyz and abc therefore for, everybody's going to use it.

23:33And you're just like, no, that's not how the world works.

23:37Danny: And we need people just because some work can be automated. There's still a huge amount of humanity that's needed around that. And if you read any of the labor market economic surveys that OpenAI or Anthropic have done, they talk about exposed occupations, right? You know, occupations that can be automated. But actually, they then conclude that when you look at previous disruption, whether that's, you know, the China trade shock, or any other big, you know, labor market disruption, it doesn't happen in the way you necessarily expect. Because life, as you say, is incredibly messy. And it isn't just like, just because you can do something doesn't mean that you should.

24:19Danny: And we know, because we're asked all the time about adoption of AI in businesses, like businesses just aren't using it really a lot of them are still sitting on their hands because they're concerned about cost it's still evolving like they don't know where the next innovation is coming from i couldn't believe it instinct bridget my new agent they just raised a billion dollars the company's like six months old man i guess this is the question i guess this is a question when we're talking about the largest infrastructure build out in the history of the world by value is Bridget worth it? Because right now I'm guessing that Bridget, maybe a couple other tools when you write in your book or whatever, prove very useful.

24:58But like, is that what all of this is for? Especially when you hear folks like Dario Amadei be like, well, the downside is civilizational extinction. You're like, but Bridget, you know, she's worth it. Norm, who was my Muse agent, I had a little moment this week. Creepy. I received an email about one of my kids and it was from a teacher and it was like nothing major, but it was like a personal thing. And Norm sends me a message out of the way by, hey, have you read this? And it was just like this moment. I was like, I don't. And this Muse is owned by Meta. And that was just like, you know I don't put my kids on Instagram but I'm giving meta access to my email so I can have more convenient reminders and also they may have all this information about my kids my family my because your email inbox is like treasure trove and I was talking to my wife who's as you know is a lawyer in tech she's like get rid of that thing she's like I don't want meta or muse or an to know about like our kids.

26:04Danny: Yeah, I think you're, I mean, I think you're absolutely right. I've still got Bridget going and she's got my bank details now and like buys stuff for me. So, you know, maybe that isn't wise. Is the juice worth the squeeze? Anyway, there we go. A note to end on. There you go. Lots to cogitate on over the weekend. So we've got the Times Tech Summit coming up, which is our big annual tech event that Danny and I host every year. we have had some superstars in the past, including Sademis Sassabis, among many others. And we've got a fantastic lineup this year. I'm really excited about it. We do. And it's in just over a month now.

26:43Tech Summit.

26:44Danny: Our annual Times Tech Summit, which you're flying over for. You know what? They'll give you hair and makeup for that. I will get HMU. Oh, yeah. Yes. I mean, maybe not a blow dry. That's just cruel. That's just cruel. But contouring. Yeah, I'm going to get some contouring. I can't wait. I can't wait. But more importantly than your hair and makeup, the lineup is looking incredible. I'm really excited. It's going to be really fun. And we're going to be up there. We're emceeing like one of the great, you know, Showtime duos. We're going to be up there being the host for the day. It's going to be really fun.

27:17Danny: Fully contoured. Fully. I'm going to be HMU'd to the hill. I can't wait. And it's, yes, it's in London in a month. November 10th. November 10th. Get your tickets before they're gone. So that is it for this week's episode. We will be back on Sunday with the interview I did with Amjad Massad, who is the boss of Replit. One of these like super, it's growing like a weed, a vibe coding company that's gone from doing okay to doing fabulously very, very quickly. And he has some really interesting takes from kind of inside the Silicon Valley machine that I think you guys will really enjoy. Here's a clip.

27:56The question is like, if you believe AI is going to kill everyone, why do you want to build it? And so these companies ultimately are commercial entities and they'll just follow the commercial logic, but they're dressing it up in this like effective altruism sort of language. Now to get to the objective claims, I think that there's no good argument for how AI would kill everyone.

28:22Danny: Thanks for listening. And if you don't already, do follow us so you don't miss an episode. Here or on the YouTube. Go find us there. See you next week. Bye-bye.

28:39Danny: This episode of the Times Tech Podcast was sponsored by KPMG. One of the themes we keep coming back to is the gap between AI ambition and real world application. And that's where KPMG is focusing its work, helping organizations move beyond pilots and think about how AI can change the way the business operates and how it fits into wider processes and decision making. You can find all their insights and how they can help your organization make the difference through successful AI workforce adoption. Visit kpmg.co.uk slash AI.

29:18This episode is brought to you by Capital One. Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called Chat Concierge and it's simplifying car shopping. Using self-reflection and layered reasoning with live API checks, it doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trade in value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One.