No Mr Blair, AI will not save us | Karen Hao

0:00Your team just added its 67th AI tool and also your 67th security blind spot. The good news? The Vanta agent works like a GRC engineer in the background, finding every app your team uses, scoring the risk, and drafting fixes for you. Vanta is the platform used by over 16 ,000 fast-moving companies like Ramp, Cursor, and Harvey, who are shaping the future with AI and staying ahead of AI risk. Get started at Vanta.com.

0:30The New Statesman.

0:37Hello, Karen. Ollie Dugmore of The New Statesman here. Looking forward to seeing you later in the studio and talking about an issue no smaller than the forthcoming technological revolution. There's obviously a lot for us to talk about, but I really like your metaphor, to be honest with you, about how AI's point of reference and point of comparison should in fact be an empire, that an imperial lens is the one that's most appropriate for understanding what's going on in these companies. Sam Altman as a figure, you know, why do people love him? Why do people hate him? As well, there is so much for us to talk about.

1:20AI, obviously, increasingly a topic of political contestation as much as technological. and also the resist list. You know, it's really encouraging or interesting, I should say, examples of people who are resisting what's going on. Like I said, looking forward to it. Take care. Hey, Ollie. Thanks so much for the note. I am very much looking forward to our chat as well, and I will see you in the studio. What lessons might be learned from Sir Anthony Charles Linton Blair? The importance of being humble, that to abandon your base is to abandon the long-term vitality of your party. Always maintain a healthy suspicion of Americans and Scots.

2:05Blair believed in the relentless march of progress, that the forces of modernisation were uncontestable facts of nature. El Nino of the Golden Arches, the topographic squiggle of the S &P 500 as Everest, but higher. If only Britain could harness those forces, or even just draft in their slipstream. I hear people say, we have to stop and debate globalisation, he once opined. You might as well debate whether autumn should follow summer. And he also believed in himself. As John Kampfner aptly charted in Blair's wars, the millennial prime minister led Britain into more conflicts than any of his modern counterparts.

2:45He turned a pacifist political party into a belligerent of five wars in six years. And his early successes in Kosovo, in Sierra Leone, led to the development of a personal hubris. A messianic psychology that met its ruin in Basra and the bloodstained desert around it. He might not have read Faust, but you'd expect Mr Blair to have a degree of familiarity with his own modern parable. Alas, he might not have read Faust, but you'd expect Mr Blair to have a degree of familiarity with his own modern parable. Alas, an essay published by his eponymous institute proves otherwise. The Labour Party is playing with fire over its future and the future of the country makes a strident case for a politics of the radical centre.

3:31The futility of personality-driven leadership contests with no care for policy and the inevitability of the impending, indeed currently happening, technological revolution driven by advancements in artificial intelligence. If only a galaxy-brain visionary were able to guide Britain's puckered mouth towards Silicon Valley's teat. Blair's essay is uncompromising about Jeremy Corbyn's leadership of the Labour Party, which is described without irony as an intellectual wasteland. Modernisation is destiny, and after all, ours is not a reason why. Particularly not about the colossal£257 million in donations to the Tony Blair Institute from one-time world's richest man Larry Ellison, whose company, Oracle, has major commercial interests in the adoption and future success of artificial intelligence within government institutions.

4:21When it comes to tech policy, Oracle and TBI are inseparable, one former senior advisor at TBI told the New Statesman last year. A TBI spokesperson said, TBI and Oracle are two separate entities. We collaborate with Oracle to help the work we do in supporting some of the poorest countries in the world, and we are proud of this. TBI does not advocate for Oracle's commercial interests, nor does it advocate for any tech provider. In his latest, Blair writes, technology policy is the critical factor. If European policy continues in the direction of addressing the dangers rather than seizing the opportunities, i.e.

4:58weights technology regulation against the technology sector, it will be impossible for Britain to go back fully into the European Union. We can't argue that technological innovation and adoption is the key challenge of modern governance and tie ourselves to a technological environment and tie ourselves to a technology environment essentially hostile to it. Very well, except the benign magnificence of artificial intelligence is not an immutable fact. On the contrary, there is a significant and growing body of work detailing the working practices of the companies at the head of this revolution. It does not make for pretty reading.

5:31One of the people spearheading this field is the American journalist and author Karen Howe. Last year, she published New York Times bestseller, Empire of AI. It's central framing. Imperial power is the only appropriate comparison for a company like OpenAI. From the pursuit of physical resources like land for data centres and its surrounding water table, to extractive labour practices that pay content moderators in Kenya only a few dollars a day. One worker described the videos produced by an AI as worse than a horror film. Recent news reporting has revealed forthcoming likely lucrative IPOs for several AI companies.

6:09One funding raised by OpenAI aimed to haul in$7 trillion. That's more than the total GDP of the UK, Germany or Japan. In fact, the total output of the world's 150 poorest economies would just about equal that sum. Howe thinks it's a strategy that then confounds people. And in that state of confusion, the company is continuing to move at breakneck speeds to do what it needs to do. How is suggesting that not only are these companies seeking domination on a global scale, they are deliberately obfuscating their intentions. So, the next time a political has-been, backed by millions of pounds from a billionaire with vested interests in AI, suggests the best indeed only adequate response to a technological revolution is deregulating the sector in order to properly suck on its divine milk, perhaps think a little, just a little, about the blood on the sand.

7:03You're about to listen to our interview with Karen. It was recorded just before Tony Blair's essay was published. In fact, I think we were sat in the studio right as it was released. So while it would be unfair to interpret her remarks as a direct response to any of what Blair wrote, neither of us were aware of its existence while we were talking, you can very definitely contextualize his thoughts within her work. I hope you enjoy the conversation. Karen Howe, you argue that AI isn't just another tech boom, it's something more than that. What is it? I argue that tech companies like OpenAI are new forms of empire.

7:41And the reason is because to think of them merely as businesses that provide products and services is fundamentally under-describing the actual role that they're playing in society. These are entities that have consolidated an enormous amount of economic and political power. They do so through the dispossession of the majority. And they are developing a controlling influence on every single facet of our lives, politics, geopolitics, work, relationships, everything. Okay, so let's drill into that metaphor then, the one of empire. And let's be specific. What is extracted from whom? You said dispossessed.

8:25Tell me about it. Yeah, data. They take the data of private individuals, intellectual property from artist writers and creators, land for hosting their data centers from communities around the world, minerals for building those data centers from communities as well, water for cooling the data centers as well as you need fresh water to generate electricity for powering the facilities. And so there's all of these different types of both digital and physical resources that are being taken from as many places as possible because of the sheer hunger that these empires have for more. It's really interesting.

9:02Obviously, the digital side of it is a relatively modern phenomenon. Typically, when we talk about empire, we're thinking in a slightly historical sense. But very definitely, those historic empires, they were explicit and they were deliberate about the acquisition and the extraction of natural resources. Yeah. How explicit and deliberate is it within these companies? Are they thinking in the same way that, I don't know, the Belgians might have been thinking about rubber or the British might have been thinking about spices? Is it as deliberate and specific and targeted as that? It really is. Yeah.

9:34I, you know, when I was interviewing people within OpenAI, they think about the world sort of through maps the same way that Empires of Old did, where they're just thinking, where do we get more of the resource that we need? particularly when it comes to data centers, because the land requirements and the power requirements are so high. And they are looking for where in the world can we get an already developed electric grid that can deliver that much power to a single place, as well as have a nice large plot of land next to it that isn't affected by natural disasters, no floods, no hurricanes, no tornadoes.

10:13And so they do scope out the globe based on these very high level ideas about, you know, how do we assemble all of the inputs that we need to continue building our technologies? That's really quite unsettling. I mean, so we've seen this obviously explosion in AI development and adoption. What does winning look like within those companies? Is it we need to strike some sort of deal here for some sovereign AI capacity for this nation so that we can get a bit of a footprint to put a data center there? Is it a pursuit of artificial general intelligence? What does victory look like? It's a great question because you'll get different answers from different people.

10:59But the official narrative that the companies tell publicly and internally is that they are all pursuing this idea of artificial general intelligence, which is this theoretical AI system that's going to match human capabilities one day. The problem with this notion is that we don't have a definition of AGI because we don't have an understanding of human intelligence. There's no scientific consensus about what actually constitutes human intelligence. What is it about the human species that seems fundamentally different from every other species? And so essentially these companies in this vacuum of meaning are able to just define and redefine AGI to be whatever they need it to be.

11:43Which is why you get such differing answers depending on who you ask about what the ultimate goal is and what constitutes winning. So by my count, OpenAI has used at least four different definitions of AGI. One is on their website, which is highly autonomous systems that outperform humans in most economically valuable work. So this is a labor automating definition. They're just saying it, you know, we're coming after the highest paying jobs. In their contract with Microsoft, they say it's a system that will generate$100 billion of revenue. So it's a financial definition, of course, very useful when you're trying to court an investor to give you a lot of money.

12:23and with congress sam altman said it's a system that will cure cancer eradicate poverty solve climate change so it's a societal definition when you're trying to ward off regulation and to consumers open ai says it's the best digital super assistant that you never thought you needed and so it's trying to motivate them to purchase more of their product so this is basically you know by by the end of the book i conclude that that empires always have to have this kind of vague civilizing mission, this thing that helps justify why they're engaged in so much dispossession, why are they engaged in so much labor exploitation.

13:03AGI is the perfect civilizing mission because it can just shift to wherever they need to put the goalposts to help them justify the next step in their conquest. Obviously, as well, Empire has this undertone, overtone of pseudoscience, racial superiority, and racism. Is there a kind of parallel that you've noticed in the same way that there's this sort of really quite disgusting ideology? More so, obviously, I'm not saying the extractive component parts of it aren't disgusting, but how do you think about that component part of Empire? Yeah, I do think that there's very much elements of that because when you think about where they began to do their extraction and exploitation, I talk in the book about how they went to Kenya to find the workers that would ultimately perform content moderation for their models.

13:55And they also found workers in the U.S. to do this kind of work. But the kind of work that they sent to Kenyan workers versus American workers was fundamentally different. The Kenyan workers got the worst kind of work, the ones that were psychologically traumatizing, whereas the U.S.-based workers were getting still, you know, not great work. But the content wasn't literally causing PTSD for some of the workers. And so you can already see this kind of hierarchy begin to establish itself. Of course, also, there's a hierarchy between these workers and then the workers within the AI research labs that are paid million dollar compensation packages.

14:34You know, you could argue that there's really no difference between the value that AI researchers bring to the company and to data annotation and content moderation workers because all of them are doing work that is essential to the existence of the very products that these companies are selling. If any one of them stopped doing what they did, there would be no Chachubutu. There would be no Gemini or Claude. And yet the AI researchers get their million dollar compensation packages and the workers in Kenya get paid$2 an hour. And so once again, you know, there's no possible justification for this other than a deep seated belief that some groups of people are more worthy than others.

15:18Yeah. Can we be specific about this at the moment? because I think most of the end users of a platform like ChatGPT, they just sort of picture it, conceive of it as this magical piece of software where they write a prompt into a chat and it replies to them and it gives them this stream of information. And what you are detailing, and I'd like you to talk more about it, is that there is actually a really quite significant hidden human cost to a platform like ChatGPT. So could you just give us a sense of the infrastructure? I know you mentioned it a little bit there about content moderation and stuff, but what does that look like?

15:52Is it happening at scale? How does it work? Yeah. So everything that an AI model can do, like the fact that chat GBT can chat is because there were tens of thousands of humans that actually taught the model what dialogue looks like. They were feeding in examples of person A's talks, person B responds, person A says something else that's kind of related and so on and so forth. And when like the training process is essentially, you know, there are AI researchers that take a bunch of data, feed it into the model, but then there's all of this refinement that goes into making these models actually feel human-like.

16:32And so those workers, there's one category of work where these workers are instructing the model over time how, in fact, to behave to different types of prompts. And they'll upvote and downvote the models when the models are not getting the behavior right. And then there's this other category of work that's content moderation. Both of these categories are very exploitative, but exploitative for slightly different reasons. So the first category of work is outsourced to people all around the world. And it is extremely piecemeal and it is gigified knowledge work. It's basically like the Uber of knowledge work.

17:12So essentially, workers don't know when the work is going to arrive, when it's going to go away. And the way that they do this work is by creating accounts on third-party platforms that are kind of like the Uber for connecting the AI companies to these gig workers. And these platforms pit workers against each other. So when the work arrives, they have to, within seconds, claim the work in order to even get the opportunity to do it. And so I've interviewed workers for over eight years now that are part of this kind of system. And they have their entire life revolves around this work. In fact, I used to interview workers that did this work overseas and in impoverished countries.

18:01And now this work is the fourth fastest growing job in the US. And so I've been interviewing workers recently that are college graduates, PhD graduates, lawyers, doctors, scientists, people that have even years of professional experience under their belts that were then laid off because their company used AI to justify some kind of shrinking of the workforce. And then they're literally turning to this job that pays them no benefits, really crappy compensation. And they're tethered to their computer because they just don't know when they're meant to actually get that work and then need to compete with the other workers.

18:43So that's one category. But then you have content moderation, which I alluded to earlier. And with content moderation, the reason why there is even a need for content moderation is because these companies are training systems that they market as everything machines. You can do anything on it. You can ask it any query. It should have an answer. And to train these types of systems, they're scraping every possible corner of the Internet. And that includes the darkest corners of the Internet. And so how do you then deal with the problem of having a text generation machine that's supposed to be able to generate anything, but also not have it regurgitate all the toxicity that you've also put into the model?

19:28That is where you then hire workers, including the Kenyan workers that I spoke to, to act as human shields for your users. And so those Kenyan workers that OpenAI contracted to do this for what eventually became ChatGPT, they were waiting through eight hours of the worst kinds of content that humanity could ever possibly create. And they were putting it into a detailed taxonomy of is this hate speech, harassment, violent content, sexual content, does that sexual content involve the abuse of children? And it completely eroded their mental health. It, in some cases, unraveled their families because individuals that break down then cannot support the community structures that they belong to.

20:15And it is absolutely devastating. It's probably for listeners just worth considering, you know, in our legal system, I think is a good point of reference for that sort of material. How is it processed and how is it handled? and most police forces have specialized units with specialized staff who spend oftentimes years training, both in terms of sort of how to do the detective work, the criminal work, the digital forensics, but also obviously then the necessary emotional strengthening, like how to cope and deal with it. And that is obviously a world away from what you are describing, isn't it? And even then, I've watched these people being interviewed.

20:59And even when people have chosen to pursue that as a career path and they're very well paid for it and they receive all the training, you still look at it and go, what that must be doing to your psychology is difficult to understand. And then not only that, when you consider the way you've just described it, I mean, they are worlds apart, aren't they? Yeah, they are. And, you know, I was interviewing. So I mentioned that there's more and more of this work happening in the US. So I actually interviewed a U.S.-based college graduate recently that was doing this kind of content moderation work. And he said something really important to me, which is there is no way to consent for this kind of work.

21:37Because, of course, you know, companies will say, we ask. We ask if workers are willing to be exposed to this content. But he was like, when you have a machine that's literally supposed to be able to generate anything, you cannot possibly imagine what it's going to throw at you. Like he said, I've watched horror movies. You know, I thought, oh, gore, violence, whatever. Like I can deal with that. And then the kinds of AI generated videos that actually came up on his screen just scarred him for life. Like he just had no way of realizing that that was what they were talking about. And so there's that element of it.

22:19But then, of course, for the Kenyan workers, yeah, they didn't have mental health support. I mean, the company that they were working for called Sama said that they offered it. But workers told me that these therapists would come in. They would have group therapy sessions with like eight workers at a time. They didn't actually know what the workers were doing. So it was this very crude degree of support where workers also were really scared of showing vulnerability because that could mean that they would be pulled off the project and then they wouldn't have the economic opportunity needed to put food on the table.

22:56And so this is the exploitation that happens is companies know that there's always going to be a really desperate labor force somewhere in the world and increasingly more because of the very technology that they're creating that will be willing to accept this job and be forced by their own economic vulnerability to continue doing it even when it's corroding their mind. We're going to take a break now.

23:23Your team just added its 67th AI tool and also your 67th security blind spot. The good news? The Vanta agent works like a GRC engineer in the background, finding every app your team uses, scoring the risk, and drafting fixes for you. Vanta is the platform used by over 16 ,000 fast-moving companies like Ramp, Cursor, and Harvey, who are shaping the future with AI and staying ahead of AI risk. Get started at Vanta.com. Thumbtack presents Uncertainty Strikes. I was surrounded. The aisle and the options were closing in. There were paint rollers, satin and matte finish, angle brushes and natural bristles.

24:02There were too many choices. What if I never got my living room painted? What if I couldn't figure out what type of paint to use? What if... I just used Thumbtack. I can hire a top-rated pro that knows everything about interior paint, easily compare prices and read reviews. Thumbtack knows homes. Download the app today.

24:24let's get a little bit specific now let's talk about open ai and sam altman specifically i mean where did he come from what was he doing before open ai tell me about him sam altman is a stanford dropout that spent his entire career in silicon valley before open ai he founded a startup that was not very successful called looped it was a mobile-based social networking platform, kind of like a find my friends type situation. And after spending several years doing that and then selling his company for essentially the same amount that investors put in, he catapulted himself into becoming the president of Y Combinator, which is a very prestigious startup incubator in the Valley that he then, as the figurehead, turned into an even greater center of mass for the entrepreneurship ecosystem within the Bay Area.

25:20And he essentially is very much a, he is a, he's a, he's a product of the Silicon Valley culture of building technologies. One of his mentors is Peter Thiel and Thiel famously or infamously established this idea that anyone building a startup should aim for monopoly. Like the goal is to create a business that no one else can compete with and that really dominates a particular space. And so you can see this within Altman's behaviors. And he used to write extensive blog posts where he talked about how he thought about his different endeavors. And at one point he says during his time at Y Combinator, we are going to invest in 10x more companies every year until we invest in every single company in the world.

26:13And there's this kind of, you already see this kind of imperialist mentality in that sentence. Because first of all, that doesn't make any sense as a strategy. Like most companies fail, why are you going to invest in every company in the world? But there's this drive to make everything under the sun mine. And when he stepped down from Y Combinator to then become the CEO of OpenAI, he said, the thing I'm most proud of is that we built an empire. For our British listeners, you mentioned in your answer there, the culture of the Bay Area. For people that might not necessarily be familiar with what that means, can you briefly, but could you just explain what is that culture?

26:55Very cutthroat. There's a winner takes all mentality. It's a zero sum game. If someone else is winning, you are definitely losing. And so that is the aggressive aim for monopoly type situation where the only metric for success is to be the last one standing. It's kind of flavoured, there's often flavoured with a degree of kind of utopianism as well, right? There are these grand ideas about, well, as evidence with, as we've already spoken about, what AGI might end up being, but effective altruism, other versions as well. There is there's this kind of like veneer of philosophy sometimes that gets painted over that.

27:35Yeah, absolutely. There's this kind of, everything in Silicon Valley is wrapped with this high-minded idealism. And there's a worship of the market. So there's this belief that capitalism is the best vehicle through which to achieve the highest ideals in society. There is really nothing else that should help you get there. So by aiming for, you know, Peter Thiel says in a lecture at one point about this concept of aiming for monopoly, that you like monopolies are good because it demonstrates that you have delivered value to the world. And so it is, there's no gap between building a really large scaled business and good, being a good actor in society let's talk about elon musk then um so how did altman's relationship with musk develop because they co-found the company together yeah how does it develop yeah so altman is much younger than musk and altman has said before that when he was coming up in the valley he he revered musk he He thought of him as really as a figure that he aspired to be like.

28:55And so he just reaches out after I think there was a connection that was facilitated between one of their mutual friends. And essentially after that, he reaches out to Musk and says, I'm thinking of building this AI lab. I've been listening to some of the things that you're saying about how you're very concerned that AI development is being dominated by Google as a for-profit company. I completely agree with you. I also think that the best way to prevent Google from really dominating the space is simply to build another lab and compete. And to put a stake in the ground and make it different, let's make it a nonprofit.

29:37And we will try to pursue, you know, the ultimate quest of the AI fields, build a kind of Manhattan project for AI. And through a few email exchanges, he gets Musk on board. What's interesting is I think a lot of media coverage paints Musk and Altman back then as good friends. That was not what my reporting found. They were professional acquaintances. They ran in overlapping circles, but they never really socialized together. It was more just Musk seemed to think that Altman was very aligned with him in a particular thing that he wanted to accomplish. And so they end up forming this organization together.

30:30And Musk is the one that names it OpenAI as a call to the transparency that the organization will uphold. What's their relationship like now? They hate each other. they essentially after co-founding the the organization they as a non-profit they realize a couple years in that they in fact need to have some kind of for-profit vehicle to raise more money because they are pursuing a path that will be very expensive and based on this idea of building monopoly aiming for number one they need more cash than they need a lot of cash to to specifically outspend Google. And they have a tussle over who's going to be the CEO of this new for-profit vehicle.

31:19And Altman ends up winning. After, interestingly, Greg Brockman, at the time chief technology officer and Ilya Setskever chief scientist, initially chose Musk to be the CEO. So through these discussions, Altman becomes CEO. Musk is like, I'm out if I can't be in control of this entity. And at the time, he wasn't really bothered by that because he thought that OpenAI would fail. Then OpenAI starts to succeed, at least in the way that Musk defines success. He becomes really upset by that. And then he sues Altman, OpenAI, Greg Brockman on the basis that they stole a nonprofit from him. They were never supposed to convert it into a for-profit and they tricked him into providing a lot of funding for the initial organization and his star power for recruiting the initial team.

32:18They went to court. The court recently determined that Musk brought the suit too late and based on that technicality, Musk lost. We're going to take a quick break now. Remember, you can listen to the New Statesman podcast ad free by downloading the New Statesman app. It's available on iOS and Android. Links are in the show notes. On this week's episode of The New Society, we speak to political theorist Jan Werner Muller about the relationship between design and democracy. Subscribe to The New Statesman today and get your first six weeks for only six pounds. Go to newstatesman.com forward slash six weeks to subscribe today.

32:54We'll be back after this.

33:0367th AI tool, and also your 67th security blind spot. The good news? The Vanta agent works like a GRC engineer in the background, finding every app your team uses, scoring the risk, and drafting fixes for you. Vanta is the platform used by over 16 ,000 fast-moving companies like Ramp, Cursor, and Harvey, who are shaping the future with AI and staying ahead of AI risk. Get started at Vanta.com.

33:33I think the most recent figures I've seen about attempted raises, planned future raises, the figure I saw was$7 trillion for OpenAI. I mean, how, what, like, who has the money? Where does it come from? That is, you know, we're talking figures that are larger than most of the world's economies put together, right, at this point. And as I understand it, we're looking at revenues in the tens of billions for OpenAI. So, you know, how are those two things reconciled? How does that work? What is happening, Karen? Make it make sense to me. Yeah. I mean, the first thing that I'll say is that I think it's a strategy of these companies to throw out numbers that large, trillions, because most people have not come across trillions of anything, let alone trillions of dollars.

34:24And that then confounds people. It makes the number unfathomable. And it's worked on me. Yeah, it is part of what then, you know, in that state of confusion, the company's continuing to move at breakneck speeds to do what it needs to do. It does not make sense. And you are right to point it out. And when you see it, it helps explain why OpenAI appears to have a very chaotic strategy. It's because there isn't really much of a strategy. The company is trying to figure out any possible way to monetize so that it can close this gap. The reason why it's setting up for IPO later this year is because it's already tapped out private markets.

35:13It's raised more money than has ever been raised before in private markets. And now it needs still more money. So it needs to go to the public markets through an IPO to try and milk more of that cash out of the system. But you see them rolling into ads. You see them talking about hardware devices. There's collaboration with Johnny Ive, the famous designer of Apple products. They tried to release a browser. They had their video generation tool store, which they then had to shutter because it was taking up too many resources. I mean, they are trying a cover the waterfront approach. They're just throwing spaghetti at the wall to take any opportunity to figure out what their business plan is because there is none.

35:59Well, if I could sort of invite why he is, Sam Altman, such a polarizing figure. Because whilst there are detractors, there are people who dislike him. I think people listening may think that's a relatively reasonable position to take. but there are people out there who are deep, deep and sincere admirers of his. Talk to me about that juxtaposition. Talk to me about that tension. Would it be fair to describe it as a kind of like a cult of personality? Is that too far? No, there's absolutely a cult of personality around him. But the thing that I realized is I interviewed over 250 people for the book, half of whom were communities around the world and half of whom were people within OpenAI or around OpenAI or around Sam Altman or across the other companies.

36:45And what I realized over time was someone's position on Altman was almost exclusively determined by their own belief about how the future of the world should be. So if they agree with Altman about his vision of the future, they think he's one of the greatest leaders of our generation because he is an incredibly persuasive person. He's very, very good at understanding what motivates people, what they desire, what they fear, and then using the resources that he has at his disposal as a billionaire to try to get that for them. And so he's able to mobilize mass amounts of resources, mass amounts of capital, talent, and just general energy towards a particular trajectory.

37:39And so that's amazing. If you agree with his goals, you know he's the best asset to have on your side ever. And when people work for him and also share those goals, they say that his ability to motivate really makes them do their best work. It opens up their own imagination about what they are actually capable of. And so that is why he develops such a deep loyal following among a certain group of people. And at the same time, equally so develops a, like, other people then for the very same reasons say that he's manipulative, abusive, a threat to humanity. And it's the same persuasive power because for people who don't actually agree with where he's going, that persuasive ability is really, really unsettling.

38:37And many people leave OpenAI or leave his orbit feeling like he tricked them over time into working for a vision that they actually fundamentally disagree with. there's probably two events i think that encapsulate that right one is anthropic and the spin-off and the other is the attempted ouster of him from open ai and then his real his reinstatement a couple of days i think later yeah in that he has these sort of like devout loyalists who want to work with him and then others who like you just said are so concerned that they're trying to set up actually that's a good question so what is the degree of difference between Anthropic and OpenAI?

39:15Because that separation, that split, is allegedly, as you've said, one about vision, one about safety. Is that what's happening? Are those concerns valid? To me, OpenAI is coal and Anthropic is clean coal. That's the difference between the two companies. And there's not a coincidence. I mean, Anthropic's not the only one that splintered off and created its own competitor under the guise of being a more ethical actor. Ilya Setskover left and created Safe Super Intelligence. Mira Mirati, who is the chief technology officer after Greg Brockman, left to create Thinking Machines Lab. Elon Musk, of course, now has XAI.

40:00And every single one of these AI startups positions themselves as morally superior to all of the other AI startups. And when you look at what they're doing, they look very, very similar. And OpenAI and Anthropic are the most similar because Dario Amadei, the CEO of Anthropic, and his executive team, almost all of whom were former executives at OpenAI and splintered off with him, they were part of the core group that architected OpenAI's strategy in the first place. And then they just copy pasted it for Anthropic. This is really interesting to me. You used the word in that answer, guys. And it's fascinating the degree to which or lack thereof that these people are sincere.

40:51Do they believe in these visions about the potential of artificial general intelligence? Have they drunk the Kool-Aid? Is it just a marketing spiel? Is it naked and explicit and extractive underneath and in reality? Are they deceiving themselves? I find that psychology so interesting. What's your view on that? Do they believe it? Do they believe what they say about artificial general intelligence? I like to think of the AI world like Dune, because in Dune, Paul Atreides understands that there's been a myth seated on Arrakis about the coming of the Messiah. He steps into that myth knowing it's a myth because he wants to use it to control the people.

41:32As he lives and breathes and embodies this myth day in and day out, he begins to lose track of the difference between myth and reality. He begins to wonder maybe he is the Messiah. Maybe because of his unique breeding, he in fact has now just made the myth come true. And this is the reason why I use this metaphor is because, you know, I have friends that have worked at OpenAI and have then left. I know people because I've been covering this company since 2019. I know people who what they thought when they first joined the organization and what they thought after years of being at the organization.

42:14And there's this phenomenon that happens where people start off with their own independent views, even skepticism about what OpenAI is doing, what AGI, you know, this quest for AGI. And they enter the organization and within a matter of months, maybe a couple of years, they become like everyone else and become a believer. And then, you know, I've had a friend who then left and then looked back and was like, whoa, that was weird. How did I suddenly, you know, become exactly the same as everyone else? And I think the reason it happens is because, you know, people at OpenAI or any of these companies, they're very smart people.

43:01They're very sincere. They're mission driven. They actually spend a lot of time talking about these ideals and philosophy and what does it actually mean to be doing good in the world. And when you're around these people and you develop a deep respect and affection for them, it becomes really hard not to mirror their beliefs. And there's this unifying, homogenizing effect that begins to happen. And I think it happens for the executives as well, where you could say that Sam Altman has demonstrated through his actions that he's very politically savvy. He understands what rhetoric to use to manipulate and control the people.

43:43But in order to wake up every single day and mobilize a multi-thousand person company, to mobilize the public, to mobilize policymakers, and also to be around these incredibly smart, thoughtful people that are telling you, yes, AGI is a thing. Yes, we are building this thing. You cannot possibly then exist in this space and not begin to believe it. You know, so maybe there was a time when these executives recognized that there was a rhetorical utility to this myth. But because they live and breathe and embody it every day, the myth blurs with reality and they begin to think, well, it might have been a myth, but I'm making it come true.

44:31you know so every single one of them exists in this kind of in-between space where i don't think they really know themselves what they believe what is true and what is myth from the course of the conversation so far there are uh plenty of moral and ethical issues about what we're talking about as you've just even mythical issues about what we're talking about but before we start talking about the resist list in a second. I just want to think about a practical issue, which is that for all of these grand designs, there is kind of a limiting factor, isn't there, in the environment and the planet that we inhabit, that we may have these people that wish to achieve extraordinary power and extraordinary wealth, but they are bound by the finite nature of where we live and the power, the water.

45:30You mentioned it earlier on in the interview. It's not infinite, is it? No. Yeah. They're limited by physics. That would be so frustrating. You know, I think it really does frustrate them. Why aren't I as powerful as God? God. I mean, you know, it also ties back to many of these people are deeply obsessed with longevity and wanting to meet death itself. And there is across every one of the endeavors that they spend their trillions on, it's this idea of ultimate control, control of nature, control of human life itself. You know, they, I think it comes from this obsessive, maybe it comes from an obsessive fear of uncertainty.

46:20They need everything to be exactly how they intend it to. Karen, tell me about the Resist List. Who are you working with? How it began? What is it? What do you want to achieve? Yeah, the project called the Air Resist List is a collaboration between a group of journalists, critical scholars, and AI researchers. One of the organizations that we partnered with is based in the UK. It's called We in AI. They're an amazing organization run by Tanya Duarte. and it is a project that came out of a need to understand better why AI is not inevitable because people who have been reporting on this technology for a long time and in my work, you know, all of it wraps around this idea that these technologies are designed by people.

47:12There were choices that were made and they could have made different choices. But where does that leave, you know, the people all around the world that are not billionaires that feel in their day to day lives that they don't actually have power of this system? What actions can they take? What can they do to actually shape the future trajectory of this technology if it is not inevitable, if we can, in fact, change it? And what we did with our collaboration was we began looking for different types of actions, resistance actions that people were taking around the world, whether individual, community, institutional, political, artistic.

47:51And we ended up documenting around 30 different initiatives from almost every single continent, maybe every single continent except for Antarctica. We're not sure if there's life on Antarctica. We were talking before we started recording. I'm sorry, belive my ignorance to the entire audience. But yeah, most consonants, most consonants, Karen. Most consonants. Yeah. To just demonstrate that there's resistance happening everywhere. It comes in many different forms. And it can be just individuals doing things that actually have an effect on the public conversation. And so one of my favorite examples is this community that I write about in the book, Quilicuora, Chile, they are one of the communities in Latin America that is the most extracted from in terms of data center build out.

48:43There's all of these companies going in and trying to build data centers in their community. And it uses their energy and their freshwater resources. And I wrote about the community because they had this amazing protest culture that they had developed to resist against these data centers. And they took it further after I published the book where they decided to create this website called Keeley.ai. And when you go on the website, it looks like a chatbot and you can type into the query box and then an answer comes back. But instead of an AI model generating the answer, it's actually being written by someone in the town.

49:23So what they did for 12 hours, they said, anyone can use this human chatbot to ask about anything and we will match your query to a townsperson. So if people were saying, I want a drawing of a dog, they would send it to their local artist named Benji. Benji would draw a dog on paper, take a photo and send it back. And it was their way of basically raising awareness to first tell people, hey, this costs something. Every time you engage in a bit of mindless, frivolous, AI-generated chatbot conversation, it costs someone's electricity, someone's fresh water, and it costs the land that was taken to host these facilities.

50:10And they also wanted to create this conversation about why is it that we keep turning to AI for these types of queries when we could use it as an opportunity to reconnect with the experts within our own community, to rebuild our social relationships. You know, the very antidote to the kind of dehumanization that the AI Imperial Project tries to undertake. And they went absolutely viral and did generate a huge international conversation around the things that they wanted to raise awareness about. So that's just one out of the 30 that we document. There's something really quite beautiful and joyful and specifically chilean about it reminds me of the um there's that i end a quote that's like um the revolution will be flavored with red wine and empanadas it's it's you know it's just i thought it's fantastic my favorite one was the poison one you know the the yeah that for listeners that i think it's a university of chicago researcher and you but effectively if something tried to trawl or scrape art artists can install i think it's a piece of code right and it basically sabotages slash attacks makes it very difficult hurts the AI that's doing that and I was quite pro that one I enjoyed that one a lot um yeah I just on the point about that community and its data centers the UK now the applications that we're facing for data centers if they're all approved and they all get built the amount of power they require is more than the power that we currently use as an entire nation yes and also the UK government recently quietly acknowledged.

51:43Well, I guess didn't acknowledge. They quietly updated their figures around the projected carbon emissions for all these data centres, which they had underestimated by 100 times. And the amount of carbon emissions that all these facilities could end up generating would be equivalent to 2.7 million people. Karen, I asked you earlier what winning looks like for these companies. What does winning look like for you? I really want people to recognize their own agency and to recognize that they do in fact have a seat at the table and can shape the future of AI development and just the future of technology development and the future in general.

52:25Because AI, as it's currently conceived by these empires, is not just a technology. It's a political project and the central feature of this political project is taking everyone's agency away and turning these entities into monopolies on decision-making power in the world. And if I can help people restore that sense of agency and make people feel the co-ownership that they rightfully have over our future, over this technology's development, that would be the greatest thing in the world. We're out of time. Thank you so much for sharing yours with me. The conversation has been fascinating. Karen Howe, thank you so much.

53:10Thank you so much. You've been listening to The Exchange with me, Olly Dougmore, and journalist and author Karen Howe. This episode was produced by Catherine Hughes and Beba Kang.

53:32AI tool, and also your 67th security blind spot. The good news? The Vanta agent works like a GRC engineer in the background, finding every app your team uses, scoring the risk, and drafting fixes for you. Vanta is the platform used by over 16 ,000 fast-moving companies like Ramp, Cursor, and Harvey, who are shaping the future with AI and staying ahead of AI risk. Get started at Vanta.com.