How the tech bros of AI are breaking our democracy

0:00If the world were like a Sleep Number mattress, everything would adapt for your comfort. Because as your life changes and your body changes, Sleep Number mattresses adapt and shift to give you personalized comfort night after night. And now it's the final days of our Everything's on Sale event. Save up to$1 ,200 on mattresses. Our Memorial Day event ends Monday. To experience a whole new world of comfort, visit a Sleep Number store or go to sleepnumber.com. Sleep Number, to a good life's sleep.

0:32This is a Global Player original podcast. OpenAI, when it came along, what was it? It was a non-profit. It was a non-profit. Which seems hilarious, no? Yeah. AI is a political project. Meta, which was a social media company and now an AI company, the kind of AI data center that it's constructing now in Louisiana is nearly 400 times the size and footprint of the first data center that it built to support Facebook. If someone is saying you cannot stop one of the forces that will come to shape how the future looks, then what's even the point of democracy after that? It is a truism that AI will dominate our lives, that it already is, that there is something inexorable or inevitable about it.

1:18As Tony Blair has said in his recent essay that artificial intelligence will change everything. I mean everything. thing. There is no point in debating whether this technological revolution is a good or a bad thing. Just know it is a thing. In fact, he says it is the thing. As if he were talking about the weather. But what if it isn't true? AI companies like to present what they're doing as Blair's done, as somehow being beyond politics, as beyond political arguments. They talk as if they are engaged in a utopian project to turn the wheel on humanity's development. Indeed, to reinvent that wheel and for that wheel to reinvent itself ad infinitum, creating a world of abundance, permanent revolution.

2:07It is a convenient fiction for politicians like Bear to cling to, a get-out-of-jail-free card for something, anything, to come along and rest us from our political malaise and economic slump. And the problem with our treating them that way, in talking about them that way, is that we don't see them for what they really are. And for my guest today, the journalist and writer Karen Howe, what they are are not companies or religions or brain trusts. What they are is empires. Something which rises above even the great corporate power that we saw in, say, the railway companies of the 19th century or the energy companies in the 20th century or the financial companies of the early 21st, that these firms are accruing and accreting such profound political, financial, cultural and resource power, such a chokehold over knowledge itself that they are best understood as early imperiums.

3:08She believes that the test of the 2020s and beyond for democracy is whether they can be brought back under democratic control to make what they do work for us and not for them. Her book Empire of AI is out now and seeks to answer that question and more. A simple proposition at its heart. We can have AI empires or democracy but not both. Why then are our politicians so unwilling to recognise it? Welcome to the News Agents.

3:46The News Agents. Karen, welcome to The News Agents. Thank you for having me. We've been devouring your book. I'm going to start by saying I'm going to try and pay you the highest kind of journalistic compliment that exists, which is that, you know, reading your book and then reading some of the things around it. It's both a wonderful thing and a really slightly irritating thing, which is like you've basically written a book that I have sort of for a while thought should exist and even thought if it doesn't exist maybe someone should write it and the thing is you've written it already and you've written it far more brilliantly than anybody else ever could which is basically treating AI companies not as kind of and the sort of AI project if you like not as sort of corporate history or sort of starry dewy-eyed kind of tech evangelism or sort of or the opposite, but actually is what they are, which is deeply political actors, right?

4:35And that is what your book's about. Thank you so much. So just before we get onto the kind of defining the terms a bit, which I think is important, why did you want to write that book? Why did you want to write a book which treated them politically and as political and economic actors? I think it's the only way that you can understand what is actually happening and how we as a society should be thinking about the products of this political project. and I was very privileged in my journalistic career that I started covering AI in 2018 before the information about AI became really chaotic and messy and when ChatGPT came out in the world and suddenly everyone was suddenly trying to understand this technology I felt like if I had just begun my journey then what is the kind of book that I would have wanted to help me introduce me to this world?

5:25So you were there at the beginning, effectively, or at least at the beginning of the modern iteration anyway. Roughly the beginning of the modern iteration. But yes, AI has been around since the 1950s. So not all the way. But in terms of a lot, you were around the cast of characters that we've become or a lot of us have become familiar with and the companies and the work like OpenAI, Sam Orton, these sorts of people. You were sort of in their orbit at that time. I was following them quite closely at a time when they weren't really well known by the rest of the world. And just remind us, because again, we think we sort of band this around a bit.

5:59OpenAI, when it came along, what was it? It was a non-profit. It was a non-profit. Which seems hilarious now. Yeah. Co-founded by Elon Musk and Sam Altman. And the idea at the time was they were going to be a fundamental AI research lab, not work on any commercial products. And they would open up all of their research to the world. and of course now they're you could argue one of the most capitalistic companies in the world they have raised more money from private markets than pretty much any company ever and they are just about to prepare for an IPO later this year that could value them at something like a trillion dollars and what made why did they make that transition i mean obviously in one sense it's obvious because they're about to do an initial public offering at a million trillion dollars but But they started, did you think that they genuinely started with an altruistic intent or is it all just a bit of a smokescreen?

6:51I think the problem with these organizations, because it's not just OpenAI, it's also Anthropic, XAI, all of these AI startups, they intermingle mission with capitalism. So there's such a culture within Silicon Valley that doesn't really distinguish capitalism from social good. They believe that the markets drive the best socially good outcome. And so when they started, they intended to create something that would actually beat Google. That was the way that they defined it internally. Google was the only game in town that was really dominating AI development. They did not like that. They wanted to create some kind of competing lab that would supersede Google quickly.

7:33Google was a for-profit. So it was a good story to create a nonprofit and recruit a bunch of researchers to something that appeared fundamentally different and then accelerate their path from there. But the reason why they now look very similar to, you could say, what Google looked like before it became a public company, which is just a normal corporate entity that is trying to amass an extraordinary monopoly, is that once they got the talent and the mission had served its narrative purpose, then they just needed money. and they needed to figure out a way to raise that amount of money. So they created a for-profit vehicle that became the priority and essentially consumed the rest of the non-profit.

8:17And Altman and Musk parted ways during that process. So when they created the for-profit vehicle, Musk and Altman had disagreements over who should be the CEO of this new for-profit. They're such retiring characters. Yeah, you can't imagine that. You have it, Elon. No, you have it, sir. and when musk didn't successfully get the ceo position because greg brockman chief technology officer and ilia seskever chief scientist at the time first chose musk but then ended up flipping to altman musk departed he was like i don't i don't want to be part of an organization where i don't have this level of control at that time he was fine with that basically what happened with Donald Trump a little bit later on.

9:00Exactly. The federal government. Exactly. At the time, he was fine with that because he didn't think Opening Eye would succeed. And then Opening Eye became very successful under Musk's definition of success. And then he became very angry. And in a way, that brings us, doesn't it, to kind of the title of your book, which is, or rather almost the title, because you talk about empire of AI, but in a way, I suppose, in a sense, it's empires of AI. That's right. It's not competing empires. That's right. And before we get on to that, in a way, I think like just breaking down, even just the sort of the main words of your title is really sort of important and interesting.

9:28isn't it? Because it sort of tells us a lot. First of all, let's actually just get really back to basics. I think a lot of listeners and viewers, sometimes we sort of bandy all these terms around and we sort of imagine everyone knows what they mean. But very often the people talking about them, something we'll go on to discuss, don't know what they mean. I think one of the most interesting things that you point out in your book and in your writing elsewhere is that AI in itself is both a highly contested term, even the bits of it, the artificial and the intelligence are highly contested, and kind of means everything and nothing.

9:59So what does AI mean? Does it mean anything? Or is it actually better understood as a sort of different set of competing things? Yeah. So I like to use the analogy that AI is like the word transportation. Transportation is actually a concept and it refers to a collection of technologies, everything from bicycles to rockets. And what unifies those technologies is they get you from point A to point B. And AI... But you would never say like, I'm taking the transportation to the airport today. Exactly. Exactly. And with AI, it's also a vast collection of technologies that are conceptually tied together based on the idea that they're performing some kind of automation of different types of human capabilities.

10:39So there's computer vision that automates our ability to see. There's chatbots that automate our ability to converse. There's generative writing tools that automate our ability to write and speak and so on and so forth. um but that was the original project of ai back in the 1950s when a group of scientists came together and said let's create a discipline that seeks to essentially recreate the human mind and recreate all the things that humans can do but in a machine but even then in the 50s as you talk about even the even talking about is artificial intelligence that wasn't the only name that can that was discussed, right?

11:19Because as you've talked about, that intelligence is even a sort of contested concept, right? Absolutely. Which is, I wasn't actually aware that there is actually sort of no scientifically agreed definition of what intelligence is, even in the human mind. Yeah, that's right. Let alone in the artificial mind, so-called. So therefore, if artificial intelligence is about trying to replicate something in the human mind, if we don't even quite know what intelligence is in the human mind, then how can we hope to replicate it? Exactly, yeah. So back in the 1950s, they first tried to call it automata studies.

11:54And John McCarthy, who was an assistant professor at Dartmouth, chose then to rename it artificial intelligence when no one gave a crap about a discipline called automata studies. It's not very catchy to be fair. It's really not. And decades later, he says during a televised debate that I coined the term or no, sorry, I invented the term artificial intelligence because I needed some money for a summer study. And he was referring to the very summer study in which he gathered up all the scientists to initiate this new discipline in 1956. And so it's, you know, he admits it right then and there that AI was coined as a marketing term.

12:32And it was meant to be an evocative narrative that would bring in money, but also bring in scientists because they wanted to rally a bunch of scientists around this new ambitious concept. concept. And that idea of recreating the human mind, recreating human intelligence has now perpetuated all the way to present day with how OpenAI and all of these other companies articulate their purpose. So there wasn't a nobler ambition, even in the 50s? I mean, you could say that, you know, that as scientists who didn't probably never imagined the extraordinary commercial supercharging of the technologies that they were developing, that they were ultimately just trying to explore the human mind.

13:18So what's interesting is there's many different types of people that come to the AI research field, or did at the time when it was an academic discipline. Some people came to it because they were deeply curious about machines and the capability of machines. Others came to it because they actually thought that it was a vehicle through which to better understand the human mind. The neuroscience. Yeah, because they were struggling to figure out how does the human mind work So we can use neuroscientific approaches, biological approaches, and we could also use computational approaches. If we could recreate something that appears and acts and behaves like the human mind, does that mean we've actually discovered the underlying mechanism of our own intelligence?

14:00And does it largely then go into abeyance for a long period? I mean, it's not something always, I mean, maybe it was, I just, I'm not aware that it's something that's talked about a great deal in the 60s, 70s and 80s. but it does seem obviously to have had this renaissance over the past decade or so. Is it the case it went into abeyance? And why does it appear to have that renaissance? Is it just technological change or is it something else? So there was indeed what's called an AI winter. Some people will say that there were multiple AI winters, and this term refers to a time when the investment for AI dried up.

14:31And part of the reason was there were different ideas around how to recreate human intelligence because they were different ideas of what human intelligence is. One branch was humans are smart because we know things. So let's create machines where we symbolically encode knowledge into the machine. And that's how we're going to replicate human intelligence. And the other branch was humans are smart because we can learn things. And so let's create machines that learn. And this is the concept of machine learning. And in the initial days or decades of AI research, it was really focused on this symbolic branch.

15:05branch and that really wasn't panning out and that's why the funding dried up and every pretty much every modern AI system that we see today including chat gpt and all other large language models now derive from the second branch the machine learning branch and we talk about that all the time don't we machine learning machine learning is the thing that trips off the tongue of a lot of the people you talk about in your book exactly and the idea was well we learn from experiences, but machines learn from data. So let's create computational systems that calculate patterns within data and derive some kind of understanding, quote unquote, about the rules of the world through the data that we feed these machines.

15:49And the reason why it took off is twofold. One is that it's an effective, you know, when you throw a bunch of data at a computer, it does find patterns and the and just that act in and of itself is highly useful and you throw more data in it it learns more patterns you throw more data more patterns and the other thing is it's particularly useful to companies because so much there's so much commercial utility and pattern matching and so in 2012 google became the first company to recognize this is something that we could commercialize and make money out of. And why is it so useful for companies to pattern match?

16:28For example, the most important business model that both Google and Meta use is ad targeting. And to target a user with the ads that they will click on is a pattern matching problem. If you have a bunch of data about their interests, you can better predict what the likelihood they might click on an ad about clothing or pets or whatever it is. And that is the most successful business model that has ever been invented on the internet. You know, that makes Google hundreds of billions a year. And so they recognized that early on. And in 2012, they then started funneling an enormous amount of money into machine learning research, which then became deep learning research, which is a derivative of machine learning that uses more powerful software for computing the patterns out of data.

17:18That software is called neural networks. and that is why all of a sudden there was this massive renaissance in AI research, AI development. It's because companies realised that they could profit from it. And Google, I suppose, particularly at that time, has not a monopoly, but something probably approaching it for a lot of markets on unbelievable quantities of data. Exactly. And then presumably its problem is it doesn't quite know how to process it and amplify it and use it best. So AI is literally, to some extent, or this renaissance you're saying is to some extent born. out of that commercial imperative, despite, again, all of the kind of highfalutin rhetoric about kind of transforming humanity and everything, fundamentally, at each stage, it's about marketing and it's about commercial benefit.

18:02Yeah, I mean, they realized that this, if you can create the rules of the game in a competitive landscape to correlate with how much data you have, then the actor with the most data is going to win. And Google realized that was their competitive advantage. If they can lean into the technology that extracts more value out of large quantities of data, they're going to win. And so they really indexed heavily on it. And then of course, once Google moved into it, then Microsoft moved into it, then all the tech giants moved into it. But modern day, the AI boom, or the post-GPT AI boom, isn't just about profit, which is what's so fascinating and why I use the empire analogy, is it's this fusion of profit and ideology because actors like open ai who are the new generation of companies coming onto the scene they are motivated by this original conception of the field this idea of recreating the human mind and this belief that if you could recreate the human mind it is somehow going to unlock the next era of civilization well i want to come on to that and particularly the imperial bit which i think is especially fascinating before that just on that, I also find this completely fascinating.

19:18I did a special episode of News Agents last August. We went to San Francisco, spoke to a load of AI companies, including Anthropic, who let us in. And we interviewed Jack Clark there, who's a very senior at the company. And I sort of got onto this question with him, which I think is really fascinating. Because obviously the thing that they talk about, and listeners and viewers may have heard of, is this concept of AGI, or Artificial General Intelligence, which is the sort of next stage. And you can kind of get a bit lost in kind of what that means isn't exactly what it is. But fundamentally, what they're saying, it's a theoretical form of machine intelligence that can understand, learn, and apply knowledge as we do, right?

19:52Independently. And perhaps be able to create new systems to replace itself. Yeah. And so on. And I sort of said, the thing that really, really struck me and stayed with me about that interview, I said, because he kept saying, and you know, this, it could be catastrophic. You know, once we've created this, we could be unleashing something that's utterly catastrophic. And Isaiah said, well, why are you doing it? Then why are you doing it? And he couldn't almost quite answer and I think it was basically because he said well we're pushing the frontiers of human knowledge and and so on but that that is the bit that really fascinates me that clearly is beyond commercial imperative right and because it does seem that they are motivated by something deeper within them that's almost quasi-religious it seemed to me do you think is that broadly right that is exactly how I describe it I do think it's quasi-religious they when you think about the narratives that the AI industry now uses to describe what they're doing, it's actually the oldest stories that we've ever told ourselves, which are found in religion, this idea of something having a heaven and a hell.

20:49And what you do in this moment, either it gives you access to that heaven or that hell. And so the companies say, if we create AGI, as in we, you allow us, the good guys to create it, we will give you access to utopia to this heaven because it's going to solve cancer it's going to solve climate change eradicate poverty and if you don't allow us to do it a bad actor is going to do it instead and it's going to decimate humanity so this is jack clark's camp is that they use these double myths, the heaven and the hell, the carrot and the stick, to ultimately justify why they are doing what they're doing.

21:34And to justify that they should be allowed to, on behalf of billions of people around the world, develop and advance this technology in a deeply anti-democratic way where they're not actually accountable to any of those people. Why do you think they're anti-democratic? Because there's no mechanism by which, formal mechanism by which people can actually provide input to a technology that is fundamentally going to shape people's future, right? Like they are, you know, we have democratic elections that allow us to pick representatives that lead us, and they are supposed to represent the will of the people because they are supposed to be an extension of what we want.

22:14These companies can make decisions that influence far more people in the world than nation-state leaders in modern day. And we don't get to elect them. We don't get to oust them if they do a bad job. We don't get to install someone new with a different agenda. There's no campaigning. Is that not always true with great corporate power, though? I mean, if you're talking about the great oil tycoons and the great oil companies of the 1950s or the early 20th century, they had huge power. They still do. We don't elect them. They're accountable to their shareholders. Is it not always true of great corporate power that to some extent it resists the kind of democratic inputs that normal political democracy does?

22:50It is true to an extent, but the scale of these companies makes it of, I think, a different kind of problem than just a different scale of problem because they now operate, you know, across all territories, they operate in ways that are more powerful than governments themselves. Like traditionally, companies are not more powerful than governments and that's how you're able to counterbalance their power is that there's still regulation from the government level that is meant to represent what people want, right? But these companies have consolidated an extraordinary amount of economic and political power to the point that they basically call the shots for a lot of governments around the world.

23:33And also, it seems to me, and I'm so struck repeatedly in terms of the way it's talked about in politics, I mean, it's kind of, I suppose, always true with some technological and corporate power, but like the asymmetry of knowledge. Yeah. between what these companies are doing and the political sphere. I mean, you know, when I speak to British politicians, American politicians, they don't have a clue, right? Most of them, but with maybe a few exceptions, they are operating almost on a different plane of reality from the political plane. And this is by design. These companies, like one of the parallels of Empire that I talk about in the book is the degree of information control and monopolization of knowledge production that these companies exert.

24:13So they, over the past decade, the AI industry has come to bankroll most AI researchers in the world. You could imagine if most climate scientists were bankrolled by the fossil fuel companies, we wouldn't get a clear picture of the climate crisis. And the amount of money they're paying for talent is so astronomic that states, even the US federal government, whatever, they don't and can't compete for that sort of quality of person. So by monopolizing that expertise, not only can they censor and control what AI research has even produced in the world for the public, they also can then go to a policymaker, go to the public and say, you don't know what you're talking about because we're the only ones that can see what's actually happening.

24:54And this is a crucial lever of power that they use to ensure that there continues to be no democratic checks on their continued expansion. And just before I want to explore more about that empire theme, but before we before we get there, just while we're on that concept of AGI, something whenever I've done episodes before, I've written about AI and something you get a lot from from people. I'm sure maybe you get it as well. Is this idea? Oh, you know, don't fall for their shtick. you know a bit like what we've been saying like you know they have commercial imperatives it's it's um there's an incentive for them to try and sort of say this is going to be transformational we want more and more capital we want more and more investment and it's a sort of dutch auction between between them yeah do you think that's right or actually in terms of the potential changes of agi do you think it is what they say that it could be is the is there a world whether utopia or dystopia is to some extent true or is it part of the kind of shtick and bump it is part of the shtick because in these narratives it's the ai system that's delivering us somewhere it's not about the people that are creating the ai systems to deliver us somewhere and i think this is the reason why you know there are people that genuinely believe in these narratives within the ai industry and also they recognize that it's rhetorically useful to use this kind of language and to put the responsibility and the onus on the thing that they're creating not themselves it's because ultimately if things go wrong it's because of the model and and it it alleviates scrutiny from them and their actions and the fact that as they they say it's a magic trick they're saying like focus on this hand where we're trying to build the AGI.

26:44And on the other hand, they're literally amassing more capital than any actor in history and more land and more energy, data, water, minerals, labor. You know, it's just like they are trying to redirect us from the real issue at hand. And on that, we will return to this theme of why they are like empires rather than companies just after this.

27:16From a range of trusted voices and award-winning journalists. Good morning, I'm Nick Ferrari. It's time to get to your calls. Find out the latest news and hear every side of the story. So just as I take responsibility for the results, I also take responsibility for delivering the change that we promise for a stronger and fairer Britain that we must build. Nah, he's still toast. He's just wasting our time now. The words of two in a growing list of Labour MPs whose patience with the Prime Minister has run dry. Is there anything he said or can say to make you want him to stay? Listen on our free Global Player app or the LBC app.

27:52LBC, leading Britain's conversation. If the world were like a Sleep Number mattress, everything would adapt for your comfort. Because as your life changes and your body changes, Sleep Number mattresses adapt and shift to give you personalized comfort night after night. And now it's the final days of our Everything's on Sale event. Save up to$1 ,200 on mattresses. Our Memorial Day event ends Monday. To experience a whole new world of comfort, visit a Sleep Number store or go to sleepnumber.com. Sleep Number, to a good life's sleep. Ask yourself, what are your best people spending their time on right now?

28:32Expense reports, receipt chasing, month-end close that takes weeks. You become what you spend on, and that's not what you're building toward. Brex is the intelligent finance platform that eliminates that work before it starts. AI agents that handle the manual stuff automatically, so your team can spend their time on what actually compounds. It's time to get Brex AF. Learn more at brex.com slash AF.