Karen Hao on the Reckless Race for Global AI Power

0:01In a world that never stands still, we're there to help you act with conviction. At HSBC, we connect the people, ideas, and capital to unlock growth, backing your ambitions at every stage. So when you're planning for the next generation, strengthening your portfolio, or expanding into new markets, we're there, bringing over 160 years of experience to help you move forward. HSBC. Opening up a world of opportunity.

0:30Get the news you need in just 15 minutes. Start your day with Bloomberg Daybreak, the podcast with a global view on the stories that matter. I'm Nathan Hager. And I'm Karen Moscow. Join us each morning for curated stories on current events, politics, business and foreign relations. Plus one conversation on the day's biggest developments all in just 15 minutes. Subscribe to Bloomberg Daybreak for a precise, thoughtful take on the stories that matter. Listen to Bloomberg Daybreak each morning on Apple, Spotify or anywhere you listen. Hey, it's Alec Baldwin. This season on my podcast, Here's the Thing, I talk to composer Mark Shaman.

1:06It's about the hang. It's the pleasure of hanging out with the people that you're with. You know, Rob and I was always a great hang. And director Morgan Neville. Film school teaches you all the wrong things about making documentary. What do you want to say? Documentary is all about your ear. What do you hear? I feel like my job is listening really, really hard. Listen to Here's the Thing on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Bloomberg Audio Studios. Podcasts. Radio. News. These companies have consolidated a historic amount of economic and political power, just like the empires of old.

1:47You do not need to unlock the benefits of AI through a corrosive approach. So why are we not holding these companies to those higher standards? Karen Howell, who says the AI industry is a reckless race for global domination. I think you do have a problem with the people at the top, right? We often obsess over the cast of characters at the top. Who's the good guy? Who's the bad guy? Who's the less bad guy? And what I'm saying is that's not the only option. We shouldn't have to live with companies that are so vastly powerful, where one person at the top is able to decide all these things. From Bloomberg Weekend, this is The Michelle Hussain Show.

2:33I'm Michelle Hussain.

2:38I don't know if you, at the point that you tap a prompt into ChatGPT or Claude, Gemini, Microsoft Copilot, whatever you use, if you ever stop for a moment and wonder how the data got there in the first place. I suspect you don't. Few of us do. These AI tools have been revolutionary in the last few years in the way that so many of us have changed how we work or how we learn. But Karen Howe, who's talked to me for this episode, wants us all to step back and to really think about what the process involves. That is not only the land used for the data centers, the water, the energy involved, but the actual process of gathering and preparing the data, where it comes from, who does the work in putting it together.

3:35Karen trained as a mechanical engineer and then did work in Silicon Valley before turning to journalism. She's written a book called Empire of AI, and it's that use of the word empire to describe the top AI companies that I ask her to explain right at the start of this. It started, I think, when she spent a few days inside OpenAI, observing them. They were not a well-known company at the time. She wrote about them in a less than flattering way. Today, it is safe to say that there is no love lost between Karen Howe and the top execs of the AI world. And you'll get a sense of that in my written notes that accompany this conversation.

4:19As ever, you'll find them at Bloomberg.com forward slash Michelle. You'll get the backstory to this conversation and what I followed up on afterwards. Karen and I spoke in London where there was something on the table that caught her eye as she came into the studio. It's a different way that I make notes ahead of a conversation with a spider's web style diagram that reminds me what I really need to cover. So that moment between us is where this begins. I love this mind map. Do you do this? Oh yes, I do do this. This is awesome. I do this, but I wish my writing was a little bit neater. Oh, it's so neat.

5:04I can just about, I can just about, but no, it helps me, but I never did it before this show began. I'm always so fascinated by the format in which people take notes because it really is such a window into how they think. Yeah. And I'm such a linear thinker. The diagram confuses me. Interesting. So what does this reveal about me that I think in a, I don't think in a linear way, in a non-linear way. Yeah, in a non-linear way, which I think is probably a lot more information dense. Essentially, it's what enables recall because, you know, you don't want to be referring to it too much. It's whatever helps imprint it in your brain.

5:45So at least I feel it. I was like, hope? Question mark. Very good. You're reading upside down. Now, yeah. Okay. Well, Karen, I would love to start with your own words, because you write in Empire of AI that over the years you found only one metaphor that encapsulates the nature of what these AI power players are, and that is empires. not engaged in overt violence and brutality, as we know from history, but they too seize and extract precious resources to feed their vision of artificial intelligence. Now, that is a real attack on the companies behind some of the defining tools of our age. What is it that's made you feel so strongly?

6:32These companies have consolidated a historic amount of economic and political power. And they are becoming fast the supreme powers in the world, just like the empires of old. And they do this through the dispossession of resources, labor, value from the majority. They lay claim to resources that are not their own, the data of individuals, the intellectual property of artists, writers, and creators. They exploit an extraordinary amount of labor through both the workers that they use and exhaust to produce their technologies, as well as the workers whose jobs get automated away by the deployment of these technologies.

7:08So there is this category of work that is involved in the production of these technologies, including content moderation, including the labeling and cleaning of training data that's called data work. That kind of work is now the number four fastest growing work in the U.S. And the types of people who are doing this work are scientists, lawyers, college graduates, PhD graduates. And they are essentially providing their knowledge and their expertise to the training of these models, but under deeply exploitative, poorly paid conditions where they are not working nine to fives. They are out there in their homes waiting for work to arrive.

7:49They never know when it's going to arrive. They never know when it's going to go away. They're being pitted against each other. You know, I've been covering this kind of work for almost eight years now. And it's the same story again and again. With every single community that it hits, you see the same exact patterns of exploitation. Are you saying that we should be thinking about the exploitation of labor at the point that we're using Claude or ChatGPT? Yeah, absolutely. So other industries have had similar problems. The fashion industry also engaged in a lot of labor exploitation and a lot of environmental degradation.

8:23The answer was never, let's just get rid of clothes altogether? The answer was always, let's hold ourselves to higher standards in producing clothes without those problems. It's still work though, isn't it? And there are so many communities where in different parts of the world, there is an absence of work altogether. I just want to ask you whether you think that there are definitely challenges in this industry and your work really details them and deconstructs them. But is there an aspect of this that is about the challenges of our age, that it's not necessarily more exploitative than what's gone before.

9:01It's the problems of our time. I want to go back to what you were saying, that it is work, though, because what is happening with this U.S. case now is that the people are actually people that are highly educated and could have gotten full-time employment. But the reason why they do not have access to better work now is also because of the AI industry. Taking away jobs. It's because of the way that the AI industry designs its technology and sells it as a tool for automating knowledge work. And so there's a shrinking job market for the unemployed population in the U.S. 25 % of them are four-year college degree holders.

9:44That's a historic record. And so it's not actually just giving work to people who don't have work. It's actually degrading the work opportunities that exist right now, taking knowledge workers out of a full-time employment kind of situation into a gig work kind of situation. So what do you say to a country like Malta, which has just signed a major agreement with OpenAI, where everyone who lives in Malta is going to get free access to ChatGPT Plus for a year? They're going to go on a course to learn how to use it. And the government of Malta is very proud of this agreement because it says it puts Malta at the forefront of the digital age.

10:26I've met a lot of policymakers from developing countries, from smaller countries that do see themselves as they can only participate in this revolution if they offer themselves up for sale. Just because, you know, the government is proud of this partnership and they think that this is what gives them the seat at the table, that it's actually leading to their citizens having a more dignified life. But their government would see it as equipping them for work or access to knowledge or everything else that they will need to use large language models for. I would argue they are being exploited. And I would be curious what the Malta people think, not what the Malta government thinks.

11:10And the reason why I think they are being exploited is because they are having their data harvested to train the next generations of models. And that's why these companies are going out and striking these partnerships. They need more and more and more data. They need more and more and more data centers to produce their technologies. And one of the challenges when it comes to searching for different sources of data is that they have run out of internet data. They've run out of high quality internet data. And so now one of the best sources of high quality data is straight from the user. And so if they can strike these whole of country partnerships, I mean, you get a whole of country database to upload freely.

11:56What kinds of thing? Like everything that's, if I was part of it, it would be everything that's on my phone, which is, which is great. Yeah, everything you use. You know, some people think of this as just, oh, well, I was using Google. I was, you know, saying a lot of things to Google and personal data as well. But I mean, I've never seen people upload their medical files to Google to ask for an interpretation of it. But people do that with ChatGPT, with Gemini, with Claude. And that's intimate medical information. Is there no positive side of it for you, Karen? Like that uploading of medical data, if it's done with consent, isn't it part of the progress of science?

12:37You might get something that troubles you diagnosed or you might be part of helping wider diagnosis. What I have problems with is the fact that it is in the hands of deeply powerful companies that have absolutely no accountability to the people that they're actually affecting around the world. And this is why I call them empires. That's the governance structure of an empire. You at the top are able to make hundreds of decisions in a day that then have cascading effects on billions of people around the world. And there are no formal mechanisms by which those billions of people can actually challenge, contest, provide feedback.

13:15that's why we've historically moved from empires to democracies because democracies are these institutionalized ways for people to say when they like something, when they don't like something, and how they want to collectively govern and self-determine their future. Are you actually saying that the power of these companies is greater than the power of governments? They are definitely increasingly becoming that way. Absolutely. Which means, like, even the U.S. government, the most powerful country in the world? I would say so. I mean, when there was the recent spat between Anthropic and the Department of War, the Department of War used the nuclear option to try and get a company to fall in line, which is to threaten to declare this company as a supply chain risk.

14:02Which would affect its work right across the U.S. government. The company, Anthropic, still did not fall in line. But isn't that because it did well in another way? It took the moral high ground, Maybe we can talk about that a bit later, but it got lots more users from the PR of that moment. Absolutely. Paying users. One in this situation, both PR and in this spat. What I'm saying is this was a moment that demonstrates that these companies can, in fact, have more power than the U.S. government. They are willing to actually go against the U.S. government because they know that whatever penalty comes at them from the government might actually be counterbalanced by the other benefits that come directly to the company.

14:46By profit elsewhere. Yeah. So tell me then, Karen, what is the answer? Because I know there's a call to action in your book. You clearly think that things can be and should be done another way. What are you hoping for? And do you think a figure like the Pope, like a unique figure, way outside the sort of government policy realm, is the kind of person who might provide some kind of roadmap here? I think it's important for all leaders in society, whether religious or otherwise, to be thinking about these questions and thinking about how to ultimately protect human agency at a time when the way that Silicon Valley conceives us this technology is in fact threatening it.

15:25And we've seen Pope Leo be quite strong on these issues and identify early on in his papacy that this is a particular issue where he is deeply concerned about the pathway of technology development undermining human dignity. And to go to your question of what is actually the answer here, I mean, one of the things that I always say is AI is like the word transportation. Transportation refers to everything from bicycles to rockets. And in the same way, AI refers to a very vast collection of different technologies, some of which are designed kind of like rockets. They're very resource intensive and they exact a lot of cost to produce.

16:05And so in general, we only deploy rockets for a very narrow subset of transportation needs. We never go around and say every single person should have a rocket and use that rocket for every single one of their transportation needs. That would destroy the environment. And it would also just be really inefficient and probably not actually get people to places faster because of the amount of time it would take to boot up the rocket. And so what I suggest in my book and throughout my other work is that we need to shift our portfolio of AI technologies to better reflect the kinds of specialized AI that we in fact need to tackle real societal challenges.

16:47Things like improving healthcare and accelerating drug discovery. DeepMind's Alpha Fold is an example of a system that I call a bicycle of AI. It is a system that predicts with high accuracy how an amino acid sequence will fold into a protein structure. This is really important for accelerating the understanding of human disease and then designing drugs to target those diseases. And it won the Nobel Prize for chemistry. This is a technology, even though it's called AI and ChatGPT is also called AI, DeepMind's AlphaFold is one that is fundamentally different from how ChatGPT is produced and how it operates.

17:25It uses very little data. It uses very little computing resources. But it's so powerful. And there's no need for labor exploitation either. And it's still able to provide a lot of benefits. So my question is, when you look at the portfolio of AI technologies, wouldn't you want to pick the ones that have great benefit, very little cost? Why would you want to pick the ones that have very great cost? Right. But these are value judgments, right? and something that you think has no value, someone else might. Like, let's take AI and defense. We know already that many armies use it for targeting and the use of weapons.

18:03You and I might see that as wrong. Others might see that this is necessary defense. It's part of the defense of your realm. And if you're not going to do it through the tools of our age, then you're going to fall behind. And the bad actors, terrorists, bad governments, whatever, are going to get you. Yeah, I think that's a particularly controversial issue where people fall in different places. But what I'm arguing is there's many different ways to produce AI. You do not need to unlock the benefits of AI through a corrosive approach. So why are we not holding these companies to those higher standards?

19:04www.hsbc.com

19:13The Bloomberg This Weekend Podcast. News, politics, and the lighter side of Bloomberg. The cutthroat competition to get a gig on a cruise ship. They get to enjoy all the amenities, and a one-week contract can pay like thousands of dollars for them. I know this is a good gig. Like you're booked through six months, and you could pay your bills for like a year and a half. And you may get norovirus. Keep going. The Bloomberg This Weekend Podcast. Subscribe today on Apple, Spotify, or wherever you listen.

19:56I'm conscious that both Anthropic and OpenAI are heading towards public offerings of their shares. Is that a moment that you think could bring greater accountability that once a company is public and it's got all kinds of investors and it's got annual general meetings, that a level of scrutiny that is lacking comes in, a useful one? I do think it could be. I am hopeful that it could be because when companies become public, they do institute more governance structures. But I lay this critique on existing public companies as well. I call Google, Amazon, Microsoft empires of AI as well. I do think that ultimately governance needs to come from the public and from publicly elected officials as well, not just the market forces.

20:46And not the founders. It sounds like you don't have faith in the actual people running these companies. A lot of your book is about OpenAI and Sam Altman in particular. Tell me the story of how you got to know him and the company back in the beginning of this work. I started covering AI in 2018 for MIT Technology Review. And that is a publication that focuses a lot on fundamental AI research. So I was reading scientific papers every week, talking with researchers at universities and corporate labs. And OpenAI, you know, it became part of my radar because at the time it was a nonprofit fundamental AI research lab.

21:26And it was one of the only ones. There was OpenAI, then there was Google, then there was DeepMind. And there were just a few others, but that was kind of it. And so in 2019, I proposed to OpenAI that there were a lot of changes happening at the organization. They had just created a for-profit arm. They had just gotten a big investment from Microsoft. that maybe they wanted to reintroduce themselves to the public. And I suggested that if they were interested in that, I could profile them. So I went to the corporate offices and embedded within the organization for three days, interviewed a lot of employees and executives.

22:02Including Sam Altman? Sam Altman was actually not very important at the time, so no. Because he had just officially become CEO. He was previously just co-chairman and didn't really have much to do with the organization at all. And so they set up interviews for me with Greg Brockman, the chief technology officer in Ilya Setskever, the chief scientist, who were the main runners of the organization's day to day. And they didn't like the profile you wrote, did they? They did not. You said at that time that there was a misalignment between what they were actually doing and their promises. What was it that made you say that?

22:37What did you see that led you to that? At that time, the misalignment that I saw was they're called OpenAI for what they said was a commitment to transparency. And this organization was already deeply secretive. And I noticed this culture in the way that employees were very, very nervous about what they said to me and where I was allowed to go within the offices. So that seemed already quite bizarre. And they also said as a nonprofit that they would never commercialize their technologies. And now, of course, they're fully for profit, which is the source of the case that collapsed between Elon Musk and Sam Altman.

23:16Is Elon Musk then the good guy in this for you? No. No. I mean, here's the thing. I think we often obsess over the cast of characters at the top. And we're always trying to think, like, who's the good guy? Who's the bad guy? Who's the less bad guy that I'm able to settle with because that's all the option set there is? And what I'm saying is that's not the only option set. We shouldn't have to live with companies that are so vastly powerful where one person at the top is able to decide all these things. It's the governance structure that's the problem. It's not that I think the problem will be solved if we just swap out one of these characters for one another or for a new person.

23:59But I think you do have a problem with the people at the top, right? Or certainly the small number of people at the top. And you kind of know the world that they came from because you were part of that same world. You went to MIT, worked for a startup. This could actually have been the industry of your life rather than you reporting from the outside. So I get the impression that you do have a problem with these characters. Yes, I do have a particular problem with the culture that they're steeped in and the ideologies that they have because it's an imperialist ideology. I mean, you know, Sam Altman said before he stepped into his position as CEO of OpenAI, he was president of Y Combinator.

24:39and he said, I'm going to invest in 10x more companies year after year until I invest in every company under the sun. And there's this kind of sense of all the people that are steeped in that Silicon Valley culture, which I was a part of. He was pretty young at the time, though. Maybe it was hubris, foolishness. Except that it's echoed through the way that he approaches open AI strategy, let's scale the models 10x more every year until we have reached the scale of the human brain. You know, like he's always thinking in these kinds of terms and all of them are thinking in these kinds of terms of let's just keep scaling and scaling and scaling and it's growth at all costs.

25:21And that is very particular to the culture of Silicon Valley innovation right now. So I do have a particular problem with the culture and with the people that have grown up in that culture. And also, I don't think that the fundamental issue that I'm articulating in this book will be solved by simply swapping in another person. Are there any less bad guys? Like, do you think Anthropic is a more principled company than OpenAI? No. Even though it stood up to the Pentagon? So interestingly, Dario Amadei, CEO of Anthropic, said that he actually did not have issues with fully autonomous weapons. In fact, he believed that this was a technology where the U.S.

26:07probably should keep up. The problem was simply that he didn't want this version of Claude to be used for that. And Claude has been used in AI-assisted targeting. It's been used in Iran. I think it's reportedly been used on the raid on Maduro in Venezuela as well. Yeah, that's right. Are you also following the Chinese AI players, DeepSeek and other companies like that. Very, very loosely. Do you believe that they behave differently or do you accuse them of imperialism also? It's really interesting. They do behave differently because of a very particular reason. So when ChatGPT first came out, all of the Chinese tech giants did the same thing as the U.S., the other U.S.