What's Really Happening Inside OpenAI? | Karen Hao (Re-Release)

0:00On this episode, Karen Hao, the first journalist ever allowed inside OpenAI's offices. She sat face to face with Sam Altman's lieutenants, asking the vital questions that they suddenly struggle to answer. She believes that she saw what seemed a secretive empire in the making, powered by billions, powered by one man's ideology, and then a race to control your future. This is the room where it happened. First-hand accounts from behind closed doors in business, inside the rooms, feeling the pressure and helping you work out what to do if it's ever your turn at the table brought to you by IFS this is the room where it happened with Karen Howe

0:42well Karen let's start by you taking us into the room where you had a conversation with Sam Altman's lieutenants what did they reveal to you that perhaps made you slightly worried and maybe even change the entire narrative of what you were going to write about open AI? Yeah, so this conversation happened in 2019 when I was a reporter at MIT Technology Review covering fundamental AI research. And open AI at the time was certainly not the company that it was today and didn't have the public reputation that it did today. But it was on my radar because I was trying to find organizations that were really pushing the frontier of AI and OpenAI fashion themselves as that.

1:28And so I embedded within the company for three days. And the very first interview that I had is the one that you mentioned, where I then sat down with Greg Brockman, the chief technology officer at the time, and Ilya Satskever, the chief scientist at the time. What I was really curious about was OpenAI fashioned itself as, first and foremost, a nonprofit working on these fundamental AI research breakthroughs for the benefit of all of humanity. But secondarily, they had this term that they were really popularizing in the lexicon, artificial general intelligence, this idea that they were specifically pushing to advance AI software to a point where it could become as capable, if not more capable than humans.

2:14And the reason why they had to do that was they, first of all, thought it was possible. And second of all, believed that they needed to like the good guys needed to get there first instead of the bad guys. So I was kind of just probing Greg and Ilya in that conversation. What is artificial general intelligence? What do you envision a future like with this technology? How are you actually going to get there? What are the harms or risks of the bad guys getting AGI first? Essentially just asking really, really basic questions about their premise as an organization, they were already three, a little over three years into developing Opening Eye as it was.

2:59So I thought that they would have relatively straightforward answers. And it turns out that they really struggled to answer even those basic questions. And that's when I started becoming a lot more skeptical about whether or not what Opening Eye said it was, was really what was happening beneath the surface. What was the thing that was said where you went, whoa, hold on, that's not what I thought you were doing here. There was nothing specifically that they said where I just had a shock factor moment. It was more like, you know, one of the first questions I asked was, why are you spending billions of dollars on the problem of artificial general intelligence instead of something else?

3:42You know, you're saying that you could build these systems to then solve climate change, But why not just deploy billions of dollars on existing climate technologies that we know are already going to make a dent on this particular problem? And they couldn't even really articulate that. Like, I remember Greg paused for about 15 seconds and then said, Ilya, maybe you take this one. And and so it was little moments like that. And then after that meeting where we had kind of, it was 45 minutes roughly, and we kind of went around a little bit in circles where basically the answer to every question ultimately ended up with, well, OpenAI's mission is to build artificial and general, is to ensure artificial general intelligence benefits all of humanity.

4:34And that is what we're doing. And I thought the logic was very cyclical, like no matter what question I asked, it kind of always went back to the mission statement. So then after that meeting, I started interviewing researchers and other executives within the organization over the course of the three days. And another thing that I started noticing was people said that the fact that no one could really define what AGI was, was a feature, not a bug. This was very much a scientific endeavor. they were exploring the frontiers of science to figure out the shape of what this technology would look like, which sounds reasonable until you consider the fact that they're deploying billions of dollars along the production of this technology that they are then promising is going to potentially bring utopia.

5:27And that is how they're continuing to accumulate those billions of dollars and continuing to get public license to forge ahead and create this technology. And then the other thing that I noticed was that the organization was weirdly secretive for a nonprofit, a company that professed to be still a nonprofit that was meant to do research that they would open source to the public. They would not let me interview people without a comms person present. They wouldn't let me go just walk around to different parts of the office. And it was all because they were worried that I would see something that was not sanctioned to be publicly opened up to the public.

6:11And it was just odd because the premise was that everything would be opened up to the public. So there were all of these just little moments that ultimately amounted to me wondering whether to actually take what they said about themselves at face value. And then after my time within the company, I continued to source interviews on my own with people who worked there. And they started to confirm that there was a disconnect between what the brand and the veneer of OpenAI was and what they actually did behind closed doors. If you haven't already subscribed, please do right now. It means you don't miss an episode of The Room Where It Happened.

6:52and it helps us attract the brightest and best new brains to the show.

7:00Shipping, billing, admin, payroll, marketing. You're managing all the things. So why waste time sending important documents the old-fashioned way? Mail and ship when you want, how you want with Stamps.com. Print postage on demand 24-7 and schedule pickups from your office or home. Save up to 90 % with automated rate shopping. That's why over 1 million small businesses trust Stamps.com. Go to Stamps.com and use code PODCAST to try Stamps.com risk-free for 60 days.

7:33ACAST powers the world's best podcasts. Here's a show that we recommend. Go touch grass. You have probably heard this phrase. Maybe you have even said it. But beneath the sort of lighthearted nature of this meme is something very real and important. A growing sense that staying in touch with our humanity and being present in our bodies matters more than ever in today's digital world. My name is Manoush Zamorodi, and I am taking over as host of TED Talks Daily this week to explore what technology is actually doing to your body and mind. In special interviews with scientists, doctors, parents, artists, and more, we're going to dig into your physical and mental health on tech.

8:23How we think about our bodies differently now. How we relate to new innovations that are amazing but also a little scary. And how we can live a healthier life in this high-tech era. Tune in on TED Talks Daily wherever you listen to podcasts.

8:42Acast helps creators launch, grow, and monetize their podcasts everywhere. Acast.com

8:53And the reason why I think this is a really important conversation for us to have, right, is that there will be some people listening to this going, well, they're a business. They don't have to tell you everything that they're doing when they're running their own business. Like you can't go and get every single secret from McDonald's or Coca-Cola or Nike. But this is not just any other business. This is the business that is effectively controlling the future of our planet and therefore the future of humanity. It's a business that we're so excited to dive into, so excited to use, so excited to work with.

9:20It's going to transform business, transform relationships, transform the world. So I feel like you, that we do need to know, like, what is the end game here? Like, what are you doing with all of our data? What are you doing with the hours that we spend on your platforms trying to run our lives using your systems? So do you feel that at any time, Karen, you have the answer to this? Have you worked out whether they just didn't really know or whether they are and were then actually doing things they just don't want the likes of you and I to know? It's a great question. And I completely agree with everything that you mentioned that this business is not any other business and the lack of transparency does have profound consequences and ripple effects.

10:06I would also add that it's funny because when I was embedding within OpenAI, it was actually not a business. It was a nonprofit. So, and it continues to kind of, this is like a key tension all the way up until present day where there are still people within the organization that are still not quite convinced that OpenAI is a business. Do you think it's a business? Do I think it's a business? 100%. But I also think that to understand it purely as a business, to your point, is insufficient for really understanding the full shape and texture of its operations. So in my book, Empire of AI, the reason why I call the book Empire is because these companies, I really think, need to be thought of as new forms of empire.

10:59They're not just operating in the business sphere. They are also operating in the political sphere. They're accruing an enormous amount of social, political, economic capital. They're accruing an enormous amount of land, energy, water, raw resources to build this technology. One of the driving motivators behind what OpenAI does is a kind of this quasi-religious ideology that artificial general intelligence is right on the horizon and it's somehow going to bring profound benefit to humanity. and this is also very imperial in nature. You know, the empires of old, they also had, they were driven by a religious ideology of we need to bring progress and modernity to all of humanity and that's why we need to engage in grabbing all of this land, grabbing all of these raw resources, exploiting labor, building these technologies that ultimately fortify the empire and could potentially end up exacerbating global inequalities and that is essentially what is also motivating OpenAI.

12:06And it is because the driver is not just financial, it is also ideological. I think different people that you talk to within the organization will have totally different interpretations of what they're doing and why, because it is every person kind of has their own personal value system that they're using as a prism through which to see OpenAI's mission and how to achieve it. And did you ever have an opportunity to sit down directly with Sam Altman and say, what is the end game here? What is this all about? No, Sam Altman never agreed to sit down with me. But, you know, other journalists have asked him that question.

12:47And he's effectively said different stories depending on what is relevant of the moment. So when he's in front of Congress or the UK government, and he's talking about why Opening Eye shouldn't be regulated and needs to continue pushing forward and advancing the cutting edge. He'll evoke these ideas of AGI will ultimately resolve poverty. It will cure cancer. It will bring, it will completely resolve all of our climate problems and paint these really far-fetched but beautiful visions of a possible future world of a profound abundance. Other times when he's talking with journalists, he will kind of just turn into an aw shucks.

13:37Well, I'm not, you know, I don't really have any endgame. I'm just trying to build like cool products and trying to help people and trying to create a super assistant that does amazing things in your life. And so the story kind of shifts depending on what his audience is. and it ultimately doesn't, you know, it doesn't actually get to the core of the revealed preference of what opening eyes actions show. What do you think is the end game from all the research you've done, the conversations you've had, the rooms that you have had access to that people like me haven't? What do you think are they aiming for here?

14:19It's very complicated. It depends on who you are talking about within the organization. because ultimately, you know, an organization doesn't have a single endgame. It's sort of a collection of different people with different endgames. In terms of talking about Sam Altman's endgame, you know, I cannot ascertain explicitly what his endgame is, but it is not lost on me that every single tech billionaire now has an AI outfit. That is an important fact to just understand. And what I would gather from the fact that they all have an AI company and they all are trying to build a chatbot is that they see this technology as some kind of ultimate quest.

15:01and if they can control developing their version of AI, they can make it in their own image and that is going to help them to continue to accumulate significant power and potentially more wealth and more control over the future of how humanity is going to go. Other people within the organizations, you know, I mentioned this word quasi-religious, there are people who just, they think that they are creating a digital god or protecting us from a digital demon and that this is their calling, that they were brought to this earth to usher in a new form of intelligence that will be the next stage of the human species or the next continuation of the human species.

15:52And so the organization, OpenAI as a whole, ends up being a strange conglomeration of all of these different beliefs, ideologies, endgames. And the manifestation is that it ends up having this incredibly imperial-like conquest where it's trying to dominate the whole world. You see, one of the things that I'm thinking about here, Karen, is I don't know what the other alternative is because we have an open AI, which feels like it's tied quite closely to one man's belief and ideology and the direction that he wants it to go. And we have all these many other people working within the organization that might have different ideas, but someone has to be driving the direction of open AI.

16:34Someone has to be deciding what our future looks like. Someone has to be thinking about what's the most valuable thing this can do for humankind. Someone has to be looking at the biggest risks that are posed to us as a species and wondering if AI can solve them. So if it isn't these people, then who is it? And is there a system, do you think, whereby we as the human beings that are going to be impacted by this deserve to have some kind of say over the people that are making these decisions for us. Yeah, I mean, this is my fundamental critique of the tech industry in general and OpenAI in particular, is that the way that I agree with you that there needs to be people that think deeply about the challenges that we face and how to get to a better future.

17:15You know, we all want to improve our quality of life. We want better healthcare, better education. We don't want war anymore. We want peace. These are all things that there need to be people thinking about, but they should also be elected representatives that represent all of us. And there should be a democratic form of governance around these extremely consequential issues and how they should be resolved and how we tackle them. And in general, that's just not at all how Silicon Valley operates. OpenAI doesn't operate that way. Meta doesn't operate that way. Google, Microsoft, Apple, they operate in a much more techno-authoritarian way where they dictate to billions of users, or in OpenAI's case, hundreds of millions of users, exactly what their technology, like these profoundly influential algorithms are going to do and integrate into their lives and shape the course of their healthcare, their economic opportunities, their educational opportunities.

18:22opportunities. I mean, ultimately, like we need to have a say in these things that are molding who we become as a society. And I think that maybe at the very beginning of this journey, we didn't feel that we needed a say because we were sold this dream that these are people building structures to protect us as human beings, right? AGI was going to solve all of our problems for us. So let us get on with it and we will do it. And actually, I wonder whether the more you've looked and the more that you've scratched and the more you've seen below the surface, you think that actually that's some really smart branding cloaking a kind of classic Silicon Valley business?

18:58Yes, short answer. One thing that's really interesting about Altman as a person is he's extremely strategic. And both his detractors and his boosters agree on this, that he's very long-term strategic. And I never got a smoking gun or found a document or whatever that said that his intention from the very beginning of co-founding OpenAI as a nonprofit was to just ultimately shed the nonprofit. But there is something to be said about how one, if you are an incredibly strategic person, how you would go about building an organization like OpenAI. So in the beginning, OpenAI was co-founded at the end of 2015.

19:46At that time, Google was the dominant AI player on the block. It already had a monopoly on some of the best AI researchers in the world. And so OpenAI, in order to catch up to Google, the main bottleneck was talent. They needed to figure out a way to appeal to some of these top AI researchers to leave Google or new graduates that were coming out of PhD programs to not go to Google and go to OpenAI instead. And so from the early days in those first few years, OpenAI's main strategy, they couldn't compete on compensation, so they competed on a sense of mission. And the nonprofit was a really convenient signal of how committed OpenAI was to that mission.

20:37And And Ilya Sudskever, the chief scientist who was a very critical hire in the early days of OpenAI that legitimized the organization, he explicitly came to OpenAI because of that mission. He took a huge pay cut to come to OpenAI. And once OpenAI then had its team, the bottleneck shifted. Then the bottleneck became about capital because now it had a core group of really smart people and it just needed to build. and they specifically chose to advance AI development in one of the fastest ways possible, which is to simply pour more data and use larger supercomputers to train a single AI model. And supercomputers are really expensive.

21:24So that's when they thought nonprofit doesn't cut it anymore. We're going to put a for-profit arm within this nonprofit and use it as a vehicle to raise billions of dollars. And if you look at that, you're like, well, that kind of seems like it was the strategy all along. You know, like you first design the organization to tackle one bottleneck. And then once you've solved that bottleneck, you morph it to tackle the next bottleneck. And it's really convenient that this original nonprofit mission-driven ethos continues to ensconce your organization and kind of warm, fuzzy feelings so that when you make mistakes or whatever, you can play or appeal to those original roots.

22:06and massage the bad PR and turn it into good PR. And another layer to this, of course, is the relationship between OpenAI and Microsoft. And I wonder what you think about this, that they are so closely aligned. To what extent do you think that reliance on Microsoft, the funding, the infrastructure, the political leverage is an issue? So interestingly, Microsoft and OpenAI are very not aligned now, as much as Altman pretends otherwise. But certainly when they were in the beginning, they were extremely aligned as organizations because OpenAI could not build its own supercomputers and needed someone to build them.

22:47And Microsoft had been struggling at that point to catch up to Google in the AI race. And so it was looking to supercharge its capabilities. And after debating whether or not to try and just supercharge it by cultivating, strengthening its own internal AI research efforts, it decided that it was just easier to essentially, quote unquote, acquire another lab and get that helping hand externally. And that was absolutely a very powerful alliance in that you have an organization that is an extremely successful, highly profitable tech giant backing another organization that has this quasi-religious mission and fervor for pursuing advanced AI.

23:40Now we're seeing the relationship really fracture because ultimately these are two very different organizations. Microsoft is at its core a business and OpenAI, as I mentioned, it doesn't really fully operate as a business. It also operates as a very ideologically driven actor. And so as the business case for AI has started to weaken and evidence has started to chip away at the original highfalutin claims that AI was suddenly going to grow the economy by 10%, solve all these problems and so forth, Microsoft has begun to pull back while open AI has continued to lean forward. And so now there's this friction where they still are contractually legally linked to one another.

24:30But there's been a ton of really great reporting that has shown that OpenAI is really desperately trying to extricate itself from this relationship because now it feels like it's beholden to Microsoft's slower pace. So interesting, Karen. I mean, you have dedicated your life, I imagine, right, for the last few months and years on this. I started covering AI seven years ago. And yeah, it's become an obsession. So the question for the likes of me and many people that will listen or watch this episode is, with everything that you've seen, what is the thing or the things that worry you the most? My biggest concern is that these companies, OpenAI, but all of the other Silicon Valley giants and tech and AI giants, they have grown so powerful that they really are beginning to completely warp and distort democracy.

25:34They are able to make, you know, they can acquire land, create digital currencies, take people's energy and water, hike up their utility bills, distort people's educational and economic opportunities just with the wave of a hand based on what they decide to deploy in their products one day to the next, based either on business or ideological grounds. And now we're seeing in the U.S. this strategic alliance between Silicon Valley and the U.S. government, which was arguably originally the only entity that was more powerful than Silicon Valley and could actually regulate these companies. And now under the Trump administration, we see absolutely no interest from the U.S.

26:20government in playing that countervailing force. And so what I see as the historical parallel is really the British East India Company and the British Crown, where you had a corporate empire and a nation state empire that were strategically aligned and benefiting from one another in a way that really ultimately led to a lot of exploitation and harm around the world. And so that is my greatest theory, is that if we continue to be on this trajectory where these companies get more and more and more and more powerful and more and more and more wealthy, where they can just buy up whatever they want and knock over whatever obstacles in their way, that we are going to lose the ability to collectively self-determine our futures, and that's when democracy dies.

27:11And why do you think the US government or other governments maybe are not willing to play the counterbalance with these organizations? I think the U.S. government very likely sees these companies the same way that the British Crown saw the British East Sydney Company as as a empire building asset to their own imperial ambitions. You know, the U.S. government has made under the Trump administration very clear that they want to rebuild the American empire and expand and strengthen it. And so they are they see these American tech companies as tools for doing so. as quickly as possible. So they're striking up these deals where they're trying to deploy OpenAI's technologies all around the world, and they're trying to help OpenAI build more data centers to facilitate that.

28:02And, you know, there was an op-ed in the Financial Times from a former Dutch parliamentarian, Mariette Schake, who mentioned, you know, isn't it convenient that the U.S. government is trying to install all this American hardware and software all around the world that they could then, in their imagination turn on and off at any time. So I think that is sort of the reason why the US government is unwilling to do anything to check these companies because it believes that it can maintain control over these organizations and continue to use them as hands to perpetuate their own political will. And why does this seem to worry you so much more than the way the world has looked for the last 20 years anyway?

28:52Because if I think about it, I don't really know who owns or runs the company where all of my personal data is stored. I don't know who owns or runs the company where all my email conversations are stored. I don't really know who owns or runs the company that provides me with my water, my gas, my electricity. The information that I seem to be watching or absorbing on social media, I assume, is real because that's the way the human brain works. Why does this AI conversation concern you so much more than the fact that really our world has been run by businesses now for decades? It's the scale. So even compared to social media, which already we can argue was problematic in its scale, AI is way worse.

29:36So if we look at Meta as an example, which both existed in the social media era and now is trying to refashion itself as an AI company, You know, under the social media era, Meta accumulated around 4 billion user accounts worth of data. When it came time for them to develop generative AI and they wanted to out-compete ChatGPT, the New York Times reported that there were internal conversations of executives lamenting the fact that they did not have enough user data to do that, which is bananas. Like they had 4 billion user accounts of data at that point. And so they were talking about, in addition to that data, what about if we bought out Simon & Schuster for all of their books so that we could train a generative AI system?

30:27And what if we continue scraping the internet? And what if we continue acquiring all this other data? And in the end, they didn't buy Simon & Schuster. They just torrented all the books from the dark web and used it to train their AI models instead. So that is just one example through the lens of data for understanding how the scale has really shifted here. But then the other thing is Meta is also, you know, they used to build data centers as a social media company, and now they also build data centers as an AI company. Mark Zuckerberg has been boasting the fact that their AI supercomputers will be roughly the size of Manhattan.

31:08like one of their supercomputers will be roughly the size of manhattan and they will use the energy draw of manhattan and that is something that is unprecedented and that is single-handedly currently reversing an extraordinary amount of climate progress that we made in the last decade because the amount of energy that we now need to add to the global grid is equivalent to adding half to 1.2 times the amount of energy used in the UK annually onto the global grid in the next five years. That's how much we would have to add to continue to support this current pace of growth. And most of that will come from fossil fuels, from coal plants that were meant to be retired, from gas plants that were meant to be retired.

31:56So the scale has just completely exploded to an unfathomable degree. And sometimes I feel like the size and scale and numbers that these tech CEOs throw out at people is purposely so large and extraordinary that the average person in the world has never encountered something of that magnitude. And therefore, it just becomes incomprehensible. And that's part of the strategy of how they can slip all of this under the radar. You know, like, what does it even mean to have a supercomputer the size of Manhattan? I mean, that's just like you can't even begin to really think about that.

32:59right now. Print postage for certified mail, registered mail, and packages in seconds. Then schedule a pickup right from your home or office. For a limited time, go to stamps.com and use code podcast for a free welcome gift. Taxes and fees apply.

33:16It's so interesting and good to think about it because all I see at the moment is people getting excited that they can bring pictures of their loved ones back to life. They don't actually think, well, what's the other side of the coin for these kind of strange little games that we're playing? What's the real game that's actually being played under the surface? And when you were embedded with OpenAI and you had the chance to ask about the environmental impact, you know, they say that AGI is going to save the world. Well, this supercomputer the size of Manhattan is the kind of thing that ends up killing the world, right?

33:47So what was their answer to those questions from you? Yeah. So at that time, there had just been a paper that had come out that, I mean, we're talking like this in 2019, large language models then do not even qualify as large language models today. We're talking about around four orders of magnitude of increase in the amount of data and the size of the supercomputers that are now training the models today. But even then, OpenAI had just developed GPT-2, which was a generation and a half before ChatGPT. And there had already been a paper that was calling out the fact that these types of language models were incredibly carbon intensive.

34:34And so I asked, you know, you're saying that AI is going, you could potentially get to an AGI that solves climate change. But also along the way, there are not potential environmental consequences. There are documented real environmental consequences. And so you're saying that you'll be able to get to AGI, this theoretical possibility, before the real consequences of the process of getting there just completely take over everything. And Greg Brachman said, I wouldn't go too far down that rabbit hole. The answer in itself tells you quite a lot, doesn't it? Yeah. And, you know, Ilya Sutskever later said to Cade Metz, who is a New York Times reporter, longtime chronicler of the AI industry as well, this technology will become so useful that the earth will be covered in supercomputers and power stations.

35:37And he considered that to be like a good future that we'll just be surrounded by data centers and power stations no matter where we go. And in many ways, Karen, you know, the more you look, the more you find. So we can talk about the carbon that's created. We can talk about the amount of water that needs to be drawn to keep these supercomputers cool. We could talk about supercharging mines and e-waste, which is the byproduct of what you've just mentioned. We can talk about AI-enabled weapons, and suddenly we could live in a world where AI is making decisions that at the moment human beings are making around warfare and other things.

36:15And we can, of course, talk about, as you have, concentrating AI power among a very, very small group of individuals and organisations. But if I'm listening to this, I'm thinking, well, hold on. What about all the great stuff? What about the advances in treatments for awful diseases or trying to create food in countries where people are starving to death or all of these issues that have plagued humankind for hundreds of years? Is there not a part of you that looks at it and goes, well, you know what, there's going to be a cost to AI, but maybe it's a cost that we should bear for the good that comes.

36:48There are many different types of AI. And I often use the analogy that AI is like the word transportation, where you can have a car, a bicycle, a rocket, all different forms of transportation, but they have very different cost benefit trade offs. And what these companies are trying to do is build the rocket version of AI, which arguably we do need some rockets for some things, but we don't need everyone to have a rocket. You know, if you were commuting from Dublin to London by rocket, like people would be like, what are you doing? This is completely disproportional to the task. And there are all of these other types of AI technologies that are more like the bicycle version of transportation, where it's much more efficient.

37:36It's much more task specific, it does one thing really well, like the advancing medical and drug discovery. So DeepMind, before it got onto the AGI train, they were working on task-specific models, such as one called AlphaFold, where AlphaFold is just a system that does one thing well. It predicts the protein folding structure from a sequence of amino acids. It's not trained on the internet. It's not trained on everyone's chat interactions. It's just trained on amino acid sequences and protein folding. Because of that, it's able to do this task with very, very high precision. And that is an important task for ultimately drug discovery, for understanding disease, for improving healthcare.

38:29And it ultimately won the Nobel Prize for Chemistry last year. And so when people say, well, isn't AI going to bring us potentially all these benefits so the costs are worth it? It's actually a false trade-off because we're talking about totally different AI systems here. There's plenty of AI systems that already have demonstrated benefit in things like healthcare, education, climate, sustainability, so on and so forth, that actually have extremely small costs in terms of their development, both in terms of data and the environmental impact. They do not need to be trained on supercomputers the size of Manhattan.

39:08They can be trained on powerful laptops in some cases. Whereas the chat GPTs of the world, which we're being told is going to bring us all these benefits, have really not demonstrated much in results in those areas. but they certainly have already created an extraordinary, you know, amount of consumption, resource consumption, energy consumption, water consumption in their creation process. Look, Karen, your book is blowing up across the states. People are talking about the things that are in it. But I suppose the most important question is what now needs to be done? If your book has one impact, what would you like that impact to be?

39:50I really want people to understand that technology development is not inevitable and that it is also not just the purview of people that work in tech companies. Actually, every single person, regardless of where you sit in society, can have a say in technology development. Because when you think about how these companies develop these AI models, they actually need, because it's so resource intensive, They need access to a whole host of different ingredients that are owned, collectively owned by different groups in society. You know, the data that they train on, that's the data of artists and writers and people posting on social media, people posting their kids' photos on platforms like Flickr.

40:42and now we're seeing significant pushback from some of those groups saying, no, we're going to sue you because we can't accept that you're just going to take our IP, that we have copyright protections on this work and that is them reclaiming ownership over a key ingredient that this company needs access to. We're seeing hundreds of protests breaking out around the world with people pushing back against data center development because they're saying, we don't want to host a data center in our community. It's hiking up our utility bills. It's creating all these other environmental impacts. It's reducing our air quality.

41:14And it's stalling. It's successfully stalling some of these projects or at the very least forcing companies to sit face to face with these communities because usually they don't. They just strike these deals behind closed doors, plop that massive data center into their environment, and then the community has no say about it. We're also seeing students and teachers have huge discussions about whether or not we actually want AI in education, whether AI is actually enabling better education or it's just degrading people's critical thinking skills. And what are better ways of integrating AI into the school environment?

41:52You know, to give OpenAI a little bit of credit, they have been responding to that criticism that ChatGPT might be eroding students' critical thinking by adding a study mode to their chatbot. That was in direct response to the huge amount of criticism and blowback and discussion that people started having around this particular issue. So like all of these, I often look at like AI development, there's a supply chain just like any other product. It's a digital technology, So people don't really think of it as having a physical supply chain. But just like the fashion industry, there is actually a physical supply chain there.

42:30There are all of these different ingredients. There are all of these different places where AI is either used or people decide to not use it. And if we consider these as different sites of democratic contestation, then we'll if and we have different movements, different groups that are pushing to actually assert what they want out of the technology, how they want the technology developed. all in concert with one another, companies have to respond. And I hope that that is the one thing that people take away from my book, is that they have an active role to play in shaping our future. And if we believe that it's the role of government to step in and solve these problems for us, do you believe that will happen?

43:11I don't think people should wait for government. The beautiful thing about democracy is that even when there is an absence of leadership at the top, there's leadership from the bottom. And I think with all these things, as we wrap up, I think there were people listening to this going, well, I'm not too concerned because I'm not sure it really affects me. And this is a business show, right? So let's say someone in the UK is starting a business and might not even be in a tech business. How quickly could that person find themselves in their new business in competition with or even facing challenges from those big AI firms?

43:43It's a great question. It depends on the industry that they're in. And it also depends on the structure of their business, whether they're going for quality or quantity. You know, one of the things that is kind of an interrelated question that is on many people's minds is how quickly am I going to lose my job to AI? And it's not you. It's not AI suddenly replacing your job. It's not AI suddenly replacing your business. it's executives that decide to fire layoff workers. And ultimately, if you are an executive of a business, it is your decision to decide how you are going to compete. You know, I've found that people have, now that they've gotten really, really tired of AI slop and poor quality services that are being AI run, that they're gravitating towards higher quality services.

44:34So orient your business towards the things that AI can't do and beat out the competition in that way. But I do think that even as certainly AI is going to throw a lot of interesting challenges at you and create a lot more pressure in the business environment for you to succeed, that there's still quite a lot of maneuvering that you can do to outperform. And to finish with a question about Sam Altman. I mean, they're very good at talking about things that I think will excite people. but I saw a headline saying artificial intelligence will generate enough wealth to pay each adult$13 ,500 a year.

45:13And I can imagine a lot of people suddenly thinking, hold on, that's amazing. I'm going to get rich. But here's a quote from someone that I'll finish with. I'd love your reaction to this. He says, productivity gains from AI could make the society of the future much less divisive and enable everyone to participate in its gains. Do you think that's what we'll see? Well, what I know is what we're seeing now, which is that the AI boom has created more billionaires at a faster rate than ever before and has left new college graduates with an inability to find economic opportunities. so what I see currently is that is creating more inequality more divisiveness and that should be the strongest signal for what would come next if we don't take our future in our own hands and change the trajectory of AI development.

46:15Karen there was so much talk about the exciting things that AI is going to bring us I think it's a really important conversation for you to show us the other side of the coin. And it's not just guesswork. You've been in there. You've spoken to the people. You've seen the whites of their eyes. You've spent years researching this. You've put in the hours. And I think people should listen to you. So thank you very much for your time. Thank you so much, Jake. So I think the biggest thing that I thought about having that conversation with Karen is how I feel like I have just seen one side of AI. All I've thought about is how does it benefit me?

46:47How can it benefit my businesses? How can I work faster? How can I work smarter? How can I get it into the hands of my teams. I've genuinely not stopped until this conversation and thought to myself, what is the cost of this? What is the other side of that coin? How risky and dangerous is it that these companies, these AI businesses, which are without question going to be a huge part of our future, are controlled by very few people? How worried should we be that big governments are not stepping in to control it, to regulate it, to protect us as individuals? But at the same time, I felt inspired by the fact that she was saying that we as individuals who seemingly are often powerless actually do have the power to make a real difference here and I don't want you to leave this thinking well that was a scaremongering conversation about AI.

47:31No one is saying that AI isn't going to make an incredible difference to your life, to my life, to the lives of everybody around us but it's really important we also have these conversations so we understand the cost, we understand the people who are behind these movements and we understand that it's not all going to be good in the future. There are going to be things that you and I really need to think about and think about carefully. Thanks so much for joining me. I hope you feel you've got loads to take away from being in the room where it happened. If you can just take a couple of seconds to subscribe, it means you get each episode directly to your inbox.

48:03And it also means we can attract more incredible guests like George Heaton, Rory Sutherland, or Richard Branson. Thanks for joining us. See you next time.

48:17Thank you.