Can the AI Industry Slow Down?

0:00Some people treat ChatGPT like some kind of smart search engine, and some use it to get work done. ChatGPT Work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs, and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at ChatGPT.com by selecting Work Mode, available on Plus and Pro plans.

0:42If you listen to financial news, you know a lot of time is spent thinking about what's next. The next opportunity. The next investment. The next move. But sometimes what matters most is being ready for what you never saw coming. For more than 75 years, Cincinnati Insurance has worked with independent agents to help protect businesses, homes, valuables, and more. Because planning for the future isn't only about knowing what's next. It's about making sure you're ready for what you can't predict. Let Cincinnati Insurance make your bad day better. Find an independent agent at CINFIN.com. wise is the smart way to manage the currencies you need around the globe when you send money abroad using your bank you could get hit with hidden fees and exchange rate markups there's a better way try wise wise uses the exchange rate you'd usually find on google with no unwelcome surprises plus most transfers happen in under 20 seconds which means your money arrives in less time than you've been listening to me it's simple and free to sign up when you download the wise app be smart get wise t's and c's apply

1:51Bloomberg Audio Studios. Podcasts. Radio. News. The stakes are high. The stakes are high. The stakes are high. And it feels like in the past week the stakes have gotten so much higher. Or at least people are talking about the stakes in different terms. The rhetoric has gotten really heated. The capabilities have not gotten dramatically better. And I want to be clear, you know, there is no current existential threat to humanity, right? Thank you. I got a call from my mom last night. Sarah, I'm just really concerned. Everyone's freaking out. Do I need to freak out? I would say you do not currently need to freak out.

2:28This is about understanding how we need to shape and train the future models of development. It's about having the knowledge about how to align these models correctly. it's not as much about the models gaining amazing power that they shouldn't have.

2:51I'm Sarah Holder, and this is The Big Take from Bloomberg News. Today on the show, the AI industry says it's ready to slow down. I sit down with Sarah Fryer, a Bloomberg managing editor who focuses on big tech, startups, and AI, to discuss why Anthropik and OpenAI are asking the government for help in setting guardrails and why they may not get it.

3:19Dario Amode, the co-founder and CEO of Anthropik, came out with this widely read, much-discussed, much-debated blog post over the weekend laying out a plan for how an AI development slowdown could work. What exactly did he suggest? He suggested, first of all, that there would be some third-party evaluation of the models before they're released, checking if they are in alignment with what humans would need from them, that they wouldn't do things like escape their quote-unquote sandboxes and access the Internet if they're not supposed to. He also thought that there needs to be some clear collaboration with the government, working in concert with the White House to make sure that what they're doing is in accordance with what the U.S.

4:05needs. And then third of all, he wants some global coordination. He and other leaders have frequently talked about the threat of China's development, and this is all coming ahead of Trump's meeting with Xi Jinping. and this is going to be a major topic of conversation in those meetings. I should say OpenAI's Sam Altman signed on and endorsed what Amadei was writing about. Yeah, he was kind of like, sounds good to me. He was like, great, let's do it. Elon Musk said he was in favor. We got a post yesterday from Satya Nadella at Microsoft saying, yes, we also want to be responsible here. So everyone's saying, sure, we want to be responsible, but the actual contours of that plan is where it gets complicated.

4:52I want to dig in a little bit more to Amadei's plan, because one of the things he suggests is embedding these independent third-party evaluators in these AI companies to make sure AI is being developed responsibly and that things do not go awry, essentially. Who could actually fill this role in the AI space, and how could you ensure their independence? Well, there's one company that he has mentioned called Meter, which has been doing some of their earlier research on these incidents. And that company, you know, they're a little under the radar. We haven't written much about them yet. I expect we'll do more.

5:32But that's the kind of company that could come in and evaluate, has the expertise to do it. But there may be others. I think maybe thinking about this in the context of what other tech companies have done when they're under intense scrutiny, it hasn't always worked out, right? Remember Meta created an oversight board of folks from nonprofits and former government leaders around the world. And they were tasked with coming to evaluate Meta's community standards and see if they were adhering to them correctly. nothing really happened. That board didn't seem to have much operational power at Meta.

6:16They were just kind of the appeals court for user posts. So all of these discussions are taking place, but the backdrop that complicates everything is that Donald Trump does not want AI to slow down at all. And he made it very clear over the last couple of days, you know, this is not a problem. You know, the robots are not taking over the world. He called it a hoax. He called it a hoax. So he seems upset that this is even being suggested. He does say, oh, yeah, there do need to be, you know, some things we'll need to do. Right. But we're still going to win this. Like, you know, AI is going to be an American win.

6:58So Amadei, Altman, other tech leaders have said, you know, we need the government to step in and help us set these guardrails. Amit, I also brought up this idea of needing an antitrust exemption from the U.S. government to pursue a coordinated safety strategy. Why is that? Well, because when you're the market leaders and you're collaborating on plans to develop your products, that's very clearly the kind of thing that antitrust regulators don't tend to like. So some folks, some analysts I talk to, some talking heads on the Internet, David Sachs, as well, the former Trump AI czar, who is a venture capitalist, have been accusing OpenAI and Anthropic already of regulatory capture of, you know, basically now that they're big enough to say, oh, the scary things are happening.

7:52Now is the time for regulation. It might decrease the amount of uncertainty ahead of their IPOs. you may also have an advantage because if you're a bigger company and you're being regulated, you're already at the frontier, then the smaller companies that need to get there and compete with you, they're going to have a few more hurdles. When you are the leader in a market and you have the staff, the infrastructure, the size to withstand regulation and keep your business growing at the pace you want it to grow, the littler guys might not have that. Well, I want to take one of these arguments at face value, which is that these companies could just act right now without government response.

8:36David Sachs, as you mentioned, a tech investor and co-chair of the President's Council of Advisors on Science and Technology, went on Bloomberg TV. And he was basically like, if Anthropik and OpenAI think that frontier AI development needs to be slowed, they should just do it themselves. Just do it. It's on you guys to make sure your products are safe. If you feel like you need to, quote unquote, pace the frontier to do that, go for it. Do that. You don't need anyone else's permission. What could companies do tomorrow to ensure that this technology has the appropriate guardrails in place? They could do a lot of pausing on their training of future models.

9:17They could try to do more with the models that are already out, try to make sure that governments and important institutions like financial institutions have access to their most cutting edge tools so that those institutions could use them to check if their code is safe from hacks, find vulnerabilities and make sure that But AI cannot cause harm to them. AI tools, you know, as much as they can be used for a cybersecurity attack, they can also be used for cybersecurity safety. So they could think about it in that way. There could be some tangible things that they do to prepare industries, governments, companies, infrastructure for what is coming if they believe what is coming to be particularly damaging.

10:09Now, they also want to sell that, right? And so if they gave it away for free to folks, they may lose out on some potential revenue, which is really key for them ahead of their IPOs. Coming up, Trump, China, and what this all means for the timing of OpenAI and Anthropics IPOs.

10:59www.chatchypt.com scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at ChatGPT.com by selecting Work Mode, available on Plus and Pro plans. Let's talk about healthcare for a second. It doesn't always work the way people expect it to. If you've ever waited on a prescription refill or had a hard time getting the care you needed, you know the feeling. The system should just work better for everyone. That's exactly what the people at Optum are trying to do every day.

11:35They're a healthcare company linking patient care and pharmacy services and using data and technology to drive the whole system so care is connected, not complicated, for patients and providers. Things like making it easier to get care that looks at the whole person, from primary care doctors to mental health support and even in-home care, and then using technology to make sure they all work together. Technology designed to help doctors spend less time on busy work and more time with their patients. And those prescriptions? Optum is working to bring costs down, save patients money, and make it easier to get refills.

12:10Little by little, Optum is helping make healthcare work as one for everyone. Head to business.optum.com to see how. Wise is the smart way to manage the currencies you need around the globe. If you've ever sent money internationally using a traditional bank, there's a good chance you've paid more than you realized. Hidden fees, exchange rate markups, and extra charges can quietly add up before your money even arrives. There's a better way. Try Wise. Wise uses the exchange rate you'd usually find on Google, helping you avoid the unwelcome surprises that often come with international transfers. Whether you're sending money to family overseas, spending while on your holiday abroad or paying bills across borders, Wise makes moving money simple, transparent, and straightforward.

12:55Wise offers 24-7 customer service and runs over 7 million daily checks to spot and stop fraud. And most transfers happen in under 20 seconds, which means your money arrives in less time than you've been listening to me. Join millions, saving billions. Be smart. Get Wise. Visit wise.com or download the Wise app today. T's and C's apply.

13:23For people who are concerned about the risks of AI run wild, it seems like having the leading executives behind these AI companies coming out and saying, we're ready to take this seriously. We want to make this technology as safe as possible would be sort of good news. But there are major hurdles to implementing even the plan that Amode himself laid out. And so I want to talk through some of those potential hurdles. So the first would be that President Donald Trump is strongly against it. Why is he so opposed to adding more safety guardrails around AI? Well, Trump sees AI as the engine of the economy, which it truly is.

14:02If you look at the stock market, if you look at the rise of chip stocks, for instance, the rise of infrastructure investment in the U.S., The hundreds of billions, if not trillions of dollars that will be spent on the data center built out, that's all AI. I mean, that is the most positive economic story right now. So if that goes away, that's very concerning to Trump. And then the other thing is China. I mean, I think that Trump has made it a big part of his plan to be strong against China and say that we will be able to beat them. and the Chinese models so far have edged closer and closer. The gap has been narrowing with U.S.

14:45models, but they have not yet overtaken U.S. capabilities. And so the concern is if we were to slow down in the U.S. that China could catch up and even exceed U.S. capabilities, and then we would lose control of the frontier. Well, Amadei suggests a global framework for cooperation that would include China. How realistic is it that the U.S. and China can actually agree on guardrails and shared safety frameworks that they can make a shared plan, both stick to it and move forward? Well, I think the politics of it are complicated, right? Because other countries might look at the U.S. and say, well, you're only saying this because you're in the lead and now anything that you do may hinder us from reaching that same position, right?

15:30So it's the same pressure as that regulatory capture question, where if the leaders in the industry are the ones making the rules, how does it affect everyone else who wants to play in that market and be an alternative to those options? And there are several people, including executives like Satya Nadella, like Jensen Wang, who have signed on to this idea that open source AI is really important because you want there to be an alternative to the market leaders, if this new industry is going to capture so much of the economy over time, you want there to be a lot of competition. And as you mentioned, this AI question is bound to come up at the Trump-She summit, which is happening next week.

16:17Oh, yes. Oh, yes. I think the AI conversation is going to be so central to that summit. And I think a lot of people are waiting for that to see where do those talks lead. that might give us a lot of clarity on where the industry is headed and if there can even be collaboration internationally on this issue. I also want to talk about the potential economic considerations here. Already, we've seen a market impact of this conversation. Some chip makers saw their stocks slide this week. Why do investors seem so focused on these potential short-term consequences of an AI slowdown? Are they looking at sort of the longer-term risk to humanity as well?

16:58Well, I think it's going to be really unusual, but perhaps not too surprising if Anthropic comes out and their S1 says something about the downfall of humanity. I mean, can you imagine? We're bringing this new company to the world. In the fine print on risk. You might die. Yeah, yeah, yeah. So I think that that's, we're in this very weird moment where, like I said, the capabilities of these AI systems have not yet caught up to the hype of the doom, right? And there are some who say that the doom is even a strategy, that hyping something is a way to make it seem inevitable and all-powerful and therefore extremely lucrative, right?

17:49So I think that the market is still kind of parsing all of this. And because Anthropic and OpenAI are not so scared that they're, like, stopping. That indicates to people that maybe a lot of this is just trying to get clarity from the government about what they're going to have to do. Well, one consequence of this conversation already from an economic standpoint is that OpenAI has delayed its IPO. Do we know? Well, by the way, that was not a consequence of this conversation. We had already reported that OpenAI's IPO was not coming until 2027. What it was was a really opportune time for Sam Allman to come out and say, we don't think it's a great time to IPO.

18:34I wonder, you know, who's going to be pressured by that, right? Yeah. I mean, I wonder about Anthropic as well. Will Anthropic delay their IPO? So far, we haven't heard anything along those lines. But one read of Allman's comments was that it was an attempt to pressure Anthropik to delay their IPO as well. It's interesting, Sarah, this conversation about risks, at least in the past couple of weeks, has really centered concerns about bioweapons, military implications, cybersecurity issues. But there are other harms that people are concerned about when it comes to AI that could be more immediate, potentially, like massive job loss or, you know, entry-level job loss, environmental impacts, surveillance concerns.

19:15All of these things are things that are happening now. I'm wondering, would the pacing we're talking about have an impact on those kinds of areas? You know, I don't think it would. I think that's almost a separate conversation. The existential human threat conversation is taking some oxygen out of those broader conversations about, you know, what do we do for the workforce? What do we do for energy costs? Like, you know, if this data center comes to our city, we need to make sure they pay for their own energy. Like, that's a really tangible local debate. I think that right now, talking about existential future threat, it's pausing those other things, which I think are also really top of mind for people.

20:04This is The Big Take from Bloomberg News. I'm Sarah Holder. To get more from The Big Take and unlimited access to all of Bloomberg.com, subscribe today at Bloomberg.com slash podcast offer. If you like this episode, make sure to subscribe and review The Big Take wherever you listen to podcasts. It helps people find the show. Thanks for listening. We'll be back tomorrow.

20:34If you listen to financial news, you know a lot of time is spent thinking about what's next. The next opportunity. The next investment. The next move. But sometimes what matters most is being ready for what you never saw coming. For more than 75 years, Cincinnati Insurance has worked with independent agents to help protect businesses, homes, valuables, and more. Because planning for the future isn't only about knowing what's next. It's about making sure you're ready for what you can't predict. Let Cincinnati Insurance make your bad day better. Find an independent agent at CINFIN.com. Wise is the smart way to manage the currencies you need around the globe.

21:15When you send money abroad using your bank, you could get hit with hidden fees and exchange rate markups. There's a better way. Try Wise. Wise uses the exchange rate you'd usually find on Google with no unwelcome surprises. Plus, most transfers happen in under 20 seconds, which means your money arrives in less time than you've been listening to me. It's simple and free to sign up when you download the Wise app. Be smart. Get wise. T's and C's apply. This podcast is brought to you by Navy Federal Credit Union. their flagship Premier card gives you four times points on all travel. Earn on flights, hotels, ride shares, and more.

21:52Plus, the flagship Premier card doesn't require you to use any booking portals and has no rewards limits. It's all the travel, none of the baggage. Visit NavyFederal.org slash Flagship Premier to find out more. Navy Federal is insured by NCUA. Terms and conditions apply for rewards.