The Tough AI Questions Looming Over the Xi-Trump Summit

0:00Stephanie Flanders:So there's a lot of noise about AI, but time's too tight for more promises. So let's talk about results. At IBM, we work with our employees to integrate technology right into the systems they need. Now, a global workforce of 300 ,000 can use AI to fill their HR questions, resolving 94 % of common questions. Not noise. Proof of how we can help companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business. IBM. This is Robert Smith from Business History. If you're listening to this, there's a good chance you're a small business owner.

0:35And like every small business owner, you started with a dream to do what you love and watch it grow. What you probably didn't dream about? Keeping up with cyber threats. That's where MasterCard can help. With access to tools that help identify cyber threats to better protect your business. Building your dream business? Priceless. For cybersecurity in a changing world, there's MasterCard. Learn more at MasterCard.com slash smallbusiness. Never bet against American grit or American energy. Through innovation, Venture Global is not only building some of the largest energy facilities in the world right here in the United States, but delivering American energy at a fraction of the cost and a fraction of the time.

1:18So while others are busy talking, we're busy building. That's Venture Global. That's unstoppable energy.

1:33Bloomberg Audio Studios. Podcasts, radio, news. President Xi and I have a good relationship. I wouldn't have a lot of different deals here. We intend, we're not just doing it for our health. I would honestly say if you considered how dependent we are in China now compared to five years ago, roughly around the time of the passage of the CHIPS Act, we're probably more dependent now. Because we've improved in one category and become much more dependent on others. And so I don't think this relationship or the US could really bear a true deglobalization or sort of this kind of two technological spheres that's been discussed.

2:07And so this relationship will just have to be managed maybe six months at a time, three months at a time, which now seems to be the pattern going forward.

2:23Stephanie Flanders:I'm Stephanie Flanders, head of government and economics at Bloomberg. and this is Trumponomics, the podcast that looks at everything in the economic world of Donald Trump. Now, we're recording this Wednesday morning, the 23rd of September. We're several hours ahead of President Donald Trump meeting Chinese President Xi Jinping's plane in person at Andrews Air Force Base just outside Washington. Now, there'll be two notable features of this tarmac encounter. The first is that no one can remember any previous president taking the trouble to go. I guess he probably won't need to hold up a sign with the Chinese leader's name on it.

3:03Stephanie Flanders:The second historical quirk is that the event is likely to be recorded only by a Chinese state TV camera, the regular White House pool cameras being on strike in response to CNN and two other outlets being banned from the White House. Now, the value of these lofty summits of powerful men is supposed to be they can talk directly on the major issues of the day, cut through bureaucratic hurdles, get to the meat of the problem, maybe even get closer to an answer. But that's not how Donald Trump rolls. And that's not been the mood music around this summit. In fact, the risk is the mood music and pageantry might be pretty much all there is, when in fact, there really are some big issues we might want these two guys to get to grips with.

3:48Stephanie Flanders:Not least, the prospect that in America and China's great race for AI supremacy, humanity might be accidentally wiped out. Now, President Trump has dismissed most of those kind of concerns as a hoax. Beijing, for its part, has signalled it is willing to discuss a narrow category of genuinely dangerous AI capabilities. But China also wants Washington to ease restrictions on advanced chips it needs to power its AI models. And trust between the two is low. So what can we realistically expect to come out of this summit? And what would be needed to reduce the risk that AI will kill us all? Well, joining me from Hong Kong again is Dan Tenkate, Bloomberg's Executive Editor for Economics and Politics in Asia.

4:36Stephanie Flanders:Dan, thanks for joining us. So late in your day. Great to be here. And joining us from Washington, our guru on all things AI, Michael Deng, Geoeconomics Technology Analyst at Bloomberg Economics. And before coming to us, he served as a policy advisor and presidential management fellow in the CHIPS Program Office at the US Department of Commerce. Michael, thanks for coming back. Yeah, of course. Happy to be here. Now, Dan, you've recently returned from Beijing, and I know you're in close contact with our team there. What is the mood around this summit sitting where they sit? Yeah, I was in Beijing a couple weeks ago talking with people there as well.

5:16Definitely China wants stability out of this summit. The expectations are pretty low for any deliverables. I don't think they have too much ambition for what could actually be accomplished. They would like to see an extension of the trade truce for two years through Trump's term. That's not going to happen. Jameson Greer has already made that pretty clear. But nonetheless, I think China views the leader to leader diplomacy, as I like to say, between Trump and Xi as something that will stabilize this relationship. and they also think they could probably keep Taiwan in check, they could keep exports going.

5:58So they have a lot of interest in just meeting and keeping the relationship from blowing up.

6:03Stephanie Flanders:One of the phrases I've heard a lot about this summit has been, calmer but not closer. There's a sort of status quo that they've worked out over the last year or so in trade front and various other things and there'll be a continuation of that. But as I mentioned at the start, There's so much discussion in the last week or two, led in part by the leaders of the main AI companies around AI safety and the risks to future development of AI. Is that discussion as live in Beijing as it is in the US and other parts of the Western world? Not in the same way. I think they're basically technology optimists.

6:42They think that AI will create more jobs than it will destroy. But the focus there is really employment. They're worried that the transition to AI will lead to a lot of job losses and that could lead to instability for the Xi government. And so that is where most of the discussion is focused. Not so much on the safety risks. I think China views themselves as very much behind the U.S. in this race. They think they can catch up, but they see themselves making progress and they don't really want to slow down on that front. At the same time, the country's top spy chief had a paper that came out within the past couple of weeks that identified a lot of risks to AI.

7:25And surprise, surprise, the top one he flagged was a risk to China's political security, so the regime itself. And I think that that shows you the thinking. Of course, it's always about the Communist Party. And if they see a threat to their own power, then we've seen them be able to take action very quickly. So they are watching that very closely. But there's not a lot of talk about extinction of humanity or anything like that.

7:50Stephanie Flanders:Not yet, baby. You know, we've long framed this as a race between the US and China, but safety is now coming much more to the fore. What usefully do we think they can discuss on that topic? So I would say for this dialogue in particular, I would keep expectations pretty modest. I think from everything we've heard so far and that we might hear once the dialogue starts, the expectation should largely be about beginning to set up this process rather than anything seriously concrete or that actually advances any key objectives. We've heard Secretary Besant say that there's some kind of incident notification mechanism.

8:30There's other reporting that indicates that may largely just be an open line of communication between him and Helifeng. So I think the key here is to establish trust and get a dialogue going in the first place. We may hear positive signals about that, but I think the challenge is going to be once they run into having to agree on basic definitions, for example, about what constitutes an advanced capability, what's normal technological competition, what it might require of each side in terms of slowing down, which both sides now have denied wanting to do. How do they get their governments and these companies to actually commit to it and to be serious about these safety commitments?

9:06So I think anything we hear out of this dialogue initially is going to be quite easy. And the hard part is going to come later, potentially at the Shenzhen meeting that's mentioned in two months.

9:15Stephanie Flanders:Scott Besson, the Treasury Secretary, he had this kind of prior discussions with his Chinese counterpart. And then there was this discussion around starting, or there was a somewhat of announcement saying they've agreed to establish an AI dialogue. I noticed now, maybe this is just, but the AI dialogue with no capital on D. So it's not like an official ongoing, it's not like the various commissions and the sort of committees, the joint economic committees that we've had in the past or anything like that. It is just literally a conversation and they've each got each other's number kind of thing.

9:47Yeah. And let's be clear, they were going to talk anyway. And so Besant and Helefeng have been talking since Trump's Liberation Day tariffs. They were talking all last year, they struck the trade truce. And this is now the main mechanism of communication between the US and China on the relationship overall. So they just added AI to the list right now. There's nothing else there for the moment. And so that's where it's going to stay unless they put some meat on the bones. But as Michael said, it's really, really thin at the moment.

10:20Stephanie Flanders:Running through a lot of this is a sort of basic level of trust. So although there was this sort of agreement, we're going to carry on talking and everything else, you know, I can't help remembering that Scott Bessent was also the person who said recently that there's no day after tomorrow if China wins. I mean, he's talking up the race aspect, even as they sort of pay lip service to some of these security concerns. So where are the points of tension? If they were going to do what we wanted them to do, which is to seriously discuss some shared framework for at least the extreme risks that people like Dario Amadei are worried about, where would the focus be and what are the obstacles to doing that?

11:02Yeah. So I think in any discussion, if we talk about the potential risks first that they want to discuss, I think you could largely group the problem with AI safety into three categories of risk. I think the first one is probably malicious use or the idea that you and a model could enable third party rogue actors or criminals to conduct cyber attacks, et cetera. I think the second one is unexpected autonomous behavior. So an AI agent sort of gaining access or taking actions that the creators didn't expect. And this is largely actually what happened over the summer with many of these cyber incidents.

11:34I think the third is international miscalculation. This is probably the most familiar in terms of government-to-government dialogue. So if you imagine a major AI-enabled incident hitting either government and making sure that it wasn't the other one doing it, I think the notification mechanism we've gotten so far is probably most useful for the third. I think progressing on the first two is going to require greater levels of trust in some kind of actual concrete action or commitment from either side. I think the difficult part that comes from this is then settling on joint definitions. For example, China has signaled they're happy to talk, but they want to draw a very clear line between normal technological competition and this kind of fenced off area of advanced capability or advanced risk.

12:15I think the problem is that line is, it's very fuzzy and many of the capabilities or sort of training methods that you would use to just simply improve your models within the normal tech competition arena inevitably will bleed into this higher area of risk. And then therefore, what you've seen from Anthropic and OpenAI is they have sort of self-volunteered or done by themselves to sort of weaken their models in these areas. The government hasn't, until very recently, actually required them to do so. So if you would want China to actually take substantive action on this, how do you convince them that in this narrow area you should specifically slow down your companies when you're in a race with the U.S.

12:53as well and worried about being behind. And so that is what really requires trust between these two sides.

12:58Stephanie Flanders:And that runs pretty deep down. I mean, the perception, which is not wrong, because you hear cabinet ministers, cabinet secretaries saying things along these lines fairly regularly, that the US wants to slow China's development in this area. Are there any areas that you think the leadership would be willing to say, okay, we recognize that this is too important to be subject to this lack of trust? I think China definitely sees it on that high level. Like, we should treat AI like nuclear weapons and we should come and agree not to kill each other. I mean, that is essentially something that is in their interest and that they would want to do.

13:38But as Michael said, the line in there is very fuzzy. I mean, we often portray AI as a race, but it's kind of a race that doesn't really have an end per se. So when Scott Besson says, okay, there's no day after tomorrow China wins, the Chinese researchers I was speaking to in Beijing, they see this going in waves. There's going to be developments where China narrows the gap and then the U.S. will have something and the gap gets a bit wider. And then so there's not a linear kind of race and then somebody wins and then the world is over type of scenario. China really views this as a long term thing.

14:16they want to be seen as an equal partner, I guess, in any of these talks. So yeah, anything that appears like it's the U.S. trying to slow China down so the U.S. can get an advantage, China would be very wary of. There's just not a lot of trust between these governments, and they're very skeptical of what's coming out of Washington.

14:37Stephanie Flanders:So that's why there's so much distrust on the China side. But Michael, as you've pointed out in your piece. On the US side, what has reduced trust, made them less open to this conversation, is this issue of distillation. So basically, it's about, once again, China finding a way to steal a US technology. But talk us through it. So at a high level, distillation is really a normal AI development technique. And I think that's where some of these definitions get muddled. So essentially, it's about using the outputs of a stronger teacher model to help train a weaker student one. There's various techniques that you can classify as distillation, but that's probably the most general one.

15:17What's happened on the US side is they specifically said that China is conducting these exploitative distillation campaigns against US models where these US models terms of service specifically prohibit them to do so. And they are using all kinds of methods to basically setting up false accounts, setting up routing methods, essentially to, And even Anthropica has accused them of essentially rerouting traffic in terms of prompts being sent to a Chinese model. They are then sending it to Claude, collecting the answers and training on the answers and sending it back to the original user. The Chinese response has been, of course, that this is normal technological competition, that this is a widespread technique in the industry.

15:57And it is. But I think the difference is it's a widespread technique in the industry when you use your own models or when you use it on models where they have allowed you to distill from. What China is doing is a little bit beyond what the quote unquote normal development technique is in terms of they're conducting these campaigns specifically against closed source models that don't want them to do so. And so that's the tricky part. And the difficult thing is it's also quite hard to stop. It's not really like many of the hardware export controls that the U.S. has implemented over the past five years where there's a physical device and a shipment that you can track.

16:32This is really about, you know, at the core of the core of API access. Again, when we talk about fuzzy or clear lines, it's very hard to distinguish between what's an illegitimate user and what is a Chinese AI firm trying to distill, because both are just sending prompts to your model that it's trying to respond to. And so this is one of the biggest areas of distrust from the US. The AI labs are upset about it. Many security agencies are upset about it. Congress is upset about it. And if there was one thing that could really destabilize this conversation moving forward, when we talk about what is enabling China to have these capabilities in the first place, I think it would be this issue.

17:06Stephanie Flanders:Hearing what you say about distillation, I mean, it would be a helpful thing for the president and President Xi to discuss. Let me put it this way. I mean, President Trump doesn't tend to embrace that kind of technical detail. So it just doesn't feel like it's the kind of thing that's going to be on the talking points. No, I think President Trump's understanding of artificial or super intelligence as he now brands it is somewhat limited. And generally, he seems to be representing points of officials or more technical officials and issues that they bring up in his agenda. I think he would bring this up as advocating again, how the US is number one, how we're in the race out of China.

17:46But this kind of distillation issue seems a layer or two below what he would actually be willing to engage in. And it would actually fall in the realm of something like this dialogue, which makes it even trickier in terms of allowing it to succeed.

18:01The thing about AI for business, it may not automatically fit the way your business works.

18:07Stephanie Flanders:At IBM, we've seen this firsthand. But by embedding AI across HR, IT, and procurement processes, we've reduced costs by millions, slash repetitive tasks, and freed thousands of hours for strategic work. Now we're helping companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business. IBM. This is Robert Smith from Business History. If you're listening to this, there's a good chance you're a small business owner. And like every small business owner, you started with a dream to do what you love and watch it grow. What you probably didn't dream about?

18:43Keeping up with cyber threats. That's where MasterCard can help. With access to tools that help identify cyber threats to better protect your business. Building your dream business? Priceless. For cybersecurity in a changing world, there's MasterCard. Learn more at MasterCard.com slash smallbusiness. Join Bloomberg for the Canadian Finance Conference, proudly sponsored by National Bank of Canada Capital Markets on September 29th in New York. Hear from influential corporate and government leaders as they discuss the strategies shaping Canada's economic future. Connect with senior decision makers, gain actionable insights and be part of the conversations driving business forward.

19:23Register at BloombergLive.com slash Canadian Finance. That's bloomberglive.com slash Canadian Finance.

19:56million business and enterprise users worldwide already using ChatGPT for work. Download the ChatGPT desktop app or contact sales to learn more.

20:15Stephanie Flanders:Dan, one thing I've noticed that sort of has crept into these conversations, although we talk about the race and we've tended to focus on the frontier, is the biggest story of the last six months that Chinese models, despite being less sophisticated, have actually vastly extended their reach in terms of usage because they're so much cheaper and because they have that open source approach. I mean, in that sense, the Chinese feel like they're coming out ahead, even if they're still behind on this kind of frontier sense. Yeah, exactly. I think China views this as a public good. Of course, they're behind in the race, So they have an advantage using open source models, but they're also undercutting the potential valuations of these big companies like Anthropic and OpenAI in doing so.

21:03They're also winning a lot of support throughout the global south because a lot of people can use Chinese AI very cheaply. There is a question about how some of these companies can monetize that at the end of the day. But for the moment, China is getting people hooked on its models. And I guess it depends how sticky that is going forward. You know, if you settle into a certain routine using whatever AI you're using, then maybe you just stick with that. And then maybe you buy add-on products, whatever, as long as that infrastructure is laid down. And so China, at least in my conversations, they plan to stick with this open source model.