People Are Finally Aware Of 'The Real Dangers AI Poses'—Now What? AI Policy PAC Founder Weighs In

0:02Hi, everybody. I'm Brittany Lewis, a breaking news reporter here at Forbes. Joining me once again is former Congressman Brad Carson, who's the president of Americans for Responsible Innovation, as well as the founder of Public First Action Pack. Congressman, thank you so much for joining me.

0:18Brad Carson: Thanks for having me back on the show. I'm really grateful for another talk with you on artificial intelligence and to discuss the moment we're in right now, because international conversations surrounding AI took a really dark turn over the last couple of weeks. A whistleblower from one of the major companies, he worked previously at Anthropic and OpenAI. He sounded the alarm on the technology's potential to kill us all. That was a direct quote. After that, industry leaders called for a industry-wide AI slowdown and more oversight. And two of those CEOs of the frontier AI companies, Anthropic and OpenAI, they addressed the UN Security Council about the pros and cons of AI's potential.

1:02And I want to focus more on those safety concerns to start off the conversation. And here's what they said in front of the UN Security Council. Anthropic CEO Dario Amadei, he said this. If managed poorly, I even believe AI could be a risk to humanity as a whole. And the CEO of OpenAI, Sam Altman, said this, we could lose control of the future to AI. So to start off this conversation, what are your thoughts on the global dialogue surrounding AI?

1:28Brad Carson: I think people are finally waking up to the real dangers that AI poses to us. People in the industry have been talking about this for a few years now, but it never really broke through to the broader public. But the Hugging Face incident in July where OpenAI's agents broke into this European AI repository called Hugging Face kind of brought this to the fore. And we have learned after Hugging Face that agents from Anthropic, Google, and OpenAI have broken into another, a lot of different systems. Indeed yesterday, we learned that they broke it into the Australian government and some systems in the Australian government.

2:05Brad Carson: and making kind of the problem worse. OpenAI didn't reveal that to the Australian government until about three months after it had occurred. So the real power of these agents, the danger they pose, is I think now becoming more broadly aware and people are going to demand some kind of response to it. What do you think the response should be? Because I've spoken with bipartisan lawmakers and they've called for regulations, they've called for guidelines, they've called for guardrails and laws implemented to kind of put these safeguards in place. President Trump seems to have little appetite for that, according to his recent remarks.

2:44And I'll just say a few, because he said over the phone and a text summit earlier this month, the robots will not be taking over. The AI will not be taking over the rest of the world. The whole thing is a hoax. And he said that the United States, he said this at the United Nations General Assembly this week, The U.S. rejects any attempt to construct a globalist scheme to control for the artificial intelligence being spoken of so much now. And he also said, I'm not going to stifle growth of something that will be bigger than the Industrial Revolution. And we will only encourage superintelligence.

3:16We're going to encourage it, not rein it in. And he has also now referred to AI as superintelligence. What do you make of President Trump's remarks on AI in this moment?

3:27Brad Carson: He says the things in that comment that are very true. This is probably going to be bigger than the AI revolution. And we don't have to have a globalist solution to it, to use his term. We need to have a domestic solution to it. These frontier lab companies are all here in the United States. They're all based in San Francisco or out in Silicon Valley. So we need a domestic response to this. But I think what people need to realize, the president himself, is that AI is only going to move at the speed of trust. and the American people have lost their trust in what's happening in San Francisco among the AI companies.

4:04Brad Carson: And so if we don't do things in this country to rebuild people's faith in this project, they will stop it. We already see this with data center revolts. You know, there's a lot of people who are deeply antagonistic to AI throughout our economy. So, you know, we have to rebuild trust. And the one thing I take solos in is the president has been shown to change his mind when new evidence appears. And if you talk to him a year ago, we were never going to touch AI. There was not going to be no government regulation of it ever. Then Anthropic released Mythos, this very powerful model that has cyber capabilities, and the government took it over.

4:41Brad Carson: But the government prohibited its release. And they now, in the West Wing, have a group of experts, the Treasury Secretary, Sean Cairncross, who's the National Cyber Director, who decide what could be released or not. So the president has been shown to change his mind when new evidence appears. And my hope is that he'll pay attention to the new evidence that is every day appearing. How do you think people regain the trust or gain the trust to begin with from leaders in this space? Because critics have said, you know, it's really rich hearing these industry leaders saying, hey, we need some guardrails to rein this in.

5:16We need to slow down the pace here. Please help us. After they've made billions of dollars in this industry. I mean, what do you make of that, that the leaders here are saying, hey, the technology we made has the potential to kill us all? Because if any leader of any other industry said that to a group of people, I mean, we would be horrified and we would say, hey, shut that down. We don't want we don't want that risk to even be out there.

5:42Brad Carson: You're right. That's why we should take them very seriously. I mean, they're admitting things against their own interests and they're not being insincere about it. they do all believe this, especially the people you often hear from, like Jacob Coxham, who you mentioned, came forward and resigned from Anthropic last week, and it made a big wave. Then he had people at Anthropic kind of double tap his comment to say they thought the probability of doom was 10 % or greater. These are people who work inside the labs, and many of the people talking are not C-suite people. They're not the chief executives who you might say, well, they're talking up their IPO or something.

6:16Brad Carson: These are people who are actually making it. They're They're the coders. They're the engineers making it. So we should take them very seriously about this. And I think what they're pointing out, this is why AI is such a hard problem for us. It's like an ambivalent technology, by which I mean, like, it can bring some really good things for us. You know, in my pocket, my phone is now a personal oracle. I ask Claude or ChatGPT something. It tells me everything I want to know. That's amazing. On the other hand, it has a lot of risks. So how do you get the best while avoiding the worst? That's a really hard political problem.

6:47and I think when you're talking about AI there's a lot of nuance in this conversation like you said there are a lot of upsides I mean it improves and enhances everyday life right I am horrible with directions now I have a map in my pocket to get me from point a to point b at all times at the same time aside from that I mean people have been saying the science advancements can be huge. This can be life-saving. This can really fundamentally change humanity as we know it. At the same time, downside, hey, it could just wipe out all of humanity. There's a non-zero chance that could happen. So how do you balance that?

7:26Do you think the pros outweigh the cons? What do you think about all that?

7:30Brad Carson: I think it's a close run thing, whether the pros outweigh the cons. But I analogize it often to pharmaceuticals. But we know like taking too much of this pharmaceutical can kill you. Many pharmaceuticals have serious side effects. On the other hand, many pharmaceuticals have saved lives, right? It's revolutionized our society. We're living longer, better, healthier lives. So I analogize them to that. And we have a lot of regulatory schemes that deal with pharmaceuticals to like tell people like, well, you need to have like a doctor's prescription for this, or here's like clear instructions. The pharmacist has a duty to you.

8:03Brad Carson: The doctor has a duty to you. The companies making the drugs have a duty to you. That's what I think we have to do with AI. It's like an amazing technology. It can kill you. It can help you. You know, there's a way to like structure the law for smart, where we avoid one fate and get the best of it. President Trump, he posted that he is going to be forming an AI force. He will be announcing in the near future an AI czar. And he said, we're leading China and the rest of the world. And I intend to keep it that way. What do you make of that? How do you think the United States can lead on AI responsibly?

8:41Brad Carson: Well, I'd like to see the details of the AI force. You know, it sounds like a pretty good idea to me, actually, because we need more technically competent people in government. And I don't know whether this is going to be a Pentagon organization, kind of like the Space Force was, or we imagine some kind of civilian core. But in either event, we need to bring people who are AI literate into government or the military. So I'd like to see the details, but notionally, it seems like a good thing. An AI czar, we kind of had that already. David Sachs was the AI czar, at least in kind of representation.

9:14Brad Carson: So sure, bring someone in to coordinate AI policy. That makes sense as well. I think we can be the leader because when we worry about China, and that is a very legitimate worry. China's a competitor. But if we have export controls, controls on semiconductor manufacturing equipment. If we don't let the chips that you make AI on be sold to China, we can leave them. They cannot overtake us. But sadly, sometimes the administration turns a blind eye to what we know is smuggling of these chips into China. So the great Chinese models that are out there today, and there are some really good ones, they're all made on American chips, NVIDIA chips.

9:52Brad Carson: And that's against American law. We have export controls on them, but they're not really being enforced. And NVIDIA turns a blind eye to it as well. So we really need to crack down on that. And that can ensure America can lead for years to come. Do you think that's going to be mentioned today at the summit between President Xi and President Trump? Because President Trump, he mentioned in a true social post that artificial intelligence, well, now he refers to it as super intelligence, will be a big topic of discussion in that summit. And he said that he and she want to leave AI exactly where it is and said our guardrail is the DOJ.

10:29But you're saying NVIDIA, the United States, turns a blind eye here to those chips. What do you what do you want to be discussed in that meeting in reference to this?

10:41Brad Carson: I think a couple of things. I would like them to get after the chips question and like stop the smuggling of these chips. You know, I think, though, that President Trump and people around him, the Silicon Valley billionaires around him, they actually oppose these export controls. They're fine to sell the chips to China. You know, they have a law against it, and Congress is strongly for that law. And so turning a blind eye is their kind of backhanded way around it. That's not a good approach. So I don't expect the president to bring that up. What I hope he does bring up are a couple of things. One is reaffirming the commitment that President Biden had with Xi.

11:17Brad Carson: where we don't have AI involved in nuclear command and control. That needs to be a human enterprise. I don't want AI launching nuclear missiles. And we have an agreement. They should reaffirm that. That's a key thing. And then more broadly, more communications. I think a deal with China on AI, which would be a good thing, is hard to reach. It's very hard to verify. I don't know that China is interested in that in any event. But we should begin more talks between senior officials, AI experts, it's their equivalents in China, just where you talk. Again, ping pong diplomacy, panda diplomacy, like we had 50 years ago with China, where lower level people are engaging in activities, talking about AI with them, where that can build more trust, maybe over time lead to some kind of agreement that limits AI from doing the worst things to the planet.

12:08I know that some people have said this summit's largely symbolic because of its length and how many topics need to be hit on. But when it comes to AI, do you think that I know it would be too short to hammer out any sort of extensive agreement and you don't think there's that much appetite for that, but do you think they will even come up with a framework agreement? Do you think that's too much or do you think it's just going to be, hey, we're open to discuss if you are? And globally, do you think that's the right approach?

12:39Brad Carson: It's probably not the right approach. I'd like to see an agreement. That'd be amazing. But of course, China has to be a willing partner and you have to verify it. We can't just trust them to get it going. I don't think you'll even see a framework. I'm not sure President Trump wants a framework. I don't think she does. What I hope comes out to this is an agreement to talk more. Already Scott Besant, the Treasury Secretary, in his preliminary meetings to the summit has said that they're going to do more of that. I think it's really important. Besant should talk to his equivalent in the People's Republic of China and start talking to each other to build trust, build knowledge, you know, to make sure the more conspiratorial views, you know, are like, are put in the box and say, like, here's what we're doing.

13:17Brad Carson: So better knowledge, more trust, more familiarity with one another would be a good thing. That's the best we can hope for right now. And do you think on the conversation of China and these chips, I know President Trump has repeatedly said, we're leading China, we want to lead China, we want to continue to be the global leader when it comes to AI. If there is still this blind eye to China getting these American ships, do you think there is a chance that China could overtake the United States in this AI arms race? There's a chance of that for sure. It will certainly keep them close to us. You know, right now people say they're six months behind us, eight months behind us.

13:57Brad Carson: So not very far behind, but they're about 10 years behind us in this equipment. They can't make their own semiconductors. They can't make all the equipment that then makes the semiconductors. We have a decade or more lead there. So the fact that they're so close to us is purely the result of us selling them our chips. Because what they do have in spades are amazing engineers being turned out from amazing universities at scale. You know, they're turning out from Tsinghua and Peking, you know, thousands of brilliant engineers every year, far more than we are in the U.S. And so they have engineers and they have a lot of technical, like, acumen.

14:34They're smart about these things, but you can't do without the equipment.

14:37Brad Carson: So we're giving them that secret ingredient they really need. So, yes, if we keep selling these kind of things to them or letting them be smuggled in, you know, it's quite possible they catch us. It's certainly possible for that. And do you think that because, hey, they don't have the equipment yet, could they build AI that teaches them how to make that equipment, that builds that equipment themselves? I mean, what could that look like? I'm just thinking, it seems like because this technology is so expansive, the possibilities really could be endless. Yes, that's exactly right. In the US already, we're trying to use AI to build better chips, right?

15:15Brad Carson: AI building systems that help us make more efficient equipment and things like that. So it's certainly possible if they had a couple more breakthroughs, you could have AI help you solve this problem. Because building chips is an amazing process. It's the world's most complicated industrial process today. ASML in Europe makes this equipment that helps you make chips. They have a monopoly on that globally. You can't even compete with them because they've got decades of tacit knowledge and things like that. But AI could speed that up. And so it makes no sense to anyone unless you're an NVIDIA executive to be allowing these chips to be smuggled in.

15:52Brad Carson: Jensen Wong openly wants to sell chips to China. He wants to. I understand that. It helps NVIDIA's bottom line. It's a huge market, and he doesn't want them to develop their own chips. But it's so contrary to American interest that every president should be against it. My last question to you is, I know you are a former lawmaker. What do you want to see from lawmakers now in this moment? I want to see both of the problems we've already talked about addressed. You have widespread Republican and Democratic support on the Hill for cracking down the smuggling, you know, both to have Congress have more say about when you can give export licenses, to actually try to put devices in the chips that can geolocate them.

16:36Brad Carson: So we know if they're being smuggled into China, through Singapore or Malaysia, which are the usual routes for that. And also on this so-called semiconductor manufacturing equipment, things upstream of the chip, where they're used to make it. We're trying to crack down on that, too. Huge Republican support for that, huge Democratic support on the Hill. So that's one part of it. The other part of it needs to be getting after the models themselves to make them safe for us. So I want to see mandatory evaluation of these before they're released to the public. You know, are they a hazard to us in cybercrime, novel pathogens, loss of control?

17:10I want to actually, because we now know some of the major problems are not from actually

17:15Brad Carson: models released to you and me. They're not to the consumers. They're happening inside the lab. So they're using better models still to help create the new models. But those can be quite dangerous. And so they need to be telling the government if they identify any dangerous capability. And the idea, for example, that OpenAI doesn't tell the Australian government for three months that their system broke in, hacked the Australian government. but there has to be much better incident reporting you know sam altman needs to be on a hotline to the australian prime minister saying i deeply apologize let's figure out how to make this right how this doesn't happen again instead it happened like just through the back door and kind of gossip you know open ai sent just kind of a casual email to the government that didn't even get read i think it might have even bounced back you know that was like how they handled it like like some kind of like mail at australiangovernment.com is all they did those are the kind of things like the government.

18:07Brad Carson: She's like, that's not adequate. Well, there's certainly a lot to look out for in this conversation. I always appreciate our talks and you making sense of the moment we're in. Brad Carson, thank you so much for joining me. I hope to talk soon. My pleasure to be on. Thank you so much.