What does the recent Hugging Face hacking incident teach us about the future of AI?

0:00It's Consider This where every day we go deep on one big news story. Today, a hacking incident and the future of AI. Earlier this month, OpenAI, the company behind ChatGPT, said two of its AI models were involved in a, quote, unprecedented cyber incident. The models autonomously hacked into another company, meaning acted on their own without direction from humans. We thought that was really unprecedented. We thought we had never really seen anyone, you know, having this type of event. That's Thomas Wolfe, a co-founder of the company that got hacked, Hugging Face. He says his company is used to being hacked, but this felt different.

0:42For the first time here, we had this, you know, this massive wave of 17 ,000 events occurring, roughly in parallel. Very, very, just like a whole swarm of attackers. Incidents like this, where AI models take autonomous action, have some people scared. Here's Democratic Congressman Ted Lieu. He recently introduced a bill in Congress called the AI Kill Switch Act. I would make sure that we have adequate testing and disclosure on the most advanced models and make sure they don't do catastrophic things like build a new virus that could cause a pandemic or cyber hack every financial institution. Consider this.

1:25The hacking incident between open AI and hugging face shows the power of AI models to act on their own. How could that shape the development of AI technology? And should we be feeling panicky?

1:41From NPR, I'm Sasha Pfeiffer. This message comes from Dell Technologies. Get long-lasting battery life on the Dell XPS laptop powered by Series 3 Intel Core. Now from$699 with exclusive student pricing from$599. Visit Dell.com slash deals. This message comes from Thumbtack. Some tasks can feel easy, but home projects can bring second guesses. Is that noise normal? Is that water damage? Who should be called? That's where Thumbtack comes in. Just upload a photo or voice note and it uses AI-powered search to match with the right top-rated local pro. So instead of guessing, there's clarity and confidence when hiring.

2:26For your next home project, try Thumbtack. Hire the right pro today. GLP-1s are best known as weight loss drugs, but they may also help people cut back on alcohol, cigarettes, and opioids. They seem to work for many types of addiction. Maybe these effects are much broader than we previously thought. Ideas about the brain, addiction, and empathy. That's on the TED Radio Hour podcast. listen on the NPR app or wherever you get your podcasts.

3:01It's Consider This from NPR. It's a science fiction nightmare. What happens if the machines take over? The world got a potential taste of that last week when OpenAI, the maker of ChatGPT, said two of its models had autonomously hacked into another AI company. OpenAI says its technology carried out the attack during a cybersecurity test. Its models broke out of their so-called sandbox and accessed the private servers of another AI company. The incident has highlighted fears that losing control of AI could pose broader safety concerns for the real world. So I asked Thomas Wolfe, the co-founder and chief science officer of the company that was attacked, hugging face, what he believed the rogue model was trying to do.

3:47Everything here is something like a normal human hacker would have done. Really, the surprising thing is that the AI was not at all tasked with attacking us, thankfully, but actually was trying to solve this challenge. So the challenge is you give a software vulnerability that is known to an AI, and you test the AI with trying to use that as what we call an exploit, which is trying to use this software vulnerability to take control of a computer. And what the AI decided to do in this case was to go on the internet and to try to find the answer to the challenge directly, just like someone sneaking out in a professor's office to basically grab the answer to the test.

4:28Your CEO has said that he does not believe there was any malicious intent on the part of opening AI, that it didn't set out to hack you. But this is the kind of security breach that taps the greatest fears that people have about how powerful AI can be. So you are, as we said, the chief science officer of an AI company. How would you address those fears? Well, I think we need to monitor this frontier system. A frontier system being a very cutting-edge AI tool. Exactly. Being able to detect better what they are doing, understand better, you know, what is the current task that they are actually doing?

5:05And it's quite likely that in this case there was just maybe not enough monitoring of what the system was doing in place. You know, the media coverage of this has been fairly alarmist. It's a fear of what could be. But you seem not very concerned. How would you describe your level of concern? I would say informed level of concern, which is I know the remediation tools for this are here. I don't think there is anything new that we need to invent to be able to monitor what is AI doing right now. It's mostly just a wake-up call to take these things seriously and to actually take security seriously.

5:45There is currently a debate which is around should we allow open-source systems to be deployed as well, or should we only have closed-source systems? And your company is open-source, Hugging Face is open-source, is that right? Our company is open-source, yeah. For people, by the way, who aren't familiar with open source concerns, the fear, as I understand it, is that if it's open source, meaning the public can come and use it, it's more likely to be used by malicious actors who are trying to do malicious things with it. But you're saying that this incident shows the importance of more open source AI models.

6:20Why do you think that? Yeah, exactly. So this system is surprising because it's the exact opposite of what people were thinking. So we were expecting, just like you said, that an open source system would be used to attack someone because they are slightly easier to access, I would say. In practice, what we discovered in our case was that we were attacked actively by a closed source system. And when we tried to defend ourselves with closed source model, they just decided not to help us because they said this is too similar to an attack. We're not allowed to help you with that. And so we had to turn out to actually an open source system to defend ourselves.

6:56And my feeling is that you need to have access to them as well when you want to defend yourself. You describe this as a wake-up call and a reason for more transparency. If that doesn't happen, what do you think the risk is? I think if it doesn't happen, the risks are quite strong. And if there is no more transparency, right, and no more access to model to defend yourself, we basically end up in a situation where we have a lot of concentration of power in just a couple of companies who basically don't have to answer to anyone or to anybody. I think that's a pretty dangerous future for the field.

7:41We've always pushed for open source and transparency. We think the more the public can understand what's happening, and the more people can, like third party, can monitor or can do post-event forensic to understand what happened, the better it is for everyone. The good thing here is that OpenAI was pretty transparent. I think that's something we really want to commend and we really want to see more and more in the future, which is this company not fearing repetition risk or from the right incentive being incentivized to actually publicly explain what happened and be more transparent. I hope that's the good takeaway of this event.

8:22That is Thomas Wolfe. He is the co-founder and chief science officer of the AI company Hugging Face. Thank you for talking about this. Thank you, Sasha. This episode was produced by Elena Burnett and Catherine Fink with audio engineering by Ted Meebane. Our director is Jonas Adams. It was edited by Mallory Yu and Tinbeet Hermes. Our interim executive producer is Courtney Dorning.

8:48It's Consider This from NPR. I'm Sasha Pfeiffer. support for npr and the following message come from washington wise decisions made in washington can affect your portfolio every day washington wise from charles schwab is an original podcast that unpacks the stories making news in washington listen at schwab.com slash washington wise the most compelling global stories straight to your ears that's what we do at state of the world from NPR. Developments in Ukraine, Iran, with Ebola, but also how British beavers are fighting the effects of climate change. So we said the beavers can do it probably a fraction of the cost, certainly more sustainably.

9:30Listen to State of the World every weekday on the NPR app or wherever you get your podcasts.