Special Edition: Hugging Face CEO Clement Delangue Talks OpenAI Hack

0:00Ed Ludlow:What would you do with 30 extra minutes a day? With LG, that time starts coming back. LG appliances are built to make everyday life feel lighter. With LG, fewer laundry mistakes, fresher groceries, and easier cleanup add up to more time back. 30 extra minutes isn't just time saved. It's breakfast without rushing, dinner without stress, and one less thing to juggle. Smart performance, sleek style, and reliability for real life. That's LG. Get your time back with LG. See the latest models and savings now at LG.com. Bloomberg Audio Studios. Podcasts. Radio. News. We are here because on July 22nd, OpenAI and HuggingFace disclosed that two powerful AI models went out of a sandboxed or closed testing environment, gained internet access, and then were able to hack HuggingFace's systems in what was seen at the time as a watershed moment in AI safety.

1:03Ed Ludlow:And now an investigation has been in part concluded and there's been time to assess what happened. Hugging Face CEO Clem DeLong is here to discuss exactly that. And I think, Clem, that's probably the best place to start. What happened? A lot happens right in the past few weeks. We noticed this closed, as you mentioned, the first kind of like public instance of an autonomous AI cyber attack. We defended ourselves against it with an open model, interestingly, coming from China. And we learned that this came from open AI. Usually when you think about cyber attacks, you think about nation states, you think about, you know, hacker groups.

1:53You don't really think about, you know, one of the most prominent American AI company. And obviously, since then, we learned that Entropic was also facing some of the similar issues. So really, you know, unprecedented conflict event happening here, new developments in the saga of AI.

2:17Ed Ludlow:We will discuss what has been highlighted by this, which is closed versus open debate. China's work on AI, America's response to this. But going back to the very basics, this was two powerful models from OpenAI, right? One released, one unreleased. But they had their guardrails lowered for the purposes of evaluation. OpenAI said to the models, they instructed the models, go out and do something. And I think it would be useful to the audience for you to explain that part, whether the OpenAI models were just following instructions or if they were AIs that went rogue. Yeah, I mean, I joked with the team a few days after it was announced that sometimes we ask agents to think outside of the box, but we don't want them to think outside of the sandbox in that case.

3:13I think it was a mix of mistakes and the systems internally, I think, weren't good enough to prevent this to happen. After that, on our side, the attack was really interesting. Over 17 ,000 different actions taken over four and a half days. So the speed and the volume of the actions taken were nowhere close to what like human cyber attacks could be. It wasn't particularly smart or sophisticated. It was more kind of like someone described it as a bear probing really everything in the system to try to find the honeypot in a way. but obviously because it's an autonomous AI system it does that pretty well.

4:18Ed Ludlow:After the disclosure on July 22nd you got on an airplane and flew from Miami to San Francisco in part to get with the OpenAI team and do this investigation. You just said that the systems were not in place to prevent this happening. Talk more about that. Prevent what happening? Where were the points of failure? Yeah. Well, I can't talk for them, but from what I've understood from what they released, and I think they're going to release more in the coming days, which I'm excited about. You know, they had kind of like an evaluation sandbox, right, which was supposed to be like a contained environment for the models.

5:02And unfortunately, you know, there were some weaknesses that led the agents to be able to get out of it, get access to internet, and decide to run a cyber attack against hugging faces. So that's one first kind of things that I think we can improve in the future, now that we understand that these systems are capable of that. Obviously, I think that the monitoring part is important, right? Because the faster you can detect that, the better it is, right? We learned about some entropy instances that they haven't, you know, seen or detected that happened three months ago or something like that. So obviously we want to monitor these systems better.

5:49And then one last interesting thing, as I mentioned, we defended ourselves with an open model, right? And some of the guardrails prevented us from using Frontier APIs to defend ourselves. So kind of like improving the tools for defenders, instead of obsessing about not giving them to attackers, I think would be a good thing in the future to make sure these incidents are not too harmful.

6:15Ed Ludlow:Okay, you've taken us there, so we'll go there. In this incident, it was two powerful but closed OpenAI models where OpenAI lowered the guardrails in place to test their full cyber capabilities. they escaped the sandbox or testing environment gained internet access and were able to access your platforms mistakenly you defended yourself using an open model but it was a chinese model but the the abilities of that model to defend you were also diminished by their guardrails is that a fair statement um yeah i mean the open model uh fortunately was kind of like uh flexible enough that we could use it to defend ourselves.

7:03So that's kind of like the kind of like efficient tools for defenders.

7:12Ed Ludlow:We're live on Bloomberg Television and Radio on Balance of Power. There'll be people in America and around the world listening to this and thinking, okay, very powerful AI models were able to escape a testing environment, access the internet, and then access mistakenly another company's platform. What was the net result? Did something bad happen as a result of this? Well, I think not as bad as it could have been. It was bad for us, for our team, right? I mean, our whole security team worked on this for quite a while. It created some challenges for us, obviously. But it could have been way worse, right?

7:56Like, for example, we're an AI platform, right? So we have kind of like good ways to defend ourselves. But a lot of other organizations, companies don't have kind of like the same defenses. Obviously, some domains are, you know, more at risk than others. But I think it could have been much worse. But we need to take this kind of like seriously as a wake-up call for us to kind of like work more on getting these systems secure. and giving more tools to all defenders and generally creating more transparency and more monitoring of these systems.

8:35Ed Ludlow:Has the US government taken this as a wake-up call? Which branches of government have reached out to you since the event was disclosed? I'm not going to talk to more the private conversations that we've had, but we obviously reported that to the relevant authorities and have had a bunch of conversations with different organizations, different congress members and people from governments. I mean, there are a couple of things that I think we need to do and get rights in the next few months. First, in my opinion, we have to make sure that cyber attacks, even when they're done by agents, stay a crime and illegal and make sure to enforce that.

9:26Otherwise, we're going to end up in a world where everyone is cyberattacking everyone with really no disincentive to it. Second, we need to create more transparency. Like, for example, why not doing mandatory disclosure when agent cyberattack is happening? And third, we have to kind of like give more tools to defenders. Like we mentioned, open models. These are kind of like some of the tools that cyber attack defenders need to make sure that they can defend themselves.

9:58Ed Ludlow:And to recap, you're saying that the illegality of an AI agent, a non-human AI agent carrying out a hack needs to be enforced and basically broader regulations, the pathway to that is legislative to your mind? You know, it's not really for me to say. Obviously, I'm not like a legal expert or policymaker. I think that's a topic that we need to think about. If you kind of like take care of like higher view, we already have some autonomous systems in our lives, right? Like a self-driving car, you know, and many others. We made sure there's kind of like liability, right? If you fall asleep at the wheel of your self-driving car, then you're responsible if you're hitting another car.

10:52In a similar way, we need to make sure that the legal framework for autonomous agents is clear and defined and useful for the field and the American society.

11:08Ed Ludlow:We're live on Bloomberg Television and Bloomberg Radio. This is Balance of Power and we're speaking to Hugging Face CEO Clem DeLong whose company disclosed on July 22nd that two powerful OpenAI models without authorization or mistakenly accessed or breached his company's systems as part of a cyber evaluation that OpenAI was doing. You know, Clem, timing is everything. Within days, some of the most important people in the world of technology came out with a letter backing America focusing on open models. I'm talking about Satya Nadella, Jensen Wang. More recently, AWS CEO Matt Garman has joined many others.

11:52Ed Ludlow:Do you think that the timing of that was a sort of catalyzed or a direct response to the July 22nd breach? Well, I think it was related. I mean, this cyber attack showed that, you know, you can create risks with models behind closed doors that are unreleased. So just working on preventing the models to be released is not the solution. And that's, you know, defenders need open models because, you know, to defend yourself against a cyber attack, you need to run it on your own infrastructure because it's usually on your private data. and you need the control that open models gives you that sometimes proprietary APIs don't give you because of guardrails, because of limitations, because of also how much it costs, right?

12:49So I think that was a perfect example and perfect validation for this notion of the world. Small companies, startups, researchers, but also large companies like needing open models wherever they come from.

13:08Ed Ludlow:I would just point out, Clem, that in the course of this conversation, we've talked a lot about it was OpenAI's models. You conducted a joint disclosure and then joint investigation with OpenAI. And of course, we've made every effort to invite OpenAI onto the network, onto the show to discuss their side of the story and the investigation. But that open-weighted letter in part was supposed to sort of outline the benefits of OpenModels. the economic benefit, the business consideration, that is very much hugging faces realm, right? Hosts models, big and small, open on its platform. The other thing was the concern that there would be over-regulation of open models by the US government.

13:53Ed Ludlow:Kind of bring that full circle for us and why you think that that concern is valid or not. Yeah, I mean, there are a lot of different topics in AI. And I think the role of a lot of people is to set the priorities. And I think what this later kind of like was outlining too is that probably one of the biggest risks in AI is concentration of power, right? Like it's concentration of powers, capabilities, wealth. in a few organizations when everyone else would be lagging behind and be left out in a way. And open models are kind of like a counterforce to that. They empower small companies, startups, organizations that are not necessarily kind of like frontier AI labs to build AI themselves, to own their intelligence, as some people said.

15:01And I think that's one topic that policymakers need to focus on a little bit more than they've been focusing on so far. So hopefully this letter puts the topic more prominently on their desk.

15:18Ed Ludlow:After you and OpenAI made the disclosure on July 22nd that this had happened, I spoke to lots of your peers in industry, and there was also the kind of acceptance that this wasn't some major scandal. Open AI has a lot of power. The closed models themselves are powerful. But generally speaking, industry said, we want to see the frontier labs doing this, doing these tests and evaluations of their capabilities, and then disclosing when something goes wrong. Just react to that, you know, in that sentiment towards the events of that week. I don't agree at all with that. We don't want any company in the U.S.

16:03running cyber attacks against other companies. This is a crime. This is illegal for a good reason. Because if you create a world where everyone is allowed to run cyber attacks against everyone, we're in for a lot of trouble. You have to remember that in our society most damages and most hurts isn't prevented because it is hard to do. If you think of it, I can go across the street and steal a grocery shop and it's not that hard to do. It is prevented because this is immoral and this is illegal. So I don't do it because of that. And it's the same thing for cyber attacks. If we end up in a world where these are normalized, cyber attacks are normalized, I think we're going to end up in a very dangerous world.

17:11The same way as I made the comparison with sales driving cars. right we don't want to end up in a world where you know bumping into another car is normalized is okay because you're driving a self-driving car like we want to make sure that our society stays kind of like healthy by uh you know keeping things that need to be must be

17:37Ed Ludlow:illegal illegal this brings us back to the central concern of the american people frankly right which is OpenAI did not instruct those two models to go out and hack Hugging Face. They instructed the two models to undergo the evaluation, right, to do the test. They were able to escape the sandboxed environment, gain internet access, and they went to Hugging Face's platforms, in part, I understand, because the models determined they could find information on Hugging Face's platform that would help them to pass that test, carry out that evaluation. But OpenAI didn't instruct them to hack Hugging Face.

18:16Ed Ludlow:Nevertheless, here we are. You know, that's the bit that people are worried about. Yeah. Yeah, I mean, you know, the same way if I'm on my Tesla and I have self-driving and I fall asleep at the wheel, you know, my intention is not to bump into another car. But, you know, you still have kind of like, I think, liability. I think they shared themselves that, you know, there were some kind of like mistakes that were done, that there was kind of like ways, you know, they could fix some of the bugs that led or some of the weaknesses in their systems that led the agentic system to be able to escape. And I think they're hopeful to actually fix this and make sure it doesn't happen again.

19:04I don't think they're going to try in the future to build a system that regularly runs cyber attacks against other companies. I think they're going to build systems that are stronger, better, to make sure these things don't happen in the future.

19:26Ed Ludlow:Hugging face here, Clem Delong, we just have about 60 seconds left in the conversation. You've talked about what needs to happen, but what's the net result of all of this? And what needs to happen next? Yeah, I think we need more transparency. I think it was a good wake up call that preventing releases is not enough. I think we need more transparency on these systems. And second, we need to kind of equip defenders better. like for example with open models that we used to defend ourselves. So we need more powerful open models for all defenders so that we're prepared for what's coming next. Hugging Face CEO Clem DeLong joining us live on Balance of Power.

20:13Ed Ludlow:Thank you very much. AI is entering its most consequential phase where scale, safety and sovereignty will determine who leads and who lags. Join Bloomberg Tech in London on November 2nd and 3rd as global leaders across business, finance and policy examine the defining trade-offs shaping the future of AI. Thank you to our presenting sponsor Salesforce and supporting sponsors IDA Ireland and Schneider Electric. Learn more at bloomberglive.com slash techlondon.

20:50The great wealth transfer includes$570 billion in classic cars.

20:54Ed Ludlow:I'm not in a position to be inheriting any classic cars. Do you? No, but my brother did inherit my non-classic car when I moved to New York. Ew. He still has not paid me for it. Coming for you, Joey. The Bloomberg This Weekend Podcast. Subscribe today on Apple, Spotify, or wherever you listen.