

4 Nov 2026 · 38m · 15 chapters

How companies can build value with open-weight/open-source AI models—covering economics (inference costs, licensing, cloud lock-in), founder decision-making (routing, “good enough” models, harnesses), geopolitics (US vs Chinese models, data-residency misconceptions), and security trade-offs (agentic hacks, “obliterated models,” and whether openness should be restricted).
Irene Suleiman, Chief Policy Officer and SVP of Strategy at Hugging Face. Background: joined Hugging Face as a researcher (Apr 2022); founded EvalEval (open AI evaluation community); previously at OpenAI on bias/social impact/public policy.
Value shifts from model training to deployment/inference; custom licenses (beyond MIT/Apache) can enable revenue sharing; open-weight model adoption is accelerating post–July incident; harness/tooling lacks standardized evaluations and is costly to test; open models face rising attack pressure (OpenCode agent hack targeting third-party API keys); openness may need better security standards rather than blanket access restrictions.
Revolut switching to open-weight models; Hugging Face defending during OpenAI “agent swarm” incident; OpenCode 48-hour agentic attack (mid-October); Chinese models like Qwen used in sovereign deployments (e.g., Rio de Janeiro/Japan); “obliterated models” (guardrails removed) used for red teaming; examples of open models for coding (Qwen 3.8 Max, DeepSeek/other named models) and Hugging Face bio model Carbon X.
Written by AI. May contain mistakes.
Discussion on the implications of open source AI models for businesses.
“Yeah, I'm so glad to talk to you again after we just recently had you at Disrupt a few weeks ago.”
Exploration of how companies can monetize open AI models amidst competition.
“Maybe the companies that are building the apps off these open-weight models.”
Examining how infrastructure affects the performance of different AI models.
“you know, work with AI tools on the other in order to build anything that's bespoke for enterprises or governments.”
Discussion on the need for evaluations of harnesses and tooling in AI.
“And, you know, it's not just about like, you know, there's a good enough model, but so much of it increasingly is the harness, right?”
Advice on GPU selection and usage for startups working with AI.
“There's a lot of work that can be done to make sure model evaluations are being done in the right way.”
Learn about optimizing coding tasks using the latest open-weight models available on Hugging Face.
“and for the choices that you're making for high-intensity coding tasks, the latest open-weight models, Kimi K3, DeepSea V4.1 Flash, and the Quen 3.8 Max are optimized for coding tasks.”
Examine how geopolitics influences the development and adoption of open-source AI models, especially Chinese vs. American models.
“because I want to talk to you about the open code attack.”
Explore the limitations and cultural evaluations of AI models from different countries, particularly regarding sensitive topics.
“Also, for context, GPT-OSS was released by OpenAI around August 2025.”
Discuss the necessity of resources and funding for open-source AI development, particularly in the U.S.
“It was probably too early for DeepSea, but they were testing some of the Chinese models, like asking about Tiananmen Square and then they wouldn't answer.”
Delve into the security threats facing open-source AI models and the impact of recent hacking incidents.
“You know, NVIDIA has been a big proponent of open models recently.”
Learn about the ongoing discussions regarding balancing security in open-source ecosystems while maintaining their openness.
“So it was around 48 hours of intense agentic attacks that were seeking third party API keys.”
Learn about obliterated models and the trade-offs involved in their use.
“So for our hugging face, we have these really interesting evaluation data sets.”
Explore the challenges of maintaining openness in AI while ensuring security.
“I've also come around on obliterated models a little bit after talking to cyber people because, you know, I wrote about this.”
Discuss potential policies regarding access control to AI models.
“But the urgency from our perspective and from where I'm sitting at Hugging Face is that we just need to equip defenders yesterday.”
Get insights on how small startups can protect themselves from threats.
“The general guidance that we need is around what is the appropriate level of access control.”

Get the gist, explore an idea or ask about something said.
On Equity, what are the key points of this episode?
Automatic transcript. May contain errors.
0:00Rebecca Bellan: Hello and welcome back to Equity TechCrunch's flagship podcast about the business of startups. I'm Rebecca Bellan and this is the episode where we bring on industry experts to help us explore a trend in the tech world and dive deep. Open source models are becoming more mainstream as enterprises, governments and startups alike rely on the promise of lower costs, more control and more flexibility. But as increasingly capable models become more freely available, how do companies actually capture value? And with Chinese models gaining ground, can American players keep up? Today, we're joined by Irene Suleiman, Chief Policy Officer and SVP of Strategy at Hugging Face.
0:34Rebecca Bellan: Irene has spent her career researching how AI models are released, evaluated, and potentially misused. We're going to talk about what startups need to understand before building on open models, from the economics and global competition to the cybersecurity threats that can put openness itself at risk.
0:54Irene, welcome to the show. Thanks, Rebecca. I'm so glad to be here.
0:58Rebecca Bellan: Yeah, I'm so glad to talk to you again after we just recently had you at Disrupt a few weeks ago. You were on stage talking to us about evals. And I'm really excited to talk to you because you can just riff, as we both know from our many conversations we've had, you can just riff on so many things when it comes to open source. So just to give our listeners a bit of background, you joined, and stop me if I get anything wrong. You joined Hugging Face as a researcher in April 2022. You became the policy director, head of global policy, chief policy officer, and SVP of strategy in February 2025. Quite a litany of titles there.
1:31Rebecca Bellan: So in that role, you're running strategy, geopolitics, research, and policy. You founded EvalEval, an open source research community focused on AI evaluations. And before that, you were at OpenAI working on bias, social impact, public policy. So stop me. Did I get anything wrong? You know me better than I know myself, Rebecca. Oh my gosh, yes. Okay. So one thing I wanted to kick off the conversation with, when we talk about open models, you know, we had Clem DeLong, your CEO, on in July. He talked about companies moving away from expensive AI services and toward open models that they can run themselves as a way to cut costs.
2:07Rebecca Bellan: We've also got investors pouring billions into training, you know, closed models. But if these models themselves can be commoditized and more freely available, who's making money? How are we making money here? Where are the threats and where are the opportunities? Well, first on the switches, the conversation you had with Clem, it's funny how prescient you continuously been. I realized it was before the incident that we endured in July. And when we were seeing already a trend of movement towards open models, it's only expedited since then. So as of, I believe, today, Europe time, I'm not sure if you saw Revolut, the kind of neo-bank for our American listeners that was founded in London and operates largely in Europe, announced that they are switching to open-weight models and using only Clotten Codex for exceptions.
3:02We are seeing this overall trend, particularly since July, of more established and startup companies announcing their shift to open weights with the explicit purpose of cutting cost. To your question about what does this mean for the model developers, there are a few ways that people earn money. And it is harder for the developers if everything's hosted on-prem, but preferable from an enterprise perspective, both for the cost, but also for the data privacy and data residency, especially if you want that in a certain jurisdiction. But particularly for making money, a lot of that is going to be inference costs.
3:45It could be a sort of cloud lock-in if folks are using inference via, for example, Alibaba Cloud, which is the parent of Quen, Alibaba's excellent flagship model. Or it's a sort of revenue sharing system, which is the kind of boring area of licenses that we can get into if our listeners don't get too snoozy.
4:08Rebecca Bellan: Yeah. Okay. So revenue, like for the open model providers, like revenue sharing built into the model license? Yes. I know this is something that you've talked about, kind of wanting to spearhead it at Hugging Face. Yes. So licenses are generally drafted in legalese, which is why they don't get a lot of active readers. And there are a couple standard ones that you see most commonly with open weights, MIT and Apache 2.0. They're quite permissive, as opposed to the latest iteration of flagship open weight models that are using custom that require some level of revenue sharing if you make some threshold of revenue.
4:49Rebecca Bellan: So we're talking revenue sharing. There's potential for cloud providers to get a piece of this pie. Maybe the companies that are building the apps off these open-weight models. Are we, like, could we possibly be living in a world where open models are creating enormous value for the AI industry while the companies actually developing them are struggling to capture much of it? It's hard to say without seeing the explicit financials and not everything is a public company. So it'll be quite an interesting next few months. The level of value from the deployers and from the developers, I think also depends on the makeup of the business organization.
5:28So for Alibaba Cloud, they're just such an enormous cloud company. It could be considered a sort of feature to bring folks in to Alibaba Cloud and Quint is used really worldwide. So it's both their smaller model, the 27B, which is Apache 2.0 and rentable on a laptop, and their larger model, which is frontier level. The latest is Quint 3.8 Max, which is open weight, can be used on-prem or it's easier to use inference. So a lot of folks in those kinds of scenarios do make a substantial amount, but we're not able to view those exact metrics.
6:07Rebecca Bellan: Right. So, OK, so some of the bigger companies, specifically we're talking China here, like Alibaba, they're offering open source models, but they're also offering the cloud and the compute. Right. So they're able to offer this as kind of a package deal. Whereas if we look at maybe the American ecosystem, that's not necessarily the case. Right. A lot of the open model providers are just open model providers. they don't have compute that they can I mean maybe they have deals with neoclouds that they're trying to like sell a package deal with but they're not the ones getting revenue from the compute so I kind of I kind of worry for them a little bit more in this in this case it's a tricky decision on if you're releasing an open model that doesn't quite reach or beat frontier level a lot of folks love using frontier I mean they're great products they're easy then how do you get those kinds of users interested in deploying.
6:57And this is where these customized licenses come in for the US model providers on the open weight side. Some of the more visible ones who have done a great job are, I mean, Meta historically has been an open weight champion. And RC, Reflection have been releasing with permissive licenses, but they also do enterprise contracts. So you'll see with reflection, they did a enterprise agreement with South Korea, and have a partnership to build a sovereign AI cloud. So there are other ways to do these kinds of custom solutions.
7:34Rebecca Bellan: Okay, so so almost like a white glove service will like, you know, you want to build your own sovereign or enterprise solution, and you want to use open source model school, we'll build it for you kind of thing. Yeah, it's for how much interest there is in AI, there are very few people that really know how to customize this relatively. Yeah. I mean, we're seeing that with, you know, the whole, I mean, it's kind of side note, but with the whole FDE trend that we're seeing, everyone's, you know, this is becoming the hottest job in AI as a forward deployed engineer, because there's so few people that actually, you know, know enough about business on the one side, and then how to, you know, work with AI tools on the other in order to build anything that's bespoke for enterprises or governments.
8:10Exactly.
8:10Rebecca Bellan: You know, a lot of our listeners are founders. So why should a founder care? Like what do they need to check before trying to build a product on an open model? Before building, I mean, this is a question of what are their preferences for the task. And this is where we get into routing. We don't have great model routers to pick and choose the right model for the task. Being able to create a stack where you have a potentially lower cost, but effective model for certain tasks, let's just say like I'm thinking a lot about cybersecurity. So some level of cyber monitoring for giving summaries for low stakes infrastructure, you don't probably need something that is the most premier frontier model, but you will need to understand what is the capability of the model for the tasks that I want?
9:05Where do I put it throughout my infrastructure? And do I have the capacity on hand to run this model or do I need to rely on inference, whether that be an open way, which could be cheaper than a frontier or a frontier model itself?
9:21Rebecca Bellan: Yeah, OK, that's interesting. Like this is something I keep hearing that not every company needs the most powerful model for every task. Right. So what kinds of work are open models? You know, some of the maybe smaller ones, are they increasingly good enough to handle? And then the second question for that is where do the more advanced models still have a meaningful edge? That word is so important, that phrase, good enough. I think this really comes down to a tradeoff for how much you also want to spend on token maxing. I shudder to think about token cost without open weight models, and this is not a reflection of the frontier labs.
9:58It's just incentives in the marketplace. for what folks want to use open models for, what they're capable for, they can do quite a lot, if not the same agentic tasks. I rely entirely on almost exclusively GLM 5.3 these days, sometimes it can be K3 using a Hermes agent for almost everything. And I think this is the trade off for it might need extra prompting. It doesn't give me refusals in the way that a frontier model would. But I think this really depends where I would say I use it for everything.
10:34Rebecca Bellan: Yeah. And, you know, it's not just about like, you know, there's a good enough model, but so much of it increasingly is the harness, right? It's the structure, the infrastructure that's built around the model itself. So you've mentioned to me before that there are like less capable models with a better harness can actually outperform a frontier model. So how do you think that this is affecting overall performance. Spot on. On these more specialized startups or tasks, one thing that we had discussed before, thinking about cybersecurity a lot, there are harnesses that can zoom in to specific cyber defense tasks and leverage models.
11:13And it will still use a generally powerful model that can address vulnerability identities, for example. these kinds of harnesses we're just so many are are popping up more and more are custom made this can be a revenue stream in and of itself of folks who are able to design
11:34Rebecca Bellan: harnesses that our economy needs right now yeah and we're seeing more and more of that and it's like you know being a journalist my inbox is filled with startups trying to like ride the lightest wave so it was you know at first it was just like app companies built on ai and then it It was agent companies. Those are still pouring in. And now it's like harness companies. I guess agent is a harness. Like the harness is the agent or, you know, they're related in a lot of ways. You know, at Disrupt, you and I talked a lot about model evaluations. There's a lot of work that can be done to make sure model evaluations are being done in the right way.
12:07Rebecca Bellan: But there's not one of the things that appears to be missing is that there's no good evaluation for harnesses and tooling. Yeah. What do you think about that? Yes. Yeah. I love generally talking with you, but especially in evaluations, it's a nerdy area that has gotten attention for how important it is for deployment, but it is an area of science. We just have a lot more science around natural language processing, the basis field for LLMs. We don't have a lot of background for harnesses as they're deployed now is relatively novel. We don't have explicit evaluation suites for harnesses in ways that are standardized pre-deployment of harnesses.
12:48And the way that I think a big barrier for this is just how darn costly they are. Agentic workloads, if anybody who's used them, just cost a lot of money for all of these tools. And so if you're running a valuation, it'll easily go up into the millions.
13:02Rebecca Bellan: Yeah, yeah. So how can a startup manage this, right? How can they know whether a cheaper model and its surrounding tools are good enough for its product or cheap enough for its product if there's no reliable way to measure, you know, the spend and ROI and just like what you're getting? It's going to be a tradeoff for the near term and long term. In the near term, it might be easier just to run inference, to call a frontier lab. If you have like high intensity tasks, if you don't have a lot of AI experts on hand, totally understandable to use what's easy. And a lot of the time that's going to be close.
13:35if you want something that you want to scale up a bit you know you'll use it longer term and you can this is why a lot of folks are buying gpus and not renting you can set it up on-prem you can optimize that model over time without renting with the cost of compute going up your long-term costs will likely plateau or decrease desirable depending on the the business that you're running and for the choices that you're making for high-intensity coding tasks, the latest open-weight models, Kimi K3, DeepSea V4.1 Flash, and the Quen 3.8 Max are optimized for coding tasks. So you can look at these types of benchmarks.
14:21You can look at the types of coding tasks you're interested in. Or if, again, you have these skills, you can fine-tune that model. There are a lot on Hugging Face, a lot of data sets for fine tuning to optimize for certain tasks.
14:35Rebecca Bellan: You mentioned, just want to quickly dig into this. You mentioned that, you know, a lot of people are buying their own GPUs these days, right? Yeah. I'm kind of curious about that. Like, do you have any tips for, because, you know, where all we hear about is GPU scarcity. And it's, you know, like, let's say I'm a startup founder. I need to mess around. I need to do a lot of experimentation. I'm using open models. I want to have my own, you know, GPU situation going on. Am I buying like how many GPUs would I need to buy? And is it okay to, you know, not get the latest, greatest NVIDIA chip? Because you probably can't anyway.
15:06I mean, yeah, the scarcity is not just to your point, the availability, but the availability of what and a lot of there is a hunger for the latest NVIDIA chip because a lot of the newest models run easiest on them. In terms of what you can get your hands on, there are just a lot more libraries and resources, including on Hugging Face. And generally, this is the beauty of the open source ecosystem that will help you adapt your model to that chip. So we see these previous iterations of GPUs, so like the NVIDIA Blackwell, a lot of folks use for frontier models. And we're going to get a little bit technical here.
15:47It's also a question of the latency for what you're deploying. And this is the level of speed of response. For our incident over the summer, we deployed a NVIDIA quantized. So this is NVFP4. It's a 4-bit quantized model. It's not going to have that same level of sort of horsepower that a fully high speed kind of inference will give you. But it'll get the job done and it'll cost significantly less than what we estimate the agent swarm cost.
16:20Rebecca Bellan: And for the listener, if you don't know what Irene is referring to with the incident, the incident is obviously when OpenAI's agent swarm accidentally kind of hacked into Hugging Face. And Hugging Face was able to defend itself using some open models rather than the closed models, which I think threw up some guardrails, which is something that we're – we'll actually talk about that again. because I want to talk to you about the open code attack. And then another disclaimer, NVIDIA has acquired or is in the process of acquiring Hugging Face as well, so just note to that. Another thing about NVIDIA, and I'm going to come to it in a roundabout way, I want to talk a little bit about China and competition for open model dominance.
17:01Rebecca Bellan: The best open models, you've listed several of them. They're all Chinese at this point. What happens if Chinese models become the default for AI startups around the world. And actually not just AI startups, like sovereign AI for governments. And we're seeing Rio de Janeiro and Japan's sovereign models are built on Quinn. So what happens? Is that a problem? It's always a dance when we're talking about open models and with the geopolitics. The geopolitics have become much more central even in the last few months. It's a question of problem for whom. So people love open models no matter where they come from.
17:40I've had to scale up very quickly with our head of APAC. It's really just me and Adina working with Open Frontier Labs, these Chinese labs who are so talented, who work really hard. And it's incredible to feel the ethos of belief and open weights. It would be even more exceptional to have a thriving global ecosystem and to have Open Frontier U.S. models. We are working closely with the Muse team, with Reflection, with RC. It's really hard to train an open frontier model, but there's a reason that governments are preferring models developed in the US, for example. There's a reason that GPT-OSS still has a large amount of downloads on Hugging Face.
18:24There is a hunger for US models. How do we stoke that, I think, is a big question around the narrative. There's been a demonization of open source, of these models being more dangerous, which if you look at things for the explanations and disclaimers, Rebecca, the last few months of cyber attacks, the most prominent ones have been frontier models, have been close weight models. And we're just balancing with what do we need to make sure that folks get access and have the tools in hand. And a lot of the time, it's open weight.
18:58Rebecca Bellan: You said that there's a hunger for GPT-OSS. There's a hunger for American open models. And a lot of people, a lot of companies, governments want to build on American models rather than Chinese. What is the main reason? Is it just the geopolitics of it all? Also, for context, GPT-OSS was released by OpenAI around August 2025. That's my bad for not explaining that. Sometimes I'm like, we're all nerds here, right? We all know what we're talking about, right? But there have just been so many measured time and latest frontier generation of models. I can't believe that was 2025. I know. August 2025 was GB2SS.
19:37All right, OpenAI, when's the next open model? Come on now. Well, we're also itching over here. Yeah. In AI time, that's a really long time for a model to be out there. There are much better open weight models since. So the fact that folks are still hungering for this shows us that sometimes it is a hesitancy to deploy Chinese models because of the geopolitics or some misunderstanding. I think some of this misunderstanding is the projection that folks are using APIs for Chinese models or models developed in China, and then their data would be sent to mainland. We are very purposeful about none of that data being sent to mainland data in France.
20:17The partnerships that you see with Chinese-based labs and AWS, for example, show you that there are ways to deploy these models without having your own GPUs but based in the U.S. Some of the reasons that folks are using it outside, using U.S. and not Chinese-based models, developed models, could be around this misperception, avoiding scrutiny. But it could also just be a preference for some of the values if you don't have those kinds of technical depths to be able to fine tune. There are some differences and there's some thoughts on what refusals you get from models developed in China, for example.
21:03Rebecca Bellan: Yeah, I mean, the basic one that I know about, and I think this is probably more of a function of their API than actually the model. But I'm not sure. Maybe you tell me like, you know, earlier, early on in the whole LLM discussion, you know, everyone was testing DeepSea or not even DeepSea at this point, probably. It was probably too early for DeepSea, but they were testing some of the Chinese models, like asking about Tiananmen Square and then they wouldn't answer. Right. So I'm not sure. Is that kind of what you mean? Or is there other other refusals? Yeah, that's the classic one. And the part of the reason that you've not seen in-depth analyses is that cultural evaluations are so fraught.
21:34we've done a few ourselves and to be able to do it justice would be PhD level of work. But just from like, this is where we've moved somewhat to vibes versus emails to a combination of both is there's these kind of, I think it's like Claude speak. There's different ways that different models speak. The kind of cultures that you're looking for to be emulated, you can, fine-tune and you can totally adapt and make that your own. But that is relatively intensive. Take some GPUs, take some talent to make work. Yeah. Okay.
22:10Rebecca Bellan: So GBTOSS, are there other open models from the US that are being downloaded or used or really popular on Hugging Face at the moment? We are seeing so much hunger for not just the Neuatron, but the general family coming out of NVIDIA and OpenWeight side. So the Nematron family is the more lit on the language model side. What is generally slept on in the open source community are bio models and science and robotics. You guys just came out with a bio model hugging face. We did. You came out with your own. That's really cool. Carbon X. And it's quite a small model. You can run it on one GPU. There you go.
22:50Yeah. And if you do run it partially because of the one GPU, It is just orders of magnitude less costly than comparable models. And that goes a really long way for everyday biologists.
23:01Rebecca Bellan: Yeah, I mean, you brought it back to NVIDIA for me. I promised I was going to bring it back to NVIDIA, but you did it for me with Mematron. So I guess, you know, one of the things we kind of talked about earlier is like who has the resources to back up a lot of the, you know, if you're going to be at the frontier of open models, like you need money. Like this is expensive to train these models, right? And of course, there's, you know, distillation helps, which a lot of people give that a dirty word, but it is a very common practice in AI training. But like, does America need an NVIDIA-like behemoth to actually solve this and not only compete with China on open source, but to compete with OpenAI and Anthropic here in the States?
23:40I mean, this is a question of what does the overall ecosystem need? And this is, we are undergoing the regulatory process of joining NVIDIA, which I'm very excited about. and as this is a thought process that I've had independently, but it's appropriate to give the disclaimer, I think we would thrive most in both open frontier and open local. So this is where you see on Hugging Face, the sweet spots of what people engage with, with both likes and with downloads, we look at many metrics are around like 20 to 40 billion parameters. And with especially quantized or really souped up, latest MacBook, DGX, Spark, AMD, Halo, you can run these locally, or the kind of one to three trillion, what we call open frontier.
24:25For an ecosystem, we need the smaller open weights that are very permissive, but we probably need the resourcing that something like an NVIDIA has to create a frontier American model that folks can explore do more evaluations on, deploy locally, be a good alternative to our close frontier.
24:52Rebecca Bellan: You know, NVIDIA has been a big proponent of open models recently. I mean, my personal theory is because, you know, the more people are using open models, the more GPUs get bought, like, just individually, or people need to use NeoClouds. And, you know, it's coming at a time when a lot of the labs are, if not actively doing it, threatening to build their own AI chips. So it's an interesting space. But definitely Jensen Wang, the CEO of NVIDIA, his embrace of open source has been parroted by a lot of other CEOs. We're coming to a point where open models, the use of them is not as stigmatized anymore, right?
25:31Rebecca Bellan: Like everyone's kind of admitting to it. It's becoming more mainstream. But at the same time, it's still facing threats, right? And so you and I spoke a little bit about the open code attack. So I'm not sure I hadn't actually heard of it by the time we talked because our algorithms are simply different. And I feel bad about this because TechCrunch did cover it. But, you know, it was around disrupt time, so it was busy for me. But anyway, you know, mid-October, OpenCode was the target of a hack. So OpenCode is an AI coding agent as part of the open model ecosystem. This was roughly a 48-hour agentic attack targeting users' third-party API keys.
26:08Rebecca Bellan: It compromised some accounts. There was some financial damage. So, you know, we've seen attacks now targeting hugging face and open code. Granted, they were slightly different in motivation. One was human motivated, the open code versus agent swarm motivated. But still, these platforms are, you know, potentially becoming bigger targets. Do you worry that the security pressures could force more closeness, closeness, close, closeness in the open model ecosystem? This is why we need journalism, Rebecca. We all exist in the algorithmic prisons of our own making, and you are by far not the only person in our field to not have heard of this hack.
26:47Especially it happened right before the PHP vulnerability exposure and just Black October took another level. The open code attack, we don't yet have public information on who the attacker was. We have some ideas of the motivation. So it was around 48 hours of intense agentic attacks that were seeking third party API keys. This is meaningfully different from the hugging face incident over the summer that we endured where kudos to our infrastructure team were able to recover. And it seems that open code was able to recover, but we did not have compromise of user data, of financial data. And this is there was financial damage here.
27:31the things that I worry about are what makes an open community targeted is also what makes it special we are necessarily permissive we want the average hobbyist in their basement who isn't it doesn't have an institutional affiliation to be able to contribute I mean they are part of who made the internet exist I mean this is like examples of Wikipedia just there's how few people are keeping Wikipedia alive that's the beauty of open contribution and we also have So this is a permissiveness and the wealth of information and data we have. So for our hugging face, we have these really interesting evaluation data sets.
28:09Open code has these juicy third party API keys. The combination of that makes us prime targets.
28:16Rebecca Bellan: Yeah, I mean, it's it's there's a there's a defense gap there, right? So where do you draw the line between securing the open ecosystem and undermining the openness that makes it valuable? This is what we're noodling on. What is that trade-off? What kind of conversations are you guys having at Hugging Face? Like anything that you can share? Like what are you noodling on? I mean, the conversations have gotten to a point where a lot of folks are – we are responding to our community. We are just seeing so much interest in obliterated models. This is a can of worms, really dicey topic. And they do instill fear in a lot of folks.
28:55Rebecca Bellan: Can you just briefly explain to our listeners what an obliterated model is? Yeah, it sounds like such a funny word. Obliterated models are models that have had all of their guardrails removed. And already some of these open frontier models are pretty permissive in their refusals and guardrails. But you can pretty easily just kind of fine tune that out and take those off and re-upload them on Hubbing Face. This is not a violation of our terms of service. And part of why we are keeping them up is we have so much feedback from cyber defenders and researchers on how this is their only option for either research or defense.
29:33Rebecca Bellan: Yes. Yeah. I've also come around on obliterated models a little bit after talking to cyber people because, you know, I wrote about this. There's a company called Obliteration.ai, I think it's what they're called. And they basically just like host or will provide you with the ability to obliterate any model. So you just there's no guardrails at all. And you can just imagine how horrible this is, right? Like you don't want to make it easier for attackers. But at the same time, this makes it easier for defenders. And I guess, you know, the argument is the attackers are going to obliterate them. They're going to do this anyway.
30:02Rebecca Bellan: It's important that the defenders have, you know, know how they're going to attack them, right, essentially. And are able to – it's really good for red teaming as well. You know, really helpful for, you know, when you're trying to do the most insane version of a hack that you're trying to test your systems on to have an obliterated model. So, yeah, it's tough. But then if open model infrastructure becomes more costly and risky to maintain, who, I wonder, would pay for the security? Would that potentially give large, well-funded platforms an advantage over the smaller open source projects? I mean, there's going to be just an advantage in the level of product support that you get from a closed-weight lab.
30:48Most of these, outside of the really well-resourced like the Alibabas, we're still 200 people while we're undergoing this acquisition. So we're not going to be able to provide that level of hand-holding. But for what folks need for cyber defense at a time where there's a lot of appropriate... I'm quite nervous about the state of the internet. we need to have some level of this is a trade-off we need to have access to the folks who know what they're doing who are getting refusals from the most frontier models there's a dichotomy unfortunately between cyber security and AI safety folks and both have good intentions but a lot of security folks love open source because of the privacy because of those on-prem considerations because of what they can do with it and really have this ideological preference for open weeks in a way that is creating a new kind of open versus closed conversation.
31:50But the urgency from our perspective and from where I'm sitting at Hugging Face is that we just need to equip defenders yesterday. And if that means giving folks a more permissive model, it seems that's the balance that we'll have to make.
32:05Rebecca Bellan: Is there anything that could convince you otherwise? Like, is there any type of attack that might happen that would make you think, actually, like, I don't want people like attackers to have access to, you know, powerful, obliterated open models? I mean, this is the visibility of who's doing the attacks and who's doing the defense. I wish that we had better ways. This is what we've asked for policy wise is better incident reporting and standards. We're so glad that we were able to be as explicit as possible, but it was quite difficult to figure out how to report it and to whom. So as folks both on the frontier side, but also who are doing defense, who are victims of attacks, are undergoing this, I would ask that they be explicit as possible in sharing openly what they're undergoing, if they have an understanding of what model is being used.
32:55To your explicit question of what would change my mind, I think it is a clear data that the risks of open weights being leveraged in attacks outweighs the possibility for defense. And a lot of this is compute constrained. I don't think there is enough compute available to a non-nation state actor to do the level of large scale at this point, which could change in the near future. But in our level of urgency, the need for defense, for even if you've seen these European grocery stores, it might not be the highest priority for critical infrastructure, but for businesses to defend hundreds of thousands, if not millions in damages seems important.
33:39Rebecca Bellan: Would you support some kind of a policy that restricted, I mean, if not restricting access to models, restricting access to compute? I mean, I think this is the kind of we're playing catch up here. Token efficiency and compute efficiency have made leaps and bounds in the last few years, particularly since DeepSeek R1. I don't think that's the right metric. The general guidance that we need is around what is the appropriate level of access control. Because I fear that we're going to start putting some level of access control on open-weight models, and then nobody has access to anything. Yeah. Okay.
34:13Rebecca Bellan: You know, a lot of startups are building on open models, open source infra. We touched on this a bit last week when we had Wiz on. a decent amount of them don't have dedicated security teams. I just want to get your quick take on this before we close out. How can small startups protect themselves now? What are they underestimating perhaps about the current threat landscape? It's understandable to not already be prepared with security teams. Even having some level of familiarity with open resources, my team's working on putting together a plan in the next few weeks. I wish we had it out sooner, but in the meantime, there are really great cybersecurity startups like Sumgraf that have openly available documentation.
34:52So scale up on the resources available to you. This is the beauty of an open internet. And it's okay to lean on what's easy for you, but it's on us and the open community to lower the barrier for folks to use cyber defense tools that they can rely on.
35:10Rebecca Bellan: Okay, great. Well, hey, I think with that, we are out of time. Where can our listeners find you connect with your work online if they want to, you know, drop in, say hi, slide in your DMs? I'm not I'm not as good with my DMs as I maybe was pre chat GPT. You can you can find us on make a hugging face account. We're all over having face. I'm most probably available by email ireenahf.co and keep following us on all your favorite social media platforms at Hugging Face. Okay, thank you again, Irene, for joining us on the show. It was great to have you. To our listeners, you can find me on Twitter and LinkedIn and you can find Equity on threads and Twitter at EquityPod.
35:53Rebecca Bellan: Talk to you next time. Equity is hosted by TechCrunch Senior Reporters and produced by Teresa Locansolo with editing by Cal. Subscribe on YouTube or wherever you get your podcasts and find out what's next at techcrunch.com slash events. Thanks so much for listening and we'll talk to you next time.
Open source models are becoming more mainstream as enterprises, governments and startups alike rely on the promise of lower costs, more control, and more flexibility. But as increasingly capable models become more freely available, how do companies capture value? And with Chinese open models gaining ground, can American players keep up?
On this episode of TechCrunch's Equity podcast, Rebecca Bellan caught up with Irene Solaiman, Chief Policy Officer and SVP of Strategy at Hugging Face, about what startups need to know before building on open models, from choosing the right tools and infrastructure to navigating the trade-offs shaping the future of open AI.
Listen to the full episode to hear about:
Why startups, enterprises, and governments are switching to open AI models to cut costs, and when it makes less sense to use the most powerful models.
Who makes money when the models are free to use, and whether the companies building them can capture the value they create.
Why the U.S. may need deep-pocketed players like Nvidia to build an open frontier model that can compete with China’s, and what’s at stake if Chinese models become the global default.
How recent attacks on open AI platforms are raising a difficult question: how do you keep an open ecosystem secure without closing it off?
Subscribe to Equity on YouTube, Apple Podcasts, Overcast, Spotify and all the casts. You also can follow Equity on X and Threads, at @EquityPod.
Learn more about your ad choices. Visit megaphone.fm/adchoices