Shifting From Renting AI to Owning It

0:00So Misha, today we are in D.C. and we're here to talk about all things AI. Why is it so important that AI labs have a presence here? Yeah, well, you can tell we're not NSF because I'm not wearing a hoodie. I'm wearing the D.C. suit. Yes. The presence in D.C., especially as it pertains to, you know, AI companies, any company building at the foundation level or application level, is very important because this is, you know, arguably the most impactful technology that's been introduced into the world since something like electricity. and part of, you know, when electricity was introduced, regulating it, ensuring that people understood, you know, electricity had both positives and negatives to it.

0:48It was actually at the time, like there were safety concerns around it and it was important that the technology and policy was co-developed together. Now, the technologies for AI is being built primarily outside of DC. And so it's important for companies in this field to have a presence here, to be able to educate policymakers, various people on the Hill, and others in terms of how this technology works so they can make the best decisions in terms of regulating it. So there's been a lot of discussion around the open weights fight. And I think for a lot of people, there's been a lot of headlines, especially this summer around it.

1:23You are so close to this work. Can you give us just a status quo update on the latest in this space? What we're seeing is that while open weights were not particularly topical, let's say, even a year ago, and it's because the adoption curve in AI has been, it's a technology that's been adopted so quickly that the timelines are compressed. And maybe one way about thinking about why open weights have become topical at this point in time is thinking about closed and open models in analogy to real estate where you have a rental market and an ownership market. So when you're young, you typically rent.

2:04As you build a family, have financial security and so forth, you move on to owning a house. And there's a market for both rental and ownership. And in the intelligence space, something similar has happened where the initial diffusion has been rental, which is effectively renting closed models. And as companies have adopted more and more artificial intelligence, they're looking to move into ownership. And because the time cycles are so compressed, we've seen this happen very quickly, where I believe six months ago, 70 % of token consumption, if you went on Open Router or Versal Gateway or these other kind of distributors of intelligence, it was 70 % closed and 30 % open models, tokens being consumed.

2:49And now it's the opposite. Now the market has grown. So, right, the pie has gotten bigger. So there's demand for both. But the market has matured and is now getting more into ownership of the asset. And that's where you're seeing the pickup for open models. That's such an interesting inflection point. I know you've stated before that China is really at the forefront of the current open models. Help us understand why that is and why we have this dynamic. I think this was a very interesting geopolitical dynamic to see play out over the last few years. At the start of the artificial intelligence revolution that's been happening over the last few years, both closed and open actually was led in America.

3:36So, you know, the big closed labs were producing the first models that set the frontier of intelligence. And actually up until maybe two years ago, even the best open models were coming from the States. But something changed two years ago and particularly when DeepSeq came out. And it's very interesting to see how this technology kind of has evolved. I think in the States, the initial business models came from monetizing the model, which is why closed model companies grew very quickly. In China, almost all the labs are actually producing open models. And part of that does seem to be motivated by the state's desire to have its technology used abroad.

4:26And because China is also in some sense a captive market that cannot use the closed models that are developed abroad as freely, the open model providers actually serve intelligence into that market as well. So I think that the development here actually came out of more geopolitical competition reasons than innate technological reasons. You bring up this point that out of China, there is this need to develop open because they don't have access to some of the closed models in the U.S. Flipping over to the U.S., why is it so important to have this optionality to have downloadable models? I would kind of talk about, you know, a couple of different customer segments because that's ultimately who gets affected.

5:11The first place where you've seen dramatic adoption of open models, where are these 70 % tokens coming from? Well, it's among developers and startups that grew their businesses on AI. So the application, you know, startups like Cursor or Cognition or Harvey, like these AI natives, Abridge, that initially started their businesses on closed models. But as their business is ramped up, reining in kind of control of the intelligence over multiple factors, over cost control by fine-tuning those models on their data and ensuring that they have the best cost for the capabilities that their products have.

5:48Compute security, right? They can get compute and run the models, and they know that that compute will always be available because it's their compute. Data security, right? Ensuring that their data doesn't leave and is sort of sovereign to them. There are a number of reasons why, you know, these startups have gone very big. they start adopting open models. And so without an open model ecosystem, without capable open models, the most promising startups cannot really become independent because they'll always rely on the intelligence provider that they're building on. And especially as that intelligence provider starts competing with them because ultimately it's a competitive market.

6:24So that's where we've seen the adoption start. And now the same exact story is starting to play out in enterprise, public sector. You can kind of think about any business or entity that is consuming intelligence at scale starts bumping into the limitations of renting intelligence rather than owning it for themselves. I think I want to end on questions around safety because that has seemed to be the topic of the last few weeks, notably the OpenAI hugging face. In the last few days, OpenAI has shared a couple more incidents. What is going through your mind as someone who's spent so much time in this space and someone who's building some of these frontier models?

7:02One of the things that's great, not just about open models, but more closed models, more kind of models of any kind, is that you get kind of a multipolar world with multiple perspectives. There has been a certain perspective around, you know, what is the safest way to build intelligence? And I think that, you know, open models introduce actually a different perspective that, you know, when we actually look at how a field like, let's say cybersecurity works, cybersecurity, strong encryption, start off closed. Those things were closed and there were arguments for you can't open them up because hackers will use them and, you know, like wreak havoc.

7:43And what happened was the exact opposite. Strong encryption protocols became open in the early 90s. And the whole field of cybersecurity relies on openness of this technology, right? And that there's a whole field of penetration testing that relies on various strong encryption protocols, software being open and tested for vulnerabilities. And the same, you know, what that implies is that the best safety mechanism, the best defense is a distributed defense where you have many people, many contributors looking at a problem. And there's a saying in open source called Linus's Law that with enough eyeballs, all bugs become shallow.

8:21And that's why open source software broadly is so useful. That's why Linux is used actually much more widely in servers than closed operating systems. And I believe the same analogy holds for AI. These systems are very complex. They have a lot of bugs in the sense that they have a lot of behaviors that are hard to predict. it's hard to believe that, just to give you a sense, that in one of these big reinforcement learning runs that a big lab might do, there might be a billion agents dispatched through that run. And so do we believe that a group of 50 people in that lab can monitor what those billion agents are doing?

9:01It's hard to believe. But if you have a whole ecosystem of people looking in, testing these things for vulnerabilities, finding them and patching them, then that seems like a lot safer, right? A safer way to deploy this technology. So we actually believe that open source enables a much more safer deployment of intelligence because it brings a much larger community of people in to find bugs and patch them. Yeah, which is almost ironic because it feels like a lot of the argument against open source today is the perception that bad actors can do whatever they want with open source. But in your view, you're saying, you know, based off of this, and this is law framing, that having more eyes actually enables a safer ecosystem?

9:43I think the bad actor argument is a very theoretical argument. But then we kind of look at what actually happens in practice. And the counter argument to it is that, you know, when you have a small set of people working on the most impactful technology in the world, they can't foresee all the consequences of that technology, even with their best intentions. And an example of this is this infamous incident that happened between kind of open AI hacking, you know, hugging face servers. That was an incident that, you know, a model behaved in a way that researchers internally, you know, did not predict, right?

10:19Because obviously they would not want their model hacking, you know, someone. And hugging face in that incident actually tried to protect itself by using closed models, but because of their safeguards, they were not allowed to use them for cyber defense. So the safeguards prevented those models from being helpful in that instance. And so Hugging Face turned to an open model and used an open model called GLM to defend themselves and remediate. And so while there's an argument that, a theoretical argument that bad actors can use open models for a number of things, in practice, we've actually seen the opposite, that when a model is closed, it just has unforeseen consequences.

10:56And it's actually the open models that came in to help remediate the incident. It's been so helpful to just pick your brain on this current moment that we're in. There's just a lot of noise in the broader open weight fight. And it's just important to understand and so critical to understand the role that open weights and models that come out of labs like reflection play in this ecosystem. So thank you for taking the time and excited to hopefully catch up soon. Of course. Thank you for having me. Thank you.