In short
The episode covers (1) a proposed U.S. bill to regulate AI agents, (2) why DeepSeek raised a first-time $7.4B round tied to Anthropic’s Mythos, (3) rising adoption of open-weight/open-source models amid U.S. restrictions, and (4) how DigitalOcean sees token/cost optimization and CPU-centric infrastructure for agentic workloads.
Guests and backgrounds
Leo Schwartz, The Information AI tech and politics reporter. Jing Yang, The Information Asia Bureau Chief. Stephanie Palazzolo, author of The Information’s AI Agenda newsletter. Patti Srinivasan, CEO of DigitalOcean.
Key claims
Warner’s agent bill would require “duty of loyalty,” privacy protections, and interoperability via an FTC agent registry; it would affect large platforms (50M+ users). DeepSeek’s fundraising was prompted by Anthropic Mythos’s early preview and the need for a “war chest” for compute/data. Open-weight models are favored because they can’t be “taken down” like closed models; U.S. government concern centers on cybersecurity. DigitalOcean argues “model orchestration” is the new FinOps and highlights CPU/sandbox primitives.
Notable examples
booking agents steering users to partner hotels; DeepSeek training with NVIDIA “black-market” chips and adapting to Huawei; Kunlun Sing (Baidu-linked chip IPO) seeking ~$50B valuation; Meta limiting employee use of CLAU D/Codex to prevent training-data “seepage.”
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOMeta's AI Tool Restrictions
0:45 to 1:36
Discussion on Meta limiting employee use of AI tools due to training data concerns.
“Today on the show, the information is first to report that a new AI agent-focused bill is expected to be unveiled today by a U.S.”
New AI Agent Bill Unveiling
1:36 to 4:00
Introduction of a new AI agent-focused bill by Senator Mark Warner and its implications.
“That is according to a scoop from the information's AI and politics reporter Leo Schwartz.”
Key Principles of the Bill
4:00 to 8:54
Exploration of the bill's principles, including duty of loyalty and interoperability.
“I think a good example is, again, go back to this idea of a booking agent.”
DeepSeek's Funding and Mythos Impact
8:54 to 14:01
Discussion on DeepSeek's $7.4 billion funding round and the influence of Anthropics' Mythos.
“That is Leo Schwartz, our AI tech and politics reporter here at The Information.”
DeepSeek's AI Chip Strategy
14:01 to 18:08
Explore DeepSeek's unique approach to AI chip acquisition and adaptation.
“And in terms of the compute, we all know that Chinese companies can now legally acquire the most advanced AI chips from NVIDIA.”
The Impact of DeepSeek's Fundraising
18:09 to 24:28
Understand how DeepSeek's fundraising changes the AI landscape.
“And with funding, too, is the point that you're making.”
The Rise of Open Weight Models
24:29 to 28:00
Learn about the increasing adoption of open weight models in AI.
“And that's what we're seeing for the larger background of this story on this IPO.”
U.S. Concerns Over Open Source Models from Asia
28:00 to 33:00
Explore the implications of open source models emerging from Asia and U.S. government concerns.
“I don't run the risk of having the model that I'm basing all my technology on kind of ripped out from under me at any moment without any heads up, basically.”
DigitalOcean's Role in AI and Model Orchestration
33:00 to 42:09
Learn about DigitalOcean's position in the AI landscape and the concept of model orchestration.
“Our next segment is with our partner, DigitalOcean.”
Demand for CPU-Based Computing in AI
42:09 to 43:40
Explore the need for CPU-based computing and innovations in software optimization.
“And the agents need a significant amount of compute that is driven by CPUs.”
Show all 12 chapters
DigitalOcean's Unique Offering in AI Cloud
43:40 to 45:03
Learn how DigitalOcean differentiates itself in the AI-native workload landscape.
“Yeah, we are different from the NeoClouds, which were built primarily to cater to their training workloads, and they do a fantastic job there.”
Meta’s Caution with AI Tools
45:03 to 46:26
Understand the implications of Meta's stance on using AI tools like Claude and Codex.
“What do you think that that story suggests to us about how big tech companies will look at these models, broadly speaking?”
Transcript
Automatic transcript. May contain errors.0:13Welcome, everyone, to The Information's TI TV. My name is Akash Pasricha. It is Monday, June 29th. Before we get to today's show, I want to flag a scoop that The Information has out this morning. Meta is limiting employee use of CLAWD and Codex over concerns that the output of OpenAI and Anthropics tools could end up seeping into Meta's own training data. That is according to our reporting. My colleague Jyoti Mann, who covers the company, reviewed some of the internal documents at Meta. I encourage you to give the full story a read on our website. Today on the show, the information is first to report that a new AI agent-focused bill is expected to be unveiled today by a U.S.
0:54senator. We'll have the details for you on that as AI and politics increasingly continue to intersect. We'll then have on our Asia Bureau Chief to discuss some exclusive reporting that she has on why DeepSeek decided to raise funding for the first time and what it has to do with mythos. We're then going to talk about the heightened prevalence of open source models with the author of our AI Agenda newsletter. And we're going to close out the show with a conversation with the CEO of DigitalOcean, who has a front row seat to the rise of open source and open weight models, and also how developers are grappling with the CPU boom.
1:32It's going to be a great show, so let's get right on into it. The seeds of a new AI agent focused bill is expected to be unveiled today by a Democratic senator. That is according to a scoop from the information's AI and politics reporter Leo Schwartz. I want to bring on Leo to share with us what he's found out. Leo, welcome back to the show. It's great to have you here. What is this bill and what should we expect? This is a bill from Senator Mark Warner, one of the top Senate Democrats, and it's focused on a fast-growing area of AI, which is agents. So this is anything from your open-claw agents that you're delegating to go out and send emails for you, or even if in the future there's a service where you can have an agent who you say, I want to book a trip to Italy, find me the best hotels and flight prices you can, and it goes out and does all of those activities for you.
2:21It's obviously a much more complicated thing than a lot of us are used to, which is just using ChatGPT or Claude or other chat interfaces. And this is a completely new frontier. So Mark Warner's bill is the first real attempt to create a legislative framework around tricky issues like how do you ensure that your privacy is ensured when these agents are off doing tasks for you? Or how do you make sure that a big platform like Google or Meta isn't blocking third party agents because it might threaten their business? And so you saw a draft of this bill? Yes. So I saw it's called a discussion draft.
2:58So it's going to go out. OK, yeah. What does that mean? What does that mean? It's not necessarily a final bill that's going to be introduced and debated. This is basically a version that will go out and say, here's what I'm thinking. Maybe he'll be able to get some bipartisan sponsors on it. Maybe the house will start thinking about its own versions. And most importantly, probably it'll start to get feedback from industry who will help shape what the final version looks like before it's formally introduced. Okay. So this outline, I guess, is what I'm going to choose to call it. So, I mean, you talk about how AI agents will be regulated in a sort of a framework around which these should operate.
3:42What are some of the key principles that the bill suggests as a way to keep AI agents above board? There's something called duty of loyalty, which basically is saying if you're using an agent, its first interest is to you, not to the developer who created it or maybe an advertiser who has some secret deal with it. I think a good example is, again, go back to this idea of a booking agent. So you say, I want to book a trip to Italy. If you're using a third-party agent that secretly has a deal with Hilton or Airbnb, it might direct you to that platform. But here it's saying, no, the agent has to actually have your best interests in mind.
4:21And another important one is interoperability. So this idea that if you're delegating an agent to go out and do work for you, a large platform won't be able to block it just because the platform wants you to use its own agents. And of course, that comes with other key things that it's looking out for, like privacy. So if you're entrusting an agent with your banking information, it's not going to go and sell that. Well, this is kind of interesting because, you know, we've talked about the corporate data wars on this show and the idea that agents, the companies developing agents and the companies that are sitting on data, sometimes they can be rival companies because the company sitting on the data might have their own agents, their own best interests.
5:03And so, I mean, the implication of this is that should this bill be introduced, should it pass, you know, it sort of sets at least some guardrails around who has to cooperate with who and what that interoperability could look like. I want to focus on Mark Warner here. Tell me a little bit about his track record here with tech companies and with AI. Has he been on top of this issue since the start? Yeah, I would say he's one of the more forward-thinking tech-focused senators. A lot of lawmakers are still just trying to understand what AI is. They're probably using Claude for the first time. And he's really been at the vanguard of this issue of agents.
5:43He had previously worked on a lot of social media-focused bills. This is actually an adaptation of an earlier social media bill that he worked on that also deals with this idea of interoperability. And he's been trying to pass some sort of legislation around agents for at least two years. He's been interfacing with other federal agencies, including the Treasury Department, to think about what agent security looks like for financial services. So what's interesting about this bill is it's his attempt to create a framework for agents across different types of platforms, whether it's social media, financial services, e-commerce or other areas along those lines.
6:26Do you think that this bill will pass if it gets introduced? Will any bill ever pass? I think this is the real question for DC. Any bill. I mean, hey, AI ages are in the grand spectrum of things. It's like the most important thing in the world, right? Hope springs eternal. I think we've remember that still no social media bill has passed like congress just does not pass tech focus bills but congress cares about ai right now there's a ton of bills that are being considered from more narrowly focused ones around issues like deep fakes or chatbots uh to broader ones like a a full framework for how the entire federal government should think about ai in general this is obviously a more narrowly focused one before you can actually think about its legislative of prospects.
7:17It has to find a Republican sponsor. It probably has to have a parallel effort in the House. Of course, the midterms are coming up in November. But I think this is an important starting off point for the Senate, at least, to start thinking about what agents actually are and how they should be regulated. Now, let me ask you a question here about whether or not this bill passes. I mean, there's going to be a lot of discussion from the companies around where they stand on this type of legislation. Do you think the AI companies are for this, against it? Does the bill target any specific AI companies?
7:54So the bill is interesting because it tasks the Federal Trade Commission with creating a register of agents, and agents can only access large platforms, which it defines as platforms with more than 50 million users or subscribers if those agents are registered. So this would impact Google and Meta and Amazon and all these large platforms. Whether they'll be on board with it, I think, is a different question. There are some interesting coalitions and groups being formed in D.C. by these large companies to deal with the issue of agents. But at the same time, obviously, an Amazon is not necessarily in its best interest to allow a third-party shopping agent to come on behalf of its users who aren't going to visit the site itself and see advertising and see all of the mechanisms that Amazon set up to try and incentivize the shopping that it wants.
8:42So I think you'll certainly see industry try to shape what this legislation looks like, whether they'll get on board with creating a framework that looks like this, I think, is anyone's guess. Great. Well, Leo, I want to thank you for coming on. That is Leo Schwartz, our AI tech and politics reporter here at The Information. deep seek's first funding round totaling 7.4 billion dollars is now complete but the information's asia bureau has some new reporting on the extent to which anthropics mythos model may have played a strong part in the reason that deep seek sought outside funding for the first time at all i want to bring on our asia bureau chief jing yang to share more with us about what we know jing welcome back to the show it's great to have you here thank you akash so jing you've told us before about the ways in which DeepSeek's first funding round ever was quite unique in the way that it was structured.
9:36You then went out to do some reporting on the rationale behind why the company sought to seek funding. What did you find? So we first reported that company was starting external funding for the first time in mid-April. And then, you know, having known the company and watched them for the past year and a half, that really shocked me. The next two months, while I and my team were reporting every step of the fundraising, one big question that has always lingered in my head is what really changed that prompted them to do this? Because we know the company well enough to know that the founder, the CEO, he's known as being quite resistant to venture capital money.
10:19So while we were reporting that and we finally, while we reported the closing, we found out that actually it was on Anthropik's preview of the Mythos model in early April that prompted all of this strategic change. So basically, DeepCity CEO, after seeing how Mythos was able to achieve advancements on on a totally different level because Ansarby was able to train the model on an enormous scale of compute and data, that made him realize something like an epiphany that if Dipsy were to stay in the game and to remain competitive in the long run, he really needed to build a massive war chest, at least in the realms of like tens of billions of dollars to begin with.
11:11And that That is very striking because remember when DeepSeq just exploded, become a global superstar in early 2025. The thing that made them so famous and so explosive was because they were able to achieve the kind of computational efficiency that US AI model developers weren't seen doing because they managed to achieve similar sort of capabilities on par with OpenAI's latest model back then, but with very little compute. So for a company like this, who's known and take a lot of pride in the kind of computational efficiency that they have achieved, to actually now embracing more data and more computing, which leading to more capital needs, is really, really striking.
12:00Now, did they get access to Anthropics' mythos model? And this was a model that nobody could really get access to. well i wouldn't say nobody um first i it's not clear from our reporting exactly how um they got access to mythos um but what i would like to point out is that um i thought they did have this uh highly curated preview right some project last swing there were 100 150 plus partners so but We don't know. I mean, it would seem obvious that DeepSeq and High Flyer Capital Management, the company, I mean, I don't think they were part of the project glass wing, right? They weren't on paper, but, you know, just bear in mind, again, I'm not pointing any fingers because I don't know.
12:52No, no, no. However, within Project Glass Wing, you do see some open source community participants. let's put it that way, that were also included, right? Because you cannot ignore that community. And then on top of that, I believe that it was an author that also reported that there was some brief unauthorized attempts at accessing the model on day one. So, you know, there could be so many different ways where they may have got a glimpse or some kind of, you know, time, screen time with the model, you know, within the really powerful and smart AM research community is really, I think a lot of people know a lot of people, right?
13:34Yeah, it's a small circle of people. So this is kind of interesting. So Anthropics Mythos comes out. It sort of spooks everyone, including the DeepSea team. They decide around this timeframe that, hey, we need to go out and raise money. And it's$7.4 billion at a$50 billion valuation. What are they planning to do with the money? So it's just doing two things, you know, buy more compute and hire more people. And in terms of the compute, we all know that Chinese companies can now legally acquire the most advanced AI chips from NVIDIA. We did report last year that DeepSeq was training their latest model, which was released later this year called V4, on NVIDIA's black-world chips they secured from the black market.
14:22And in addition to that, we also reviewed in our latest story that they actually were the one taking the initiative, contrary to what others might have thought. They were the ones who were taking the initiative to adapt their models with Huawei chips. And so we would imagine that in spending CapEx on compute, that there would definitely be a Huawei piece of it. And then the company also announced that they are going to at least double the head counter in every department. Which you pointed out was it's a rare announcement for DeepSeek to make it all. Yeah, they rarely communicate with the public.
15:07The only time they do is probably when they have a new model coming out. And then they have a blog post and sharing a bunch of technical details for the model. So for them to actually come out with such an announcement detailing their hiring plan and what kind of positions they're hiring, including interns, is quite rare. Right. I want to go back to the Huawei chips. So what is it that DeepSeek and the leadership over there see in Huawei chips? So I think essentially this boils down to two factors. The first one is that it is simply just the reality that Chinese companies have this urgent like compute shortage.
15:49And so you'd better just embrace and then try to, while you work with whatever invadia stockpile you have, you might as well also take the initiative in adapting to Huawei, which will become the new reality and the new normal in training and models in China. That's reason number one, out of practicality. And really number two, maybe a little bit more driven by vision. As we also reported that DIPC CEO is, to use today's popular term, I guess, is AGI-pilt. and he then to that extension, he believes that technology should be inclusive and then while AI should be inclusive, which is why he has insisted on making Deep Sea models open source on the chip and infrastructure front.
16:39He also believes that there should be just one ecosystem dominated by one company. He wants to play his part or Deep Sea should play his part in diversifying the entire AI chip you know, ecosystem beyond a video-led order as well. Right. Jing, you said in the story that DeepSeek's expansion could be a watershed moment for the AI sector. What did you mean by that? Because up until recently, DeepSeek still has been seen as a bit of an outlier, you know. Yes, they're very smart, but they're small, and they're also extremely low-profile because they had never raised money, right? In my opinion and my observation, they were, until the recent fundraising, they were the only holdout in the major model developers and AI labs in the entire world that has not embraced capitalism, that has not come out and seek funding and hire extensively from the outside.
17:47And then that just changed with this fundraising. And so that just makes the US-China AI race very different from this moment on. Suddenly, China has this real established serious player. And God knows what they can do going forward. And with funding, too, is the point that you're making. They were able to royal markets without any funding at all. Just imagine what they can do with a lot of capital. Yeah, and I'm sure this won't be the first time that it raced, right? What company raced is only one round. Right, and going back to the chip story, I mean, look, if you have a well-funded company that is banking on Huawei and chip companies other than NVIDIA, I mean, suddenly you do have these two ecosystems that finally there is more capital to spend on chips.
18:44and it sort of gives credence to the idea that there's going to be more competition going forward. Yeah, yeah, absolutely. Jing, before we let you go, I want to ask you very quickly about a story that our colleague Channer published about a company, a chip company, that Baidu has a significant stake in that is planning to go public. She had some good reporting on that. What did she find? Yeah, so this chip company is called Kunlun Singh and the name Kunlun is actually the name of a super low Munte range in China revered in the Taoist mythology as the ancestor of Munte. Have you been there? I unfortunately have not, but I just want to set the stage here that that shows you this company's, this chip company's ambition, right?
19:33So this was essentially Baidu's chip effort. It was created as a subsidiary inside Baidu back in 2011 or 12, so pretty long time ago. And now, basically, the company is going public and seeking to go public Hong Kong. And there are two interesting things that we revealed. First is that Kun Lun-Sing is seeking a valuation of$50 billion as IPO to put that in context. that is nearly 40 % more than what Baidu, which owns 58 % of this company, is currently valued by the market. And then that shows you sort of how hot this trade is expected to be. Do we know how much Baidu owns in the subsidiary? 58%, okay.
20:27So the subsidiary is 58 % owned by the parent, but it's 40 % more valued. Okay, that was the number I was missing. More valued by its current. Yes. And then the second interesting thing we reported is that while the company is talking to investors and trying to firm up this lineup of cornerstone investors, which is essentially a bunch of investors, you know, anchor the shell sale, very important role. They asked some of these investors, In the case, if they were Chinese government-backed funds that have a mandate in investing in AI data centers or semiconductors, they asked these kind of investors to consider buying their chips, their products.
21:14And some of them were asked to place or commit chip orders worth multiple times of the IPO share subscription they're planning for. and this is quite unusual. It feels like it's sort of close to the circularity that we have talked about here in North America at least and the risk that I see, correct me if I'm wrong here, but if you have IPO investors that are being told to buy chips, kind of artificially inflates demand, at least in the short term, right? So indeed, there has been so much circularity going on in the US among the tech giants, but I can tell you that we simply haven't seen this happening in China until what we reported, number one.
22:03And number two, in terms of whether it is inflation of your sales, let's put it this way, right? All the credible stock exchanges in the world, including Hong Kong, have rigorous rules on disclosure of transactions with connected parties. If an investor also becomes a, If a substantial investor also becomes a substantial customer, that definitely qualifies as connected transaction. And companies are expected, according to the rules, to make disclosures. And then when that happens, it is up to the regulators and investor to decide whether these transactions, these sales, are bona fide or just, you know, friends and family.
22:49One-off deal. Yeah, friends. That's a good way of putting it. Jing, last question for you. What does this IPO and this chip IPO tell us about the chip landscape in China and in Asia, broadly speaking? So we already saw about five AI chip designers going public between mainland China and Hong Kong since December. So this Baidu unit would be the sixth. And then this high valuation they're seeking tells you that investor sentiment certainly is quite euphoric toward these kind of stocks. However, the fact that they also have asked some investors to commit buying the chips also might show you that the market is getting really tough out there for AI chips, especially chips designed for mostly inference types of work.
23:47Essentially, what I'm saying is that while the U.S. export controls have pushed China to be more technologically self-reliant, it also has created a really crowded market where basically most chip designers in China these days, AI chip designers, have cornered on this market segment of inference chips because it is easier relatively to make and design than chips used for training. But then when you have a crowded market with the homogeneous competition and products, then everybody just goes to race to bottom and then try to lower their prices. And certainly it's not the healthy competitive dynamic.
24:31And that's what we're seeing for the larger background of this story on this IPO. Great. Well, Jing, I want to thank you for coming on. That is Jing Yang, our Asia Bureau Chief, here at The Information. The White House's clampdown on top AI models means that open source models as a whole are becoming a lot more attractive to developers. My colleague Stephanie Palazzolo, who authors our AI Agenda newsletter, published a column on that with some inside reporting. I want to bring her on to share with us more about what she is hearing. Stephanie, welcome back to the show. So you talk about companies increasingly turning to open source and open weight models.
25:11We've talked about open source models, of course, on the show. So what's an open weight model? Let's just get that definition out of the way. Yeah, so actually it's kind of funny because whenever a lot of people talk about open source today, they're actually mostly referring to open weight models if we're trying to be technical about it. Essentially, you know, open source models means that everything about the model is public and available to download, including both the kind of code and the weights behind the model, as well as the data that it was trained on. So actually today, most of what people call open source is actually open weight.
25:47And what that means is that the kind of code to run the model and the weights of the model, which are kind of like the settings of the model that determines how it responds, those are available to download. But most of the time, the kind of training data is not, which is why it's called open weight and not open source. Got it. So I'm imagining like one of those, like, not the Venn diagram, but like the circle within the circle where like every open. So it's like every open source model is open weight, but not every open weight model is open source. Got it. Okay. Well, let's just call them open models for now.
26:24Okay. How about that? Because it sounds like they're somewhat interchangeable. So look, you wrote this column today about the rising use of these open source, open weight models. uh why is that we've talked a little bit about cost on the show uh sounds like maybe there's there's a regulatory angle to this argument as well yeah so as you mentioned obviously a really big um factor here is cost and that's something that we've seen lots of founder ceos talk about um including you know the coinbase ceo over the weekend talking about how he's able to really cut down on cost by using um newer open source models that uh are you know fairly close to the frontier But a second reason, which I wrote about in the column and which is coming up more frequently in the kind of past couple weeks, is the fact that a lot of founders and developers have been burned by trying to build apps on top of these really advanced closed source models like Fable, for instance, or, you know, the equivalent from OpenAI.
27:24And then seeing those models get taken down or removed from the market just days later. right so you can imagine how annoying it is for a ceo if you're in the middle of building a new app or moving an app onto you know the fable model and then all of a sudden it's gone and you're like okay i just spent you know hours or days trying to move things over and build an app with these better capabilities and now it's it's kind of uh useless to me so basically a lot of developers are saying okay you know open source models once you kind of download them and the weights are out there, it's basically impossible to kind of, you know, take the cat out of the bag once it's out.
27:59And so they're kind of just saying, okay, maybe I should be using open source models more because I don't run the risk of having the model that I'm basing all my technology on kind of ripped out from under me at any moment without any heads up, basically. Well, and just connecting this to the segment we just had with Jing, I mean, there are a lot of great open source models coming out of Asia and out of China. And I see a lot of discourse online, you pointed out in your column, that they are becoming increasingly popular too as the open models to use. Is there any concern from companies in the U.S.
28:40that the open source models or open weight models are coming out of Asia at all? Well, I would say in terms of from companies in the U.S., I would say They're actually mostly excited because they're, you know, excited for anything that's cheaper, that's matching the capability of, you know, today's most advanced models. I would say there's definitely concern coming from the government. Right. That's right. Exactly, yeah. So, you know, even over the weekend, there were reports that some of the models coming out of China have actually, you know, reached the level of anthropics, you know, mythos or fable models whenever it comes to cybersecurity, which is of a huge concern to the government, obviously.
29:21So I think it's something that they're very, very worried about. And this kind of China versus U.S. race is also playing a major role in, you know, this debate around, OK, does it make sense for the U.S. to clamp down on these advanced models or to take them off the market? You know, a lot of people are arguing that the U.S. is basically handicapping itself by, you know, making things harder for the model developers and kind of giving China a chance to catch up while Anthropic and OpenAI are tied up and not able to release their models to the public. And our open source models, going back to what you're saying, when the cat's out of the bag, it's really hard to put it back in.
30:00Are they sort of unregulatable in a sense? Once they get released, once they get downloaded, can the U.S. government actually do anything to prevent people from using these models? Well, it's tough to say. I mean, this is kind of uncharted territory. So theoretically, I think there probably would be some way for them to put some guardrails around this. So you can imagine, you know, maybe the U.S. government says, you know, companies that do business with us are not allowed to use these models. Otherwise, you won't do business with you. And then that obviously puts pressure on companies that have the U.S.
Read the full transcript
30:38government as customers, for instance, to stop using those kind of Chinese open source models or else they risk losing maybe a really big chunk of business. um but you know once again once the cat is out of the bag once that software is out there and a lot of the times these models are super small they can fit on you know a phone on a laptop right like it's it's not even connected to like to the cloud at that point it's literally just running on your laptop like that is super hard to you know pull off of people's devices and i don't even know how the government would would go about trying to do that one of the arguments you make in the column is that you say that you think that model purgatory is here to stay, this idea that the government has restrictions, I guess, on how these models get rolled out.
31:25You actually make the case that maybe that will be some kind of a norm going forward. Why do you think that? I mean, you know, as I talked about a bit last week, after Leo and Amir and I broke our story on Thursday, I think a lot of model developers are waiting for this voluntary framework to come out. So, you know, in the background, companies like OpenAI and Reflection AI, they're all working with the government to figure out this framework that is, you know, opt-in or voluntary that basically lays out, okay, here's the process for how model releases should go. The company should share models with the government, you know, X days in advance.
32:04And then there's this process that they go through to evaluate those models. So once that is like kind of out there, people think like, OK, that's great. There's a set process. We just if we just follow that, we're going to be good. I think I'm trying to argue that clearly the Trump administration has no issue kind of ad hoc, you know, randomly like making exceptions to this and saying, you know, actually like anthropic, we're just going to tell you like this time, like you need to take that model off the market or we're banning that to foreign, you know, to non-U.S. citizens. So I think even if we have this voluntary framework, that's not going to protect the companies from like the actions of the Trump administration and kind of them ad hoc to saying like, hey, I know this isn't part of the voluntary framework, but like we're really concerned about this model.
32:51So you need to do X, Y, Z or take it off the market. Right. Great. Well, Stephanie, I want to thank you for coming on. That is Stephanie Palazzolo, author of our AI Agenda newsletter here at The Information. Our next segment is with our partner, DigitalOcean. The heightened focus on open source models and tokenomics, and also the focus on CPUs is one that DigitalOcean has been at the center of as the company sells products for developers. To break that down more, I want to bring on Patti Srinivasan, CEO of DigitalOcean for a conversation. Patti, welcome to the show. It's great to have you here.
33:27Yeah, thank you so much for having me. So Stephanie mentioned on the prior segment, but we saw this post from Coinbase CEO Brian Armstrong over the last few days. He talked about how Coinbase has been able to half, essentially, their spending on AI while remaining fairly strong in their use of tokens. I wonder if that aligns. I mean, he talks about some of the methods that he was able to do that with in the post, which we'll get into, but does that broadly align with what you're seeing in the market right now? Yeah, absolutely. We are seeing this. And as a cloud platform provider, this is music to my ears.
34:07You know why? Because as with cloud, the optimization started after 15 years of cloud becoming mainstream. If cloud started around circa 2010, it wasn't until after COVID that we all woke up to the runaway cost that cloud had become and companies started optimizing it. In this era of AI inferencing agentic era, the optimization and the discipline around FinOps has started really early, and that is fantastic. And as I like to say, model orchestration has become the new FinOps. Model orchestration has become the new FinOps. What do you mean by that? Yeah, what I mean by that is, and this again goes back to what Brian Armstrong was writing on X over the weekend, where it is not about just standardizing on a single model.
34:58So often production AI is an ensemble of models. You need to pick the right model for the type of workload that you're trying to execute. In many cases, even a single prompt gets routed to multiple models so that they can all do the best job for the task that they've been optimized for. So in a way, when you orchestrate the models based on their cost performance trade-offs, you get the best performance for the task on hand. And that's what I mean by model orchestration is now the new FinOps in the AI-agentic era. Now, Brian Armstrong also talked about two other factors. He talked about a number of factors, but better defaults and better caching were two other things that he talked about.
35:45Can you explain what he might have meant by that? Yeah, so caching, Akash, is a concept of when you have seen enough of the same context, especially in coding, maybe you're working on the same code base. So much of the context can be cached and the model knows your context without having to reprompt. So the caching in many cases can reduce the token by an order of magnitude. In many cases, like 80-90 % of the tokens can be cached based on what your context is. And that is becoming a very, very common pattern to reduce the amount of costs, or reduce the amount of tokens and hence the cost. So there are multiple patterns that are emerging.
36:28So number one thing is just routing, right? Where you have a very small language model that is making the determination to invoke the right model for the right workload. So that's number one. Number two is there's another pattern called model cascading, where you almost always start with a very small, cheap, open-made, open-source model. Then there's an evaluation that happens to see if that answer is good enough. If not, it escalates to a much bigger model. This works when the task is often in an asynchronous mode. There's another one where you have an ensemble of models where you run the same prompt through multiple LLMs and there is a judge.
37:09It looks at the LAMA output or the Nemo Tron output with the Opus 4.8 output, output and makes the right decision. And then finally, there is an orchestration of models based on the workload. So there are multiple design patterns that are emerging to manage all of this. So let's turn to then what you do, how DigitalOcean fits into this. So, I mean, help us understand your platform. Are you essentially a platform that companies can access these open source, open weight models, closed source models through to help build their applications? Is that how it works? Yeah, exactly. So we've been in the business for about 15 years.
37:51We are a full-fledged cloud platform provider, much like AWS or Google Cloud Platform. And now over the last couple of years, we've started focusing and almost exclusively serving the needs of AI native companies. And the AI native workloads use a lot of inferencing, but they're also becoming very agentic. So you need to have the ability to think, which requires a lot of inference tokens, but you also need the ability for these agents to perform actions, orchestrate workflows, remember state information across different calls. So you need a full-fledged computing platform with modern cloud primitives to be able to do the action part of what the agents are capable of doing.
38:36So that's what DigitalOcean provides. So let me ask you a question about these open source models. I mean, we are in this moment right now where they are becoming more prevalent, more popular because of, we had Stephanie on, there's the regulatory reasons for going to open source models. There's the cost reasons. you know my sense is that the top AI labs like Anthropic and OpenAI I mean they're not just going to stand idly by as people figure out that open source models are a cheaper alternative so my question for you is do you think this lasts I mean do you think Anthropic and OpenAI find a way then to make even if it's the model that was you know launched six months ago for for example, I mean, do you foresee a shift back to closed source models or is this shift permanent now?
39:32Yeah, so this is a fascinating question, Akash, and this is one that I'm very familiar with in terms of a pattern. I used to work at Microsoft in my early days and we saw the evolution of Windows Server versus Linux Server, and it took a few years or a couple of decades to pan out, but then Linux became the dominant open web operating system, but Windows Server became the dominant enterprise use case where you need more compliance and regulatory considerations have to be taken into account. So I think a similar thing will emerge here where it is going to be a mixture of open source models becoming the dominant usage model, while a majority of the dollars will accrue to the closed systems.
40:18but I think in production, especially for AI native workloads where companies are building and monetizing software, they're going to have a mixture of models. It is not going to be just one and done. So I presume that there's going to be a tremendous amount of open source adoption, especially as we shift towards an agentic world where agents are going to make the decision, not humans. So we need to put some guardrails in place so that there's the right balance between cost and performance. What about the shifting dynamics for chips happening underneath this? We've seen agents, as you mentioned, the craze around that has actually led to a lot more conversation around CPUs.
40:58How do you think about that from a software angle and how people optimize their use of CPUs, given that, you know, we'll talk about the memory crunch in a second, but there's only so many CPUs to go around, right? Yeah, exactly. And we are seeing this over the last several months. OpenClaw came out in January and there's been an explosion of personal productivity agents, coding agents everyone knows about. And this concept of agentic software is just now permeating to all kinds of SaaS workloads and also workflows that have not been automated so far. And the estimates are token consumption is going to be 30 to 40x between now and 2030, which is only three years away, if you think about it.
41:46So when we get to 30, 40, when we start consuming four or five trillion tokens per year, 70 % of the tokens are going to be generated by agents. So the agentic workloads are going to be multiple orders of magnitude bigger and more important than human-based workflows or workloads. And the agents need a significant amount of compute that is driven by CPUs. It will, of course, require a lot of tokens, but a majority of the time agents spend is an actual orchestration and execution of the workflows, all of which run through some kind of a sandbox CPU-based environment. It needs databases, it needs memory, it needs everything that cloud platforms have.
42:33And so bringing it back to the chip side of all this, then is there a way to use software then to optimize for use of CPUs or even use of memory, given that there's the memory crunch happening? Absolutely. There are new computing primitives that are taking shape now as we speak. Us, along with other participants of this ecosystem, are inventing new secure sandboxes, which are somewhere between virtual machines and a formal like Lambda functions. These are very, very small footprint virtual machines that can persist memory and state information across execution. So they are really quick, like less than 100 milliseconds to hydrate and dehydrate.
43:18They execute for a couple of seconds and then just go into a sleep mode. So a same CPU can yield significant amount of compute power compared to a more heavy-duty virtual machine. All this to say we need a lot of CPU-based computing, but we are also inventing in the software layers to make use of what we currently have. But if you fast forward three or four years, we're going to need a lot of computer. So, Patti, let me ask you then, if that's the solution that you're developing in DigitalOcean, how does that differentiate itself compared to all the other companies out there that develop similar software that can help navigate the chip landscape and the tokenomic landscape?
44:02I mean, how are you different? Yeah, we are different from the NeoClouds, which were built primarily to cater to their training workloads, and they do a fantastic job there. We don't really compete in the training compute workloads, which often requires large-scale bare metal clusters of GPUs. We are in the AI-native workload business, and AI-native workloads need an AI-native cloud. So as I like to say, from silicon to agents, we have a five-layer architecture that enables AI-native workloads to run seamlessly in a single stack rather than having to stitch together inference tokens from one vendor, GPUs from a second vendor, and cloud primitives from a hyperscaler.
44:49We enable AI-native companies to do all of that in a single stack, which is also very open by default. So that's our differentiation versus the other players in this ecosystem. And, Patti, before I let you go, we also published a story this morning about how Meta is also encouraging their employees to basically not use Claude or Codex as much because of the risk here that perhaps the outputs from the Anthropic and OpenAI tools could actually seep into the ways in which Meta is training its own model. What do you think that that story suggests to us about how big tech companies will look at these models, broadly speaking?
45:39Yeah, so there are two angles there, right? One is the cost optimization angle, of course, the token maximization to token optimization. I think that is well documented and Meta is the latest company to jump on that bandwagon. But there is something bigger than that here, which is the whole concept of governance and ensuring that you're not breaching any kind of intellectual property norms here. So I think as you're trying to also compete with these frontier models, I'm sure there's a lot of heightened awareness in terms of what kind of techniques are being used to develop your own intellectual property.
46:20And we want to make sure that you're safeguarding the interests while protecting the rights of your partners. Great. Well, Patti, I want to thank you for coming on. That is Patti Srinivasan, the CEO of DigitalOcean here on TI TV. that does it for today's show a reminder we are on this stream monday through friday at 10 a.m pacific 1 p.m eastern if you can't make it then episodes are available on theinformation.com on our youtube channel or wherever you get your podcasts make sure to follow us on social media on x on instagram on tiktok and on linkedin i am already excited for our next show tomorrow have a great rest of your monday bye-bye for now
47:03Thank you.
47:33Thank you.
From the publisher
DigitalOcean CEO Paddy Srinivasan talks with TITV Host Akash Pasricha about token optimization, caching, and how developers are managing the CPU boom. We also talk with Tech & Politics Reporter Leo Schwartz about Senator Mark Warner's upcoming discussion draft targeting AI agent guardrails, and Asia Bureau Chief Jing Yang about DeepSeek's massive $7.4 billion capital injection triggered by Anthropic’s secret Mythos model preview. Finally, we get into the rising developer shift toward open-weight models and the resulting regulatory tension under the Trump administration with AI Reporter Stephanie Palazzolo.
Articles discussed on this episode:
https://www.theinformation.com/articles/sen-mark-warner-unveil-ai-agent-bill
https://www.theinformation.com/newsletters/ai-agenda/open-source-models-benefitting-white-house-clampdown
Subscribe:
The Information: https://www.theinformation.com/subscribe_h
Sign up for the AI Agenda newsletter: https://www.theinformation.com/features/ai-agenda
TITV airs weekdays on YouTube, X and LinkedIn at 10AM PT / 1PM ET. Or check us out wherever you get your podcasts.
Follow us:
X: https://x.com/theinformation
IG: https://www.instagram.com/theinformation/
TikTok: https://www.tiktok.com/@titv.theinformation
LinkedIn: https://www.linkedin.com/company/theinformation/
Chapters:
00:00 - Introduction
01:13 - Meta Restricts Employee Use of OpenAI & Anthropic Tools
02:55 - Inside Mark Warner’s New AI Agent Legislative Framework
10:25 - Why Anthropic's Mythos Spurred DeepSeek's $7.4B Fundraise
14:57 - DeepSeek's Black Market Nvidia Chips & Huawei Adaptation Strategy
20:06 - Baidu Subsidiary Kunlun Xin Targets Massive $50B IPO
26:17 - Why White House Clampdowns Are Driving Open Weight Adoption
34:04 - DigitalOcean CEO Paddy Srinivasan on Token Caching & CPU Demand
