AI Safety Push Sparks Demand for Watchdog Groups, China’s View of AI Safety, Compute’s Down Payment

26:47There is clearly, you know, and Dario is quite explicit about intending to keep China behind. So there's sort of dual concerns that I see. One, we're behind right now. And AI is also, by the way, one of the ways that we use to defend against attacks, right? So you're effectively taking away our shield. In fact, if you go to GLM and their blog, one of their blogs from August on cybersecurity, that is what they're saying. They're saying we want everyone to have AI as a shield against these attacks because these attacks are going to come fast and furious in the future because that's where the capabilities are at.

27:29And then the second thing is that we don't see this existential risk of rogue AI agent swarms just wiping out humanity because we don't fit in their goals or whatever as an immediate risk. And the Huawei chairman or one of the Huawei executives actually just said, it's quoted all over media, Chinese AI models are just not at that point yet where that is a key factor. I mean, he doesn't represent the state, but that is a view that I hear. So looking ahead to next week then and this meeting that's going to happen at the White House with President Xi and President Trump, how do you expect that to come up in conversation?

28:18What are you ultimately hoping gets discussed, comes of that summit? Are we expecting there to be any decisions get made or is it really just going to be discussion? Um, you know, personally, uh, I think that there has been, uh, you know, the, the, the things are moving so fast, right? So a couple of weeks ago, I would have said my expectations are very low. I think that is kind of widespread across the tech community on both sides of the Pacific. But now, you know, you see the U.S. government kind of putting out forth certain overtures to show that, you know, they're taking this discussion very seriously.

29:08And I think on the Chinese side, by the way, there's always been a lot of discussion. I always get asked by Chinese companies, Ray, what do you think of what's happening on the 24th? That's what they just referred to as the 24th, you know, because it is a very momentous occasion. It is like, I think the first time there is going to be a serious good faith effort to come to some understanding. That being said, you know, it's such a weird time in our election cycle and there are so many problems. All these issues are so new and unknown. I would personally find it very surprising if some substantive framework was introduced.

29:44I think just an open conversation would be good. One of the things that we've talked about on this show is China's ambition to basically create a parallel AI ecosystem of sorts where they have their own chip companies domestically, their own cloud companies, their own model companies. I mean, they can really just emulate what is happening in North America on their own and therefore be unreliant on any other outside entity. Can you give us sort of a status update on that? You talked about the models not being good enough. We obviously know that the chips are another factor here, and I think they would like to get their hands on the latest and greatest NVIDIA chips if they could.

30:35Where does all this stand right now? How much progress have they made towards that goal? And is it really like the chips that's the limitation here, or what's your read on it? Yeah, the chips are absolutely the limitation. I think it is for globally, really, right? But for China specifically because of these very stringent export controls, then I think the self-sufficiency drive, what we've seen the results, and this week is a good week to ask because the leading player for semiconductor self-sufficiency is Huawei. And they're having their annual conference this week, Huawei Connect, where they've unveiled some of their latest advances.

31:17And what I would say is that right now, while they are still behind, they're trying to use other methods to catch up, right? So instead of, I think you've probably heard of this, instead of having the exact chip for chip, they're just going to go, well, I'm going to connect a lot more systems together. Oh, sorry, a lot more chips together into a giant system. And their goal so far, what they've announced for next year is a 4 ,000 chip cluster as sort of a pod. Yeah, I should say pod, because then their goal is to connect that into a million chip cluster. And in fact, they said that they have a 256 ,000 chip cluster in deployment right now.

32:00Now, in deployment might mean that it just got started building, but that is very, very large. And it should theoretically support like a 10 trillion parameter plus model. So we had an article earlier this year where we a little cheekily said China closed the AI stack. This is the month China closed the AI stack because you have seen, while it's not particularly effective just yet, that Chinese model companies are training as well as serving inference. Serving inference has already been happening for a while, but are already training one trillion parameter models. They're just, you know, not the frontier models just yet, but that might happen with the next generation or two with Huawei chips.

32:45We don't know. Right. Let me ask you one last question before you go. So you are an investor yourself as well. Help me understand what the current state of venture capital activity between China and the U.S. is right now. You know, we saw sort of some shakeups a couple of years ago. Sequoia obviously split its arm. How much U.S. venture capital is going into China right now? Where are the trends going? Talk about that. Yeah, so a lot of our research is read by investors, but public market investors for private investors is very little. It's basically decoupled. I don't see anyone actively really investing in Chinese companies at scale, And part of it is because different capital exits, right?

33:35The Chinese companies are increasingly exiting in U.S. – sorry, in Chinese markets. Therefore, it just doesn't make sense for U.S. investors. Also, all this geopolitical tension, right? You see something like the Manus AI deal unwind. So it's basically – it's frozen right now for all intents and purposes. For all intents and purposes. I think there are companies with operations on both sides of the Pacific that get investment, but this is a very small number. Right. And Ray, I mean, maybe just again, looking ahead to this meeting next week and where this could go, what would it take to revive that funding channel, really?

34:20You know, how do we get to a point where U.S. investors are comfortable investing in China again? I think it would just be very difficult because, like I said, China is also trying to build up its own capital markets. And, you know, even like a relatively harmless company in terms of technology, at least like Xi 'an, couldn't get approval in the U.S. or London stock markets. So I think that's a pretty clear signal that I think there is desire to decouple, especially when it comes to high tech companies. They're getting much higher valuations in the domestic market. So I'm not sure that just that alone and the currency controls, I think, would discourage that kind of investing for a long, long while.

35:07Right. Great. Well, Ray, I want to thank you for coming on. That is Ray Ma, founder of TechBuzzChina here on TITV. rcai a company developing open weight models this week raised 150 million dollars in a funding round led by vista equity partners cambium capital and emergence capital the company has a pre-money valuation of 1 billion dollars i want to bring on mark mcquade founder and ceo of the company for a conversation mark welcome to the show it's great to have you here hey thanks for having me so the way i understand what you guys do i mean you're you're really in the uh open weight race here right?

35:42You're trying to be the American leader in open weight models. Is that the idea? Yeah, that's the goal. The goal is to, you know, the best models in the world today are being developed in China. And we want to be the US, you know, kind of counterbalanced to what the what China is putting out into the world from from open weight perspective. So yeah, we're excited to take the next step here. Okay. And and I mean, it's a pretty competitive playground here. I mean, you've got nvidia uh in the mix you've certainly got uh the other neo labs uh going for it here so uh where are your efforts at and what gives you confidence to that you can win this race yeah i mean there's it's actually uh there's not that many players in the u.s which is uh you know kind of i hope that changes uh you know i hope more people come in uh into the open weight space in the u.s but i mean the u.s is is pretty much dominated by the closed systems right and open ai Anthropic, X, Google.

36:40On the open weight side, yeah, there's obviously competition. But we've spent the majority of 2025 training models at a much more efficient clip than most labs do because we just didn't have the capital. So through that, we were able to really craft and perfect our training architecture and our model architecture to the point where now that we have some real capital behind us, We're very confident in what we'll be able to accomplish with our with our next generation models, which are which are training right now. So, you know, full confidence over here on what we'll do and using, you know, what we were able to accomplish in 2025 as the proof point.

37:19When can we expect those new models? We'll have the first of our next gen models released, you know, mid to end of October. And then we'll fast follow with a few more. So I'd like to say I like to say we'll be on a we'll be on a generational run here and in the open model ecosystem in the US over the next few months. Yeah. So let me ask you about the headlines of the moment right now. So this whole conversation about AI safety, who is best to regulate it, the conversation about pacing the frontier. I mean, this whole perspective that the closed source labs pacing the frontier priorities here, the idea that they're just saying that because they're afraid of open weight competition.

38:06Do you buy that argument? I think it probably has something to do with it. I don't know if it necessarily is kind of the core reason behind it. I think, you know, from my perspective, I absolutely believe, you know, in safety and alignment of models, you know, especially at the frontier. I think, you know, obviously OpenAI and Anthropic are at the absolute frontier. And, you know, I absolutely think they should, you know, monitor and safety and police themselves, you know. And I think, you know, Zuck had a good way to look at it when he posted on it was, you know, they spent an extra few weeks, you know, not releasing a model.

38:46right to ensure the safety and the alignment of it and it's really the responsibility of who's building the model to ensure that and and they did a great job before putting it out into the world uh and he had a great point saying that you know if if a model isn't you know uh aligned with what you want as a human then it becomes a bad product and no one's going to buy it therefore you're not going to make any money all right no one's going to use it so um i think that yeah i mean i believe in it uh but having you know open ai and anthropic pace the frontier themselves and decide what others do i think is a little much uh but i absolutely believe they should do that to themselves yes uh the absolute frontier right they should have that in mind well and and so you know we've talked about the the the lag between open weight models and uh closed source models on this show a little bit and you know that lag sort of it it widens narrows i mean it's fluctuating all the time, week by week, all of the fears around RSI and agents going rogue and stuff like that.

39:44My understanding is a lot of that is really to do with the closed source models right now. And my question for you is, are we going to get to a point where our open weight models also going to approach that level of risk or approach RSI? Are the open weight models going to get that good as well? What do you think? Yeah, it's a great question. I think that over time, absolutely. I think open weight models have, you know, really gotten stronger and stronger and closed that gap significantly over the past 12 months. But again, open weight models will also be dangerous then that means. Well, I think that it depends.

40:20I think open weight models can be dangerous if not trained, you know, with the appropriate safety and alignment in mind. I think that they will get to a frontier level at a certain point. I actually think that, you know, if you look at the hugging face attack as an example, it was actually open weight models that helped catch that, right, and fix that, where the closed models were the ones kind of going out. So I think, you know, I don't think it's a closed versus open, what can be dangerous, what cannot be dangerous. I think it's really a matter of, you know, ensuring that these models are appropriately released with the appropriate safety and alignment in mind.

40:56Yeah, no, it could be an open weight model. Absolutely. Why was that, by the way? I mean, the idea that an open-air model was able to help investigate it, rectify the issue. Why? Was there something about that particular model, some reason they weren't going with a closed-source model there? Or what's the context there? I think it's more just, you know, Hugging Faces is an open model platform, not a closed-source platform. So they had to use the tools at their disposal. And yeah, they were able to utilize an open-weight model to catch that. So yeah, there's kind of a good counterbalance, right?

41:25So RCAI is doing work with the Department of Energy as well. And I want to ask you about your experience working with the government and also their perspective on what AI they are comfortable using. Are they comfortable using only open way to combination of closed source open way? Where do they stand right now? Yeah, I mean, I think it's pretty well known that inside the government, they're utilizing both open and closed to a certain extent. For what we're doing with the Department of Energy, it is really focused around automated science, right? So it's their science environments. And it's training a model that is specifically built to become the best automated science model in the world.

42:11So in order to accomplish that, open wait is really needed because they need to adapt it further across, you know, the 17 labs they have from within their department. So, yeah, I mean, it's really focused around what environments the scientists use from within the DOE and what data they have. And then ensuring a model can be used, you know, as part of that wider system, really as a tool from within the automated science workflows, as opposed to, you know, just hitting an API. So all these concerns that Alex Karp and company are suggesting about data safety, data sovereignty, and stuff like that.

42:52I mean, if the government is using closed source models, from your perspective, what you're seeing, they don't seem to have that same concern then. Is that right? Well, I don't know how extensive government's using closed source models. I'm sure if they are using, you know, say an open AI or an anthropic, they have some kind of agreement with them in regards to what their data, you know. Yeah, because they're very, I mean, they're sort of bespoke deals as we've reported a lot of them. Yeah, I would assume. I would assume, you know, I don't have the inside details of that, but I would assume they are, right?

43:21You know, any deal there would be the data cannot be released. But, you know, there's also a level of customization that is required, especially in the work we're doing with the government, right? That these models need to be customized specifically for the labs. very hard to do when you're dealing with a closed source API. And with an open weight, that becomes much easier. And it's really one of the biggest advantages of open weight is the ability to customize and adapt the model for your use case. Yeah. Do you think that the government should be involved in regulating AI safety or should this come from an independent review body?

43:56I think it should be independent. I think, you know... Not the government at all. Yeah, and I think that it, you know, maybe there could be some kind of, you know, input, but I think from a, you know, a regulation standpoint, I don't know if it's, I don't, you know, believe in regulation at the government level of models. I believe firmly in what I said previously that, you know, open AI anthropics should be able to regulate themselves essentially and be, you know, do the best thing for the world as they release models into the hands of everybody. Um, so taking that on themselves, I think is, is the most appropriate action, um, and ensuring, uh, you know, safety is, is first and foremost a priority.

44:38Right. Let me ask you one last question about, uh, open weight specifically. So, you know, we've had a number of enterprises on the show who are training their own open weight models and, you know, I've sort of been trying to figure out what friction still exists for these enterprises to post-train their own models. Can you walk us through a little bit of sort of the challenges with that? Because that's what companies are doing. I mean, it's very clear that they want their own stuff. But how hard is that to do right now? And what technical challenges still need to be overcome to make that process cheaper or easier for any of these businesses to do it?

45:18Yeah, it's hard to post-train models, to have them specifically trained for you, for your workflow, your agent. It's not simple. It's not like click a button at this point. You know, there's a lot of nuances to it. You know, what data do you have? How is that data, you know, readily available? What format is that data in? I think it's becoming easier. I think that, you know, a lot of businesses, enterprise, you know, government are taking that ability to post train and customize and do reinforcement learning on their models and actually just kind of, you know, putting it into action. But it is still difficult.

45:56You do need. Like what's the rate limiting step here on this? I think it's talent, really. I think it's, you know, having the appropriate, you know, technical resources in-house to be able to take on that kind of workflow. And, you know, it's getting better. It's hard, though. Training models is hard. Training models is very hard. It's almost, especially post-training. Post-training is very much like an art. So if you have people that are, you know, very talented inside, you know, to the research level, then you can absolutely do that. Or, I mean, there is plenty of companies out there that can assist with that, you know, post-training and reinforcement learning.

46:31So it's getting better. It's getting better, I think. Great. Well, Mark, I want to thank you for coming on. That is Mark McQuaid, co-founder and CEO of RCAI here on TI TV. The AI build out and soaring CapEx bills right now are very much a story of building data centers and buying chips as fast as possible. All of that is extraordinarily expensive. And our editor, Meredith Mazzilli, wrote our weekly finance column this week on how all of this funding is coming together. That is the subject of this week's editor's cut. I want to bring on Meredith for our conversation. Meredith, welcome to the show.

47:07It's great to have you back. Hey, Akash. Happy Friday. Happy Friday to you as well. I want to start with the name of your column, which, as you'll see on the screen, is why compute needs a big down payment. Why does it need a big down payment? What's the story here? Walk us through it. Sure. So kind of got the idea for this column because we were putting out a few different scoops over the past couple of weeks about startups that were having to go out and raise, you know, billion dollars for a small, you know, new startup, basically to get in line to reserve compute. And, you know, that plus the Fed rate hike or rate decision looming seemed like a good time to kind of visit this topic and basically zoom out.

47:56And, you know, I'll just start by saying it's a good reminder that so much of the AI infrastructure boom is being built with borrowed money. And there are lots of different ways that is happening. We could probably be here all day describing all the different ways. I mean, the headline is people are getting creative, right? There's all sorts of ways to do it. Yeah. But basically, a lot of what you're seeing is cloud providers borrowing to buy the GPUs that customers then use. Various parties borrowing to build data centers, add capacity against big customer commitments. So if you're a customer, somewhere along that chain, something is most likely being financed.

48:41the stuff that you're running your AI on is being financed somewhere. And that helps explain some of what we're seeing in the market when it comes to capacity and shortages. So getting in line right now increasingly means making a big financial commitment. So providers want to see long-term contracts. They want to see that you can actually pay your bills when they come due. too. And that's also, we're seeing a lot of requests that's translating to a lot of requests for big upfront payments. Because like I said, all this capacity is being financed. So you can think about it like if you put a bigger down payment on a house, your mortgage is going to be smaller, right?

49:25So same idea here. And there's a lot of nuance to this. And that's like a very, very oversimplification of the situation, but who the customer is also matters in a really big way. This was one of the factors in the column that it touched on that one of these cloud providers flagged that if we're borrowing GPUs, borrowing to pay for GPUs that Microsoft's using, we're paying roughly 6%. If it's a non-investment grade customer, more like 9%. So big difference there. So let's I want to unpack some of this here. You mentioned the borrowing costs going up. So is that just the Fed raising rates? I mean, is that the underlying lever here?

50:13Or I mean, is it is it competition for deals? It's it's terms, as you said. I mean, it's not just the Fed is the point, right? Right. And short answer is it's complicated, like much of the debt market. But the Fed and expectations around the Fed do set at the very simple level, like a baseline for where you're going to build off of to reach ultimately what kind of interest rate you're paying. So that, and also the expectation of that baseline going up has been flowing through other things. But then you have a lot of factors that come down to each specific deal. So again, who is the ultimate customer of this?

50:57How can they pay? What about the KOTU Maddox deal? This was an interesting one that we had reported on previously. Just remind us about what that deal was and also what that says about this topic that we're talking about. Yeah. So that one is super interesting and kind of gets back to the bigger thing of what's happening. And basically, to sum up what I say before, AI borrowing costs generally aren't going up. But you also see some of these really increasingly creative deals coming out that essentially help a startup in a very convoluted way get access or get in line to things like compute capacity, chip components, manufacturing capacity, hopefully in a similar way that some of the big established players do.

51:48Um, so with the, the discussed deal with CO2 and MedEx, um, we don't know a ton of detail at this point. It sounds like a lot is still being hammered out, but based on other things that CO2 has done, what it probably looks like is, or what it could look like is, you know, there's this joint venture and they've done this before this joint venture. It then goes out and raises a lot of debt itself. Um, so it's a, it's a separate joint venture. In the case that Kotu's done this before, sometimes you have a backstopper come in like Google kind of helping. Being a guarantor. As a guarantor, essentially, it gets the lenders a little happier with, you know, given everybody else is pretty unproven there.

52:33So, you know, that's similar to what is happening on the GPU access side, but, you know, even further up the supply chain. Right. And Meredith, all this debt, is the main risk just demand at the end of the day for all this compute that is trying to be built up? Are there other risks that we should be aware of? Oh, there's so many risks. But a lot of it does come from the demand or things that flow from the demand. So you have the customer credit risk. Will the customer be able to pay? Not just do they really want this stuff. And also, as financing costs are going up, the provider is either going to have to take a lower return on the infrastructure they're offering or pass on costs themselves.

53:22So what does demand look like in the future? But then you also have all sorts of variables like construction risk. When is this thing actually going to be done in the case of data centers? Right. Power. and then a whole other bucket is the GPUs and okay, at the end of a contract, what are they really going to be worth? Which is a huge open question right now. I'd say a lot of debate around that. But a lot of it does come back to ultimately demand. But just to, I mean, just a question here on demand. So, you know, I'm thinking about the supply and demand dynamics here of the compute market and let's just take neoclouds, for example.

54:04I mean, there's been a whole discussion that right now demand is much higher than supply, and it means the neoclouds, they have a lot of pricing power. I mean, you could effectively charge whatever you want. When those dynamics sort of equalize, then pricing may not be as high. But what I'm trying to figure out here is if the borrowing costs go up the way that they are and those get passed on to the customer, how do you think about pricing here? Because I'm basically trying to figure out if pricing remains high, because you could make the argument that, well, prices will come down as the neoclods have less power, but maybe the borrowing cost keeps them high.

54:49What do you think of all this? So also complicated, but definitely the neoclods are talking a lot about new contracts they're signing at high prices right now, and especially short-term contracts. I think even Corwee was reiterating that yesterday and announcing a new convert deal. But I think the important thing to remember is when they're talking about new contracts and new pricing, that doesn't change all the huge amounts of existing contracts that they have. And it also doesn't guarantee, as those long-term contracts are rolling off in the future, what kind of prices they're going to get. But what they're talking about is very specifically right now, we can sell three to six months at some crazy price.

55:34And part of the reason they're probably able to do that is you do have these companies that can't or won't or don't want to do these longer term contracts. They need stuff right now. And it doesn't necessarily mean their whole book of contracts is going to suddenly one day reprice. Right, right, right. Well, Meredith, I want to thank you for coming on. That is Meredith Mazzilli, our senior editor here at The Information. That does it for today's show. A reminder, we are on this stream Monday through Friday at 10 a.m. Pacific, 1 p.m. Eastern. If you cannot make it then, episodes are available on theinformation.com, on our YouTube channel, or wherever you get your podcasts.

56:15Make sure to follow us on social media, on X, on Instagram, on TikTok, and on LinkedIn. I'm already excited for our next show tomorrow. Have a great rest of your Friday. Have a good weekend. Bye-bye for now.