U.S. AI Execs Warn as China’s AI Push Grows, Substack Adds AI Detection Tool | Diet TBPN

0:01John Coogan:China, China, China, China is in the news. Top American execs are sounding alarms. The White House is divided on how to respond to recent advances in Chinese AI, has weighed crackdown measures. We talked about how this was coming. It is interesting. There's a lot of like hurry up and pedal to the metal and then pull back and then ban the American model and then are we going to ban the Chinese model in the similar way? and lots of contradictions, lots of insider baseball. Everyone's critiquing everyone else. Lots of interesting takes. Also just some interesting data about what's actually going on with these models.

0:40John Coogan:How advanced are they? Where do they come from? How cheap are they to make? What are the economic impacts in America? But the Wall Street Journal has a nice little roundup here. We can take you through. So Silicon Valley and Washington are debating a multi-billion dollar question. Should American companies be able to use Chinese artificial intelligence models. They certainly have been able to. It hasn't been a problem for years, but now they're getting stronger. OpenAI and Anthropic executives are sounding the alarm about the rise of cheap AI, particularly powerful new models produced in China, suggesting that they will lead to, quote, a dystopian AI future and present unacceptable security risks without regulation.

1:20John Coogan:And that is the point of debate. Are you on the side of dystopian open source or liberating open source, the George Hatz frame. Some analysts who study the AI industry say the two companies which are preparing for public listings in the next year, they just want to eliminate the competition. The emergence of highly capable open autonomous AI systems, including Moonshot AI's Kimi K3 model and Alibaba's Qen 3.8 Max. And Alibaba is actually a massive investor in Moonshot. They have their own anthropic Google over there going on. Alibaba is like very much the Google, too, in the sense that they have a large position there, which were released in recent days and viewed favorably by investors and users has turned the AI race on its head once again.

2:11John Coogan:Kimmy K3 was also competitive with U.S. models on some benchmarks. The debate over the new models, which are, quote, open weight, but not yet. They're about to open source them. We'll see what happens next week. Allowing users to download and customize them with company data for specific tasks. Very, very good. Very useful. Coincides with division in the Trump administration about whether to take steps to limit the use of models in the United States. Quote, one probable outcome of an open weight model dominant world is full AI communism, which is precisely what China proposes rather than a market product, AI is a public good, which will ultimately be provided by the state as a kind of digital public infrastructure, Dean Ball, OpenAI's head of strategic futures, said in an ex-post Friday, Dean Ball made it to the pages of the Wall Street Journal.

2:54John Coogan:Now, the AI communism is just at the model level, like there would still be a bunch of, like a lot of people who are long compute, long neocloud, they say like, this is where the value You should accrue. The data centers will accrue a reasonable margin, and they will still continue to make money. And so it's not that this automatically leads to nationalization of compute in any way. People would still be paid, but the margins might be lower, is at least the formulation here. So, Ball, a former Trump administration official, called such a scenario a dystopian hellscape. and he said he expected the Trump administration to make moves toward reducing the use of open source models.

3:38John Coogan:Ball later clarified that he wasn't advocating for the United States government to discourage Chinese AI. He also highlighted a central challenge with the AI race. Top tier AI companies raise billions of dollars to pay for the vast computing resources needed to continue improving AI systems. If everyone uses AI systems that people largely don't pay for, there would be no way to finance continued frontier AI development. Ball, who started at the company earlier this month and focuses on policy and strategy, and OpenAI have said his comments don't reflect the company's official positions. He's just posting.

4:07John Coogan:The company has developed its own models, own open models, and Chief Executive Sam Altman had previously said that OpenAI's past approach of only developing closed tools was unwise. Other top AI executives have warned that open models present huge risks since developers and policymakers lack control over how they are used and modified. A lot of risk of cyber attacks, a lot of risk of annoying spam attacks. The cyber thing is very concerning and annoying if that really ramps up. But the good news is that the closed source models have been available to cybersecurity firms and a lot of big major companies.

4:42John Coogan:The finance industry, for example, has had access to Mythos and GPT Cyber for almost six months. And then just today, Google announced a specific model, Flashlight Cyber, I think. So they're trying to offer a similar capability, but really go down the cost curve because Because a lot of the cyber tools, there was incredible breakthroughs with Mythos, GPT Cyber, like great benchmarks on those, but could be very expensive. We heard, I think, wasn't it Nikesh Aurora said, I think he spent like a million dollars to find some zero-day bug and it was like totally worth it. But that's expensive for smaller companies that might just need, it would be great to just be able to go to a cheap cyber-tuned model, get approved by Google and say, hey, just go and read every line of code and just let me know if there's anything that's obvious.

5:27John Coogan:And that could be good. Yeah, and just to be clear, they released 3.5 Flashlight and 3.5 Flash Cyber. So it's not like Flashlight Cyber. Okay, but Flash is still cheaper than the big model, right? Yeah, there's like the Pro and then there's the Pro. And so I have to imagine that if you, given the choice, if you're like a mid-sized company and your choice is like, go beat down the door to get into Project Glasswing to spend$5 million to have Mythos read every line of code. or you're like, oh, well, I'm already a Google partner. I have a really solid business relationship with them because I've been buying ads and I have Google Webmaster tools set up and I just want to go set up Flash Cyber, not Flashlight Cyber.

6:09John Coogan:I imagine that the cost would be cheaper because they fine-tuned this model for this specific thing. So Anthropics CEO Dario Amadei for years has warned about the risks of powerful and open AI systems, saying in a recent Bloomberg interview that having AI models with advanced cybersecurity capabilities that are free to download could be harmful. He said it's a serious concern. Use of Chinese models, which are far cheaper than U.S. counterparts, is surging at U.S. companies, prompting some investors to question the staying power of top model makers, such as Anthropic and OpenAC. The current U.S.

6:40John Coogan:open source frontier is starting to catch up with China's. On Wednesday, Thinking Machines Lab, led by former AI technology chief Miramirati, released its first AI model as open weight. Nvidia's Nemo Tron 3 Ultra is starting to see traction. That's interesting to hear. I haven't heard nearly enough buzz around Nemo Tron. I know Jensen has been investing in it. He's also investing in a lot of up and down the stack. I've been hearing buzz about the name, John. Nemo Tron's a good name. Nemo Tron. Why? Very cool name. People are, you know, there's some talks bubbling. People are saying it's one of the best names for a family of models ever.

7:18John Coogan:What comes after Ultra? Isn't that what the, didn't Apple first do that with the M4 Pro, the M4 Max, the M4 Ultra? So we got to come up with an even better word for what happens after Ultra. Giga, maybe. Hyper. And then Exa. The market's faith in Anthropic and OpenAI continuing to build more capable models that push the frontier, AI frontier has been at the heart of the boom, helping justify trillions of dollars in spending on infrastructure in the coming years. The threat that new players will vastly undercut what they can charge for advanced AI pushed down some tech and AI company stocks prices last week.

7:52John Coogan:David Sachs, the venture capitalist and White House AI advisor, suggested that he read Ball's post as a confession to a regulatory capture strategy. Sachs and others have long seen calls for AI regulation by companies such as Anthropica's efforts to use new laws and policies to stymie competitors. The weaponization of regulatory uncertainty as a competitive tool should be completely unacceptable, he said. Security-focused administration officials in the past year have considered a slew of measures to crack down on Chinese AI companies developing open models. People familiar with the discussion said they've weighed adding the businesses to trade blacklists, security warnings on Chinese AI companies, and a potential executive order targeting open models.

8:33John Coogan:So it does seem like even though Tyler Cowen, we were going back and forth with him about this, about like, can you even ban it? You can't really ban anything in the land of the free. Like the solution to piracy was not really to ban piracy. Like you can still torrent things, but it's just so easy to use Apple TV and Spotify and Netflix that most people just wind up just paying$5,$2 for a song or$10 a month for something. that it effectively killed it just because the product experience was better. This morning, our Treasury Secretary, Besant, was on Fox and said, this administration supports open source models, but what we do not support is IP theft.

9:18If we see, especially that overseas models are stealing from our great companies, we have the ability to sanction them because of this theft. So it will be if the admin does take serious action here, I would expect that to be the line of reasoning where they could just sanction the companies behind these models, which I imagine would create a real opportunity for American open source. You would ultimately see a lot of these, you know, neoclouds unable, I imagine, to serve the traditional Chinese open source models. I imagine that even OpenRouter would probably have to stop supporting that.

10:00John Coogan:But again, with an open source model, how do you even define it? Because if there's a research paper and then you go and implement that and you train it or you take it, you fine tune it, or you use the data to bootstrap your own model, just like where in the supply chain, how many steps away do you need to be to not count on there? Didn't Perplexity do like a 1776 DeepSeek very quickly? They did a fine tune. I think you did a fine tune on DeepSeek as well. Yeah, it was like using sparse autoencoders. Okay. I did it first, so. But, and what nationality was the model that you used as the base? Well, it was DeepSeek.

10:36It was Chinese. Wow, confession.

10:39John Coogan:You can go now. Yeah, people are already angry at Besson's comments and are saying, Funny that he's not mentioning that yesterday a U.S. judge approved Anthropics'$1.5 billion copyright lawsuit settlement. I think they settled that they were in possession of books that had been torrented or basically acquired unlawfully. I don't believe that— Acquired unlawfully? Sounds like they stole the book. I think they bought the books. No, no, no, no, no. Wait, what? No, separately they bought a bunch of books, trained on them, and destroyed them. And that just made people upset because no one was the idea of books being destroyed, of course.

11:25I think those were rare books.

11:28John Coogan:The Wilmanitis collection was destroyed. Rare PDFs. Judge William Alsup of the U.S. District Court for the Northern District of California issued a preliminary approval of the settlement last year after ruling that Anthropic had illegally downloaded and stored millions of copyrighted books. and the settlement was signed off on by a different judge yesterday. The payout will deliver$3 ,000 per work across an estimated 500 ,000 works shared among the authors and publishers who hold rights to them. Anyway, there was some news about whether or not Kimi K3 is distilled. Tyler, can you take us through this Lisan Al-Ghaib post?

12:08Yeah, so he didn't actually make this chart. It was from someone else. But basically, this show is a bunch of models compared to another, and it's like the kind of semantic similarities between them. So it's like if you use similar kind of diction or phrases, stuff like this, you just like prompt the models a bunch of times and you kind of can tell what's similar to each other. And you can see in this version, it looks like Kimi-Ki 3 is very similar to Fable 5.

12:31John Coogan:Fable 5 and then also Opus 4.7, Opus 4.8 and Sonnet 5. So they might have gotten tokens from like all four of those models sort of mixed them together. Also, I think this exact example, like only like a few prompts were used. It's not like super involved. It's also like this is very much just like correlation. This is not like proving that they just don't learn anything. They could have just used like similar, you know, post-training techniques or something like this. Totally. But notably, GPT 5.4, 5.5, 5.6 share very little similarity to the Anthropic models and vice versa. The Anthropic models don't seem to be correlated with the OpenAI models, although the open ai models are correlated with them with each other so uh 5.6 luna looks pretty correlated with 5.4 because they probably were like well we're building on the shoulders of the work that we've been done and and it's the same thing with anthropic where fable has some of the same flavor as an opus or a haiku and that makes sense that makes sense because people have like been saying oh maybe those models were actually distilled from the big like mythos pre-trained Yeah, or vice versa.

13:38John Coogan:Or vice versa. You have a bunch of users that are doing rollouts on Opus, trying to vibe code different things, and you take all that training data and then you feed that into Fable. And Fable sort of like picks up on a lot of that because that's what the user said. Wow, you really did a great job on this Opus. Or you did really poorly. It pulls that out. But it's still learning the textual similarity. Well, people will go back and forth on this and debate the level of distillation and also what the punishment for distillation should be. Should it just be a slap on the wrist or should it be something bigger?

14:13The only thing I would say here is like distillation against the terms of service.

14:19John Coogan:Yeah. And I think that suing Chinese companies, it would be a complete and total waste of time. Ask anyone who's built a product that's been knocked. I disagree. Incredible aura farm. If you personally serve Xi Jinping the next time he's in San Francisco meeting with a world leader, you show up to the UN. Serve the world leader. There's something about that that's extremely bold. But no, I agree with you. Very difficult. I don't even know if you went to your law firm and you said like, hey, we want to sue this Chinese company, like how they would trace that down. Jim Cramer says we must not let our companies use these Chinese models to save a few bucks.

15:05John Coogan:OpenAI and Anthropic are correct. This is vital national security. Please, please read Bing West's just released Catch 5. I respect the Chinese people greatly, but these companies are run by the PLA for heaven's sakes. George Hots fires back under the tiny grad handle. He says, is this a joke? I'd be happy to use Anthropic if they put their weights on hugging face like the Chinese. American tech companies are rent extraction machines that have lost the plot. No point continuing to defend them. They need to change if they want to win people back. Meanwhile, China is also thinking about export controls in the Financial Times.

15:41Apparently, Alibaba, ByteDance, and Zipu. Keep going back and forth. Moffcom. Moffcom. Talk to AI companies, including Alibaba, ByteDance, and Zipu, on limiting the transfer of key data for the training of their models overseas, as well as allowing their model weights to be downloaded by foreign users.

15:57John Coogan:Probably their Department of Commerce. That seems more like it. China would still let overseas customers access the models and services, but it sounds like they're leaning towards not making the weights open. We're going to see with the new Kimi K3 if they actually release the weights next week. Dean Ball says, Wow, two days in a row where things I said are likely to happen seem likelier to happen. from two adversarial countries, almost as though my analysis was right and as though people should not have deliberately misinterpreted me as of making prescriptions. Dean Ball, victory lap. You got to take a victory lap for Dean Ball.

16:35John Coogan:He called it. No, I mean, he was saying, like, I don't necessarily agree with this, but just from his experience, this is how things might play out. And people were very upset with him about that. Raj in the chat says, Ministry of Commerce, People's Republic of China. It's their Department of Commerce. That would go hard on a business card. It would. The great slop wars continue. Let's pull up this video from Substack. They say, today, Substack is launching an AI detection feature via an integration with Pangram. Going forward, you'll be able to scan posts, replies, and comments on the Substack app to see an estimate of how much of it was written by a human or with AI assistance.

17:13We care about this at Substack because it gets to the core of our mission to build an economic engine for culture.

17:19John Coogan:This is going to be bad for Tyler since he's mastered the ability to write exactly like an AI. What was that sentence you wrote by hand that was like, according to studies, you're absolutely right. And you just threw in every possible AI-ism and got flagged 100 % by Pangram. No, I think people will enjoy this. The revealed preference is probably that people don't care if it's good and they enjoy the content. But we'll go back and forth and debate it with Chris Best today and see where all of that is going. Now, do you think that Substack writers should admit when they hire essentially ghostwriters?

18:02John Coogan:Probably. Because sometimes you start a brand under a name. Like not every post on Bloomberg.com is from Michael Bloomberg. In fact, none of them are. They're all from other people, but they disclose this is from some writer. This is from Joe Weisenthal, right? But some new influencer brands have grown, even in the pre-AI era. I had a writer on my team when I was doing YouTube videos where I would write something. He would do a draft. We'd try to go back and forth. I'd put him in the credits, but I wouldn't kick it off with like, hey, just so you know. The chat says book authors don't. That is certainly true.

18:42That's 100 % true.

18:43John Coogan:Some book authors will put like this is written by this person and this person. It depends on like the aura of the ghostwriter. Some ghostwriters actually do get credits in the book. But yes, most book authors. Like Tom Clancy I think passed away and is still producing books. There's a number of writers where their individual brand grew so big that you can have like a new Tom Clancy. They become a business. Yeah, they become their own brand. Yeah, I would imagine that 30 % of the books in like an airport bookstore are just 100 % ghostwritten and probably don't disclose it. I do think it should be possible to get to the truth if you hunt hard enough, but I don't necessarily need you to put it front and center.

19:27For a Substack author or creator, just put it somewhere that you have a team of people that help you create your work.

19:35John Coogan:Well, Dave Eggers took to the Financial Times saying that no one is going to read AI books. He's, of course, the author of The Circle. He's an American writer, editor, publisher. Heartbreaking work of staggering genius is his 2000 memoir, bestseller, finalist for the Pulitzer Prize. He says, gullible and ridiculous. No one will read AI novels. I think people are already reading AI novels. So maybe he means like no one who's cool or no one who he likes or something. Let's read through a little bit of this Financial Times article. Dave Eggers offers a hand so I can clamber aboard his 23-foot sailboat moored in the shadow of San Francisco's Golden Gate Bridge.

20:20John Coogan:It's a grand setting with few frills as we sit on the stern a meter apart, fuel cans and fishing nets at our feet. There's no one around to disturb our conversation. The nearest toilets are port-a-lose, a short bike ride away. The birthing fees are negligible, making it one of the cheapest places to live in uber-wealthy Silicon Valley. I hadn't thought of that. Sailboat, if you're at a tech company, this has got to be the next big lock-in trend. Antonio Garcia Martinez did this. He did. He talked about it in his book, Chaos Monkeys. I won't be satisfied until I see LARPers posting images on Twitter being like, hey, I just moved to San Francisco.

20:56John Coogan:I'm building the next one person$25 billion company. I'm finding a houseboat in San Rafael. Yeah? How much? For$1 ,000. It actually works out to$1 ,483 a month, apparently, if you finance it through BoatTrader. Goodbye, Tyler. But it's only about$200 ,000 to live in a 2022 houseboat. 2022? That's a brand new houseboat. Yes, you have to think about the slip fees. But you could easily come in, have your own house on the water for a few grand a month. The average one-bedroom rent in San Francisco is like two grand, three grand right now. Yeah, you could have a whole house and a boat. And a boat. It's amazing.

21:36John Coogan:This is the hack. If you get a job in tech and you're printing, making six figures, move into a houseboat. Let's pull this up. Is this an option in LA? Should members of the team be doing this? Should we recommend this to guys? Look at this brand, almost brand new. You have a 2024 La Mer Modern 11. It's a 36-foot houseboat. We have a video here. Well, Dave Eggers says they call it a$10 dock with a million-dollar view. Look at this, John. This? Yeah. A couple thousand bucks a month. This is where the next great startup will be born. Yeah. For sure. Garages are out. Garages are completely out. And also, like, oh, take your shoes off.

22:17John Coogan:It's natural to take your shoes off when you're on a boat. You know? Right? Unless you've got boat shoes. Wear boat shoes. The La Mer Modern 11 offers reliable performance and strong power with two Yamaha 25 high thrust engines providing a total of 50 horsepower. It's two cabins and one head ensure overnight comfort for extended stays. How about year long stays? How about a decade long stay? The boat's design includes an upper deck terrace with flexi teak flooring, enhancing its appeal for relaxation and entertainment. And might I say locking in standard features such as air conditioning. We have air conditioning on our houseboats with reverse cycle heat, heated floors, LED lighting.