Open Source vs. Closed Source, Memory Chips Eat AI Profits, Comcast Restructures | Diet TBPN

0:01John Coogan:Well, on the front of the Wall Street Journal today, this is how you know this is the whole AI 2027, Washington waking up. The AI stories are making it to the front page, the world news section, not just the business and finance section, more and more. So the picture is about the heat wave, but the lead, the story with the largest text is about artificial intelligence. China resets the AI race with the United States as security models mark gains. We're going to get into it. This is a fascinating debate because I thought that we'd have a conclusion to the open source AI debate by now. Either the frontier would have collapsed and there would be perfect commoditization, or they would have fallen so far behind.

0:49John Coogan:There will never be a conclusion. It'll just go, it's over. We're so back. It's over. If you're in open source AI, that's exactly how it feels. The big story is centering around GLM 5.2 from z.ai. It was officially released June 13th, so it's taken a couple weeks for it to really break through to the front page of the Wall Street Journal. But there's seen some strong performance on benchmarks, some positive reviews from developers. I have a whole review from Tyler we can go through in a little bit. But we're now entering another round of debates around open source AI. What can the model actually do?

1:23John Coogan:Is this a threat to national security? What are the geopolitical ramifications here? And so I'm sure this will be an ongoing conversation throughout this week, probably next week. We have some guests lined up to help contextualize it. But laying down the facts from the journal, security researchers said that a new AI model released this month by China's Zipu AI, also known as Z.ai, can match the latest US models when it comes to finding security bugs, a development poised to reset the global tech race and pressure the White House in its overhaul of US AI policy. So unlike models from Anthropic or OpenAI, Zipu's GLM 5.2 is open weight.

2:03John Coogan:You can just download it, run it anywhere. You don't need to go to an API. You don't need to go to a private company and pay them. You can run it on your own server, provided you have the electricity and GPUs to do so. It is expensive to run, as we'll go into, but it is open-weight. That means it can be downloaded, run on hardware, operated by anybody, and can be modified and used without supervision. Scary stuff. Open-weight models are ideal for users who want unfettered access to systems they control, but they're also ideal for hackers who want to run them in the shadows. Unfettered intelligence.

2:36John Coogan:Unfettered. Oh, that's a good... We were completely out of names for new Neolabs. That's a good Neolab name, yeah. Unfettered intelligence. Unfettered intelligence is good. GLM 5.2 has ranked as one of the top 10 most used AI models, according to data from Open Router, a company that provides access to more than 400 AI models. And what a fantastic business. Alex Atala over there, absolutely cooking at Open Router. It's such an exciting way to plug into the AI race without actually needing to play the benchmark game so much, be the front door. Anyway, in some benchmarking tests, according to cybersecurity company Semgrep, GLM 5.2 bested Anthropics Claude Opus 4.8 model, which was released in May.

3:23John Coogan:When given further instructions, Opus 4.8 and GLM 5.2 can match mythos in bug binding ability, according to researchers. So prior to this launch, and there's a chart that we should pull up here about overall AI capability. We can talk to Tyler about what this chart actually means. But there was this narrative brewing that open source AI was slowing down relative to the closed source frontier. And I saw a lot of American AI fans sort of cheer for this. Hey, we have the capital markets. We have the data centers. We have the researchers. And so we are able to push the frontier at a different rate.

4:01John Coogan:And if we're actually growing at a faster rate in America within the closed source labs, that will compound and there will be a stronger takeoff in the American closed source AI industry. Now, this chart sort of goes back and forth, and there's some debate over it. It's in the newsletter. You can go sign up at tbpn.com. While we're pulling that up, let me tell you about Codex. Codex is a powerful workspace for getting work done with AI agents. Whether you're writing code, analyzing data, creating content, or automating business workflows, Codex helps you move projects forward from start to finish.

4:34John Coogan:This chart, which we can pull up, shows progress from GPT-40 to 01, 03 mini, 03, Opus 4, GPT-5, 5.2, Opus 4.6, GPT-5.4, GPT-5.5, showing a linear trend in this ELO, which is a blend. It says GLM 5.2 sounds too much like a gray market peptide taking. It actually does. It does sound a lot like that. And then you can see the red line are the Chinese models, which are also improving over time, but at a slightly lower rate. And so the question was, are they going to plateau while America's progress continues to advance? this latest model, GLM 5.2, seems it's very hard to apply it to this particular benchmark because this ELO was, can you give us some background, Tyler, on where this chart came from, what this is demonstrating?

5:27Yeah, so this is by Casey. I think that's how you pronounce it. The Center for AI Standards and Innovation. They have this way to calculate the ELO model. It's basically approximation of a bunch of different benchmarks. Some of those are proprietary. They're not open, so it's actually hard to run these. Also, because I was basically trying to bench all the recent models since this was published. I think it was, I want to say May 1st.

5:47John Coogan:Yeah, it'd be great to throw 5.6. So mythos and fable. It would be great to just continue this chart because it's an interesting trend. So a lot of those benchmarks aren't actually public. So it's very hard to estimate. But I tried, I got, you can look at like some of the benchmarks that are public that you can reference. You can kind of match them up to previous models. 5.2 looks like it is like a big step up from the like Chinese trend line, right? I think the group of benchmarks that were chosen for this ELO definitely accentuate the gap between U.S. and Chinese labs. I think there's a bunch of other groups like Epoch AI has done a chart.

6:26They basically see a relatively stable gap between closed source and open source models since like 2023, like a long time.

6:32John Coogan:Yeah. And perhaps at this point, the discussion should be more centered around cost per task more than cost per token. Yes, definitely. Because even like, you know, new models, a lot of times when they come out, like, okay, maybe the token price is actually the exact same, but the token efficiency is much better. Then when you do a lot of these tasks, it's like, it's not the price per token, it's the price per, you know, something completed. And then you actually see it go down. And there's a lot of test time scaling laws where you can just throw a million dollars of compute at a particular problem and all the models do really well at it.

7:07John Coogan:But it's completely non-viable for any real enterprise use case and probably not even viable if you're trying to be a nefarious hacker. Yes. Most people are saying like 5.2 is very token hungry, right? So it uses a lot of tokens. So maybe it definitely is much cheaper than the Frontier models. On a per token basis. On the per token basis. But on the per task basis, it might be more expensive. Yeah. I mean, that's still, it's generally not. But on specific tasks, you can get, you know, if you have low thinking models, low thinking mode on the closed source ones, you can see. Well, let's revisit John Ludig's post from 2024, May 2024.

7:43John Coogan:This is pre-DeepSeek talking about his prediction about why the future of foundation models is closed source. He got a lot of pushback from this because a lot of people like open source models. But he laid out a thesis around closed source data, flywheels, exponential, capex, intensivity of training. And he said, open source will have a home wherever smaller, less capable and configurable models are needed. Enterprise workloads, for example. But the bulk of the value creation and capture in AI will happen using frontier capabilities. The impulse to release open source models makes sense as a free marketing strategy and as a path to commoditize your complements.

8:25John Coogan:But open source model providers will lose the capital expenditure war as open source ROI continues to decline. And that was the thesis around the time that the open source AI discussion was primarily driven by Mark Zuckerberg's work at Meta on the Lama family of models. The idea was that Meta would benefit from attracting talent. It was good marketing. It told the story that Meta has an AI story and has AI talent in-house. even if they weren't monetizing it and sharing, you know, a really fast takeoff in ARR around those models. It showed that, hey, they're able to develop these models and that might help them cut their costs in the long term.

9:05John Coogan:Very interesting that that wound up being very different in 2026, looking at the news today, which we'll go into about them spending a lot on Gemini. There's been reports about them spending a lot with other closed source frontier labs that they should have commoditized with their open source plan. But nonetheless, that was the idea with Meta. But then China sort of woke up and DeepSeek's launch at the start of 2025, and the game theory became way more complicated. So George Hatz sort of summed this up nicely. He has a take in AI will be massively deflationary, a post from just a few weeks ago, as to why China benefits from investing in open source more than American firms.

9:45John Coogan:He says, this explains why the Chinese are giving the much more moderate resources to train models away for free. They love to see deflationary economics in the U.S. It is much less of a service-based economy, and so if they can go and give away free tools that deflate the value of the service sector, that is an advantage to the Chinese economy in his formulation. He says, even if you don't regulatory capture of the US government. Nobody is getting a monopoly on AI. We don't live in a unipolar world anymore. And so he likens what's happening in DC to sort of rearranging deck chairs on the Titanic.

10:22John Coogan:It's a very fun, fun piece. So we're back to this discussion of what are the consequences and the impacts of open source models, particularly in the United States. And there's been this clip that's resurfacing from Dario Amadei when he was testifying in front of Congress in 2023. And it's now recirculating. And it was reposted like he just said it, and he did not. So be clear about that. This is from three years ago. But some of his predictions were very prescient as of where the frontier is today. So he said, I'm very concerned about where things are going. If we talk about two to three years for the frontier models for the bio risks, it's sort of a bad transcription of what he was saying.

11:04John Coogan:He's talking about 2025, 2026. Remember, he was saying this in 2023. We're there now. I think the path that things are going in terms of the scaling of the open source models, I think it's going down a very dangerous path. And again, if the path continues, I think we could get to a very dangerous place. So he was worried about cybersecurity and bio risks being open sourced and then not having a counterweight to that. Now, the good news is that we've talked to the CEOs of cybersecurity firms like CrowdStrike and Palo Alto networks and they've been working with Mythos and GPT 5.5 cyber for months now to harden systems from LLM driven attacks.

11:42John Coogan:And so there's still this gap between closed source and open source models and that gap allows white hat hackers to implement fixes before black hat hackers have a chance to exploit easy bugs. There still will be a bigger discussion here though in DC over the next few months as the frontier models roll out and the gap doesn't appear to be widening at the moment, so security stances must adjust. It's not a closed source is falling behind, so it's never going to be an issue. There will be this gap, and how the American cybersecurity industry and eventually the biosecurity industry implements changes and fixes before open source catches up or commoditizes and makes that particular capability widely available is going to continue to be important.

12:25John Coogan:So let's go over to Tyler's quick review of GLM 5.2. Why don't you take me through your bullet points and you can tell us like what is the shape of this model how are the reviews yeah so I think so far one of the main things is like people are saying it's oh it's distilled right this is this been a big thing with all these open source models especially the Chinese ones oh the only reason that they're good is because they're distilled it's very hard to actually figure out how true this is it certainly seems like that there's some you know I think it's a fanthropic models didn't anthropic openly accuse Alibaba of distilling.

13:00John Coogan:Yes, a number of these labs. Yeah, and there's also been a big professionalization of the gray market where a whole bunch of different sort of individual groups will connect a whole bunch of different entities and user accounts and subscriptions and APIs to then create a front end to the model that can be served at a very high rate through a VPN, most likely. What's interesting is that you would think that if you were going to do a training run, you would just find and replace on the other lab's name before you hit run. Is that not something people can do? I don't understand. Yeah, I mean, it also depends on what you're actually, like, maybe you're not directly distilling on the API, but, you know, you're training on, you know, public GitHub, you know, repos, and those were all used, those were all, you know, made with resource models.

13:53So then you're kind of like distilling, but it's not really like, is this really kind of distilling? I don't know. Yeah. So if you're convinced that these are super distilled, the only reason that they're good is because they're just basically taking the closed sourced labs.

14:06John Coogan:There's also this weird thing with distilling where as more and more of the public internet and GitHub broadly and open source repos become LLM outputs, if you train on that, you are in some ways distilling because an LLM has a quirk like it's not this, it's that in text, and you wind up training on a whole bunch of Amazon Kindle books. You're going to wind up learning it's not this, it's that. And the same thing applies for different code conventions in open source repos that have effectively been completely been rewritten by closed source models. Yeah, and so I think it's safe to say that we've generally seen that distilled models generally will generalize worse, right?

14:48So you'll see really good benchmark scores. Maybe they're benchmarks, maybe they're not. But even if they're not directly benchmarks, you still find that they generally...

14:55John Coogan:Yeah, they're kind of accidentally benchmarking. Yeah. So I think initially you should just be a little bit suspicious of these super high benchmark scores. Yeah. But they lack that big model je ne sais quoi. Yeah, and this is anecdotally reinforced. A bunch of people have been saying, for coding, these models are really great. GLM, it's a very good model for creative writing. or something like this, where you'd imagine it's a bit harder to kind of bench max this. I wonder, have people been testing it with the Tiananmen Square bench? Does it reject that stuff? Because it felt like that was something that was widely misunderstood by American audiences, that in fact, that might not be the biggest deal for the CCP anymore.

15:36Yeah, also, I think even if that's true, the model is open source. You can kind of just fine tune it to not that. Maybe it's a bit harder than that. But I think you can kind of get around like that kind of stuff.

15:45John Coogan:Okay. Yeah. So we talked about the token hunger and the API price. And in general, I mean, you said, I'm not convinced that there's a big market for this class of model, especially as frontier models get more efficient. If you look at OpenRouter, the most used models are the smallest open source models, presumably being used for specific tasks that need to be repeated over and over again. Yes, I think like what we've seen is, you know, a marginal IQ point of the models is like extremely expensive. Frontier models are getting very expensive. People have to cut back. They're token maxing. This is like a massive bill on their balance sheet or whatever.

16:21It seems like there's now basically like two classes of models that people really use. There's like the frontier ones. And they're using coding agents. They need the best thing. If you're doing cyber, like you just need the best model because, you know, the risk of someone hacking you, it's so great. You just need the best thing. You pay whatever it is. And then there's the second class, which is like these very small, very fast, very cheap models that you can use for these kind of point solution things. Maybe you have some orchestration where you're using a really big model to have these little agents using these very cheap models.

16:52I think in the middle, it's hard to actually figure out what is the real use case. Maybe it's like hobbyists using these coding agents and they don't want to pay the super expensive tokens of the closed source labs. You see this on OpenRouter where like what are the top models by token like usage? It's these very small models. It's like DeepSeek Flash.

17:12John Coogan:Yeah, because you're spamming them for like every receipt that goes into RAMP gets processed by an LLM at this point. Does it need to be a frontier model telling me that I spent$10 on a coffee? No. It can just do standard OCR. That would be my preference. Yeah, you want super intelligence overseeing your expenses most likely. But no, you use the right tool for the job, and that's clearly what's happening on it. Yeah, but also I think like it is a very good model, right? Like we should not fully dismiss. I think the idea that, oh, the gap is widening. We really don't have to worry about these models.

17:47I think they are like very good. Yeah, yeah, yeah. And maybe if you're super worried about distillation, maybe something changes if the models are, you know, kept to these big partners, right? Like what we've seen recently with government coming in. But I think we can't really fully dismiss these labs.

18:02John Coogan:Yeah, it throws a little bit of a wrench in the monetization potential, like how long can you monetize a new frontier model? That's more tricky. And then the other one is just like if you're going to keep a model behind KYC or behind an approval for specific companies like the government has been sort of edging towards and moving towards, it gets a little bit tricky if all of a sudden you just wait three months and oh I was waiting to get approved for this one for like GPT-7 or whatever but by the time I the government got back to me my company got access to GLM-6 and it's close enough and so that in that just throws another wrench that I think the government will have to figure out how it puzzles together with the rest of the strategy, which has been back and forth as always.

19:00Google caps Meta's Gemini use as AI demand strains capacity in the financial time. Surging appetite for advanced models is turning computing power into the tech industry's scarcest commodity. And they have a picture here of a Google Gemini bicycle, which looks fantastic.

19:20John Coogan:What does that have to do with Meta? I think that was just the best. Google Gemini picture. Google has put limits on Meta's use of its Gemini AI models after the social media giant sought more computing capacity than the rival tech group could provide in the latest evidence of the infrastructure constraints facing even the world's largest AI providers. Google told Meta around March that it could not provide all the Gemini capacity the company wanted to purchase, according to three people familiar with the matter, in a move that has disrupted and delayed some of Meta's internal AI projects. So I don't understand how this is possible.

19:56John Coogan:Yeah, yeah. So one. Google spent$200 billion on CapEx. Okay, so of course, like around this time, token maxing was becoming a thing. A lot of every company in the world, at least every tech company in the world, kind of going a little bit crazy from a spending standpoint. You know, I could see Meta going and like wanting to basically buy a bunch of capacity and then being told like, hey, we can't fulfill that. But I'm wondering, is it worth reading? I mean, it sounds extremely bullish for Google. Like, if they're actually out of the past, like that's insane. And this tracks with what they talk about on earnings calls.

20:31Yeah, yeah.

20:32John Coogan:Yeah, Google Cloud, acceleration. But you do have to wonder, like, could distillation be part of this story? Could that be a factor here? I have no idea. I don't know. Zero Hedge said, meta puts limits on Claude and Kodak, spearing distillation, the information. But so this story is different. This is Meta telling its own employees, don't use Claude and Codex in certain parts and certain parts of our business because we don't want to accidentally do distillation, is what Meta is saying. So that's different. I was wondering, like, is Google thinking like, whoa, that's a lot of, you know, cool it, you know?

21:12Owing to the restrictions which remain in place as well as a broader push to streamline AI costs, Meta has encouraged staff to be more efficient with AI tokens. Several other Google clients have been affected by the restrictions, although to a lesser extent, Meta has been particularly impacted because of its exceptionally high demand for Google's models.

21:30John Coogan:Interesting. Very interesting. On the topic of Meta, Meta shared this morning a new milestone. It is a mind reader. Mind reader? Non-invasive brain-to-text decoder research. Brain to QWERTY V2, building on V1, which was published today in Nature Brain 2 QWERTY V2, is the highest performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication. So if you thought Instagram was listening to you, if you thought it was listening to your conversations, now you can have a new conspiracy at home, which is that they might be just listening to your thoughts.