Meta Tokenmaxxing, Intel Joins Terafab, Frontier AI vs. China | Diet TBPN

0:00John Coogan:Well, there is a whole bunch of news to run through. The first story is that Meta employees are apparently token maxing and competing on an internal leaderboard called Claudeonomics for status as a token legend. This is from the information. Over a recent 30-day period, total usage on the dashboard topped 60 trillion tokens, and this sparked a huge debate over how much is Meta actually spending with Anthropic. of course the other big news is that Anthropic just passed 30 billion in run rate revenue with one of the probably the steepest revenue growth chart in human history absolutely legendary yeah this this this uh you know chasing status as a token legend reminds me of kind of maybe it was a year ago at this point you were saying like will tokens ever become like eyeballs the way eyeballs were during the yeah during dot-com era yeah right just optimized for for eyeballs obviously not every eyeball visit to a website is created equally yep but people were optimizing for uh eyeballs and now uh you know i don't the reaction to this i think has been generally at least online like been uh i guess reassuring a lot of people are saying uh gary basin says uh you why marty says good heart's law when a measure becomes a target it ceases to be a good measure and so who knows what's actually going on internally but we do know zuck is pushing the entire company to be as a ai native as possible and this guy loves spending money too right i i have a crazy bull case here that i will run through let let's get through some of the story um first uh we got to pull up this uh this comic from xkcd uh in the comments here when a metric becomes a target it ceases to be a good metric it's uh right under the the leading post there we go and says uh and the the other counterparty says sounds bad let's offer a bonus to anyone who identifies a metric that has become a target uh it is good uh i don't think that's going on here lighter was texting a friend at meta and uh sent the the post we just discussed on token maxing and said true, and the person said yes.

2:15It's pretty sad. But I mean, imagine, so meta has been, there's been rumors of meta layoffs for a while now. Sure. Unclear how many, if any, if any have happened. But if you're sitting there, Zuck is saying like, we need to get AI native. Boss is saying we need to get AI native. And then suddenly there's a token leaderboard. You do not want to be at the bottom of the list. I will say that, right? Yeah. You know, you don't want to be the you don't want to be the guy who's having to explain like, no, well, I've actually getting the most out of each incremental token. And the other guy is just like set up an agent that just counts one, one, two million over and over and over or something.

2:55John Coogan:Yeah. Yeah. I mean, you have to measure the actual output, the impact on the business. I mean, fortunately, Meta has been a huge beneficiary and a huge winner of AI. The ads are getting better targeting. They're seeing they're delivering more ads and and the quarterly earnings have been strong. The headline number here that sort of took everyone by surprise is that Meta staff used 60.2 trillion tokens over 30 days, which would pencil out to about a third of Anthropics ARR was the number that was thrown out But both of these claims are pretty questionable And so Tyler did some back-of-the-envelope math to show that the one-third revenue estimate is way way too high And I don't know do you want to take us through some of the reasoning there?

3:35John Coogan:And then we can talk about the knock-on effects of all this. Yeah. Okay. So 60.2 trillion tokens is the number. We can just assume that's true. So basically, I'm going to assume all the employees are basically just using Opus 4.6. Yeah. So then there's basically three numbers you need to look for in the API cost. So there's input, there's cached input, and then there's output. So for Opus 4.6, it's$5 per million tokens on input. It's 50 cents per million tokens on input, cached, and then it's$25 on output. Yeah. So if you multiply that 60.2 trillion tokens at the highest possible rate,$25 per million tokens, then you do get like a billion dollars.

4:16John Coogan:Yes. Which is crazy. That's not what's happening. Crazy number. But you have to think about it like, you know, if you're using like cloud code or any of these coding agents, you know, the vast, vast majority of the tokens used is input. Yeah. Because like, so imagine you're working on some, you know, coding file, right? Yeah, there's a thousand lines of code in the file. Maybe the model is only changing like 10 at most, right? So that's very small percentage. So the output tokens are gonna be a very small percentage of the total tokens going in, right? Open router publishes like a lot of this data So you can kind of use those ratios to figure out what is actually like what are the actual numbers of the You know input versus cash versus output.

4:52Yep

4:52John Coogan:So just to get sort of like market standard averages. Yeah baseline benchmark Yeah. Now, Meta could be using these tools differently. But if we are to assume that the shape of their agentic coding efforts are similar to the average, this is what the numbers might look like. So maybe there is like some bad incentive where people are just saying to the model, like count up to a billion and then do it again. Yeah. So then it's like totally skewed. But if they're doing it relatively normally. Yeah. So on Open Rider, it's about 98.9 percent of all tokens are input input. And that's including the hashed ones.

5:24John Coogan:Yeah, because you're stuffing the context window with all your code base or a huge amount of context. Yeah. That's not changing every time so you can cash it. Yep. And that's like 1.1 % is output. So basically, if you basically get all the numbers, a million tokens is going to be$2 and like around 26 cents. So that'll get you to something like$136 million a month for the 60 trillion tokens. So that's like way less than the 900. So that would be 1.6 billion a year, like run rate. It's still huge. That's a lot, but that is still in the max. Assuming they're in the top. Yeah, that's assuming that open router, the kind of breakdown of how they're using the tokens is the same as open router, which I think it's not.

6:04If we assume that, that's like$4 ,500 per engineer if there are, I think, 30 ,000 engineers at Meta every month. $4 ,500 on tokens.

6:16John Coogan:$4 ,500. That's actually in line with what I've heard a lot of other people spending in terms of their token budgets. Yeah, that's not like absurd. That's not absurd. If you're trying to incentivize people to use them. Yeah, yeah, no, not at all. So you can actually see the breakdown on OpenRider of how people are using tokens. So 17, the biggest plurality is OpenClaw, which is 17.6%. Yeah. And then CloudCode is 16.8%. Sure. So I think if you think about CloudCode, you would imagine that, like, in CloudCode, there's the kind of percentage of cache tokens is going to be higher than in OpenClaw.

6:48Yeah. So I think Meta's usage is actually going to be more heavily based on the cash tokens. Sure. So if you do it just based off like cloud code usage, you'd actually see a higher percentage of the input tokens, of the total tokens. So it's only like 0.8 % is the output. So then if you get all those numbers through again, it's only like 55 million a month, which would be 669 million a year. And each engineer would be like$1 ,800. Yeah, that's actually pretty low. which is like, I think very reasonable. John Chu over at Coastal says, plenty of my meta friends told me folks have been building bots that just run in a loop burning tokens as fast as they can due to this policy.

7:27It is an app. It's an absolutely stupid policy and is similar to how meta uses lines of code to measure engineering output. Managers are supposed to use it as a proxy and dig in to understand work complexity, but plenty of managers are lazy and just don't. That was in response to Christina over at Linear saying, ranking engineers by token spend is like me ranking my marketing team by who's spent the most money. We may not have hit our KPIs, but Joe spent$200 ,000 on a branded blimp that only flies over his own house. So he's getting promoted to VP.

7:55John Coogan:I'm pro-branded blimps, though. I like that idea. So my take on this was that, yeah, it sort of ties to what Jensen Wong was talking about at GTC. He was saying that an engineer that's making$500 ,000 might soon command something on the order of$250 ,000 a year in token budget. Andre Carpathie had a similar line. he said it's all about tokens. He said on a podcast last month what is your token throughput and what token throughput do you command? And so Meta actually has two different harnesses internally. They have a version of OpenClaw called MyClaw and then they also of course acquired Manus but it appears that they're running Claude maybe Opus under the hood to actually generate the tokens that come through those harnesses.

8:46John Coogan:The interesting thing is that at 250k AI budget per engineer, you're at like 20 ,000 a month. And so based on Tyler's math, this feels like, okay, there's going to be another maybe 4x to get to Jensen's prediction. I think it makes clearer the strategy with Meta Super Intelligence Lab. Because if you're looking at, you know, it's clear that they're spending hundreds of millions of dollars on this, just for internal code gen tooling, like running their business, they are going to spend an inordinate amount of money on frontier inference. And so training a model there, they will be able to amortize the training cost of the next model that they build, not just over can they get a product out that goes viral and becomes its own standalone chat app that people pay for, or maybe it's ad supported.

9:36John Coogan:Like just on the internal usage, they could be running a multi-billion dollar token bill that they would have to pay another lab. And so if they develop that internally, it's pure vertical integration. And then you also have everything that's happening on the actual ad targeting and content delivery side. And when you add up all of those, all of a sudden, the big question has been like, is Meta going to be able to launch an entirely new AI product, like Vibes or something like that. And this is a data point that to me says they don't need to. Just from a pure vertical integration story, the investment in MSL can pencil out.

10:17John Coogan:What are you laughing at? I just want you to get to your schizo theory. What's the schizo theory? That this whole like token maxing thing is like a barrage while they distill the model. Oh, oh, yeah, yeah. I mean, there is a world where if you're running, if you're generating trillions and trillions of tokens of a frontier model. They're like, Meta is really like burning through a lot of tokens. And you wonder what they're up to. It's like, oh, we're just token maxing. Yeah, I mean, there's another story about distilling we'll get to later in the show. But there is a question about if I have a, if I write an essay and then I have a model rewrite it, those tokens, they are from that model provider.

10:54John Coogan:they I buy them they become mine can I train on them that's probably out of terms of service so you would think no but you sort of wind up in this ship of Theseus world where if meta pays Anthropic a hundred million dollars or a billion dollars to go rewrite every line of code every email every slack chat every internal message like basically map the entire organization rebuild it they wind up with an incredible training corpus that they can use for their next model but i would imagine that they can't and i imagine that the uh that the enterprise contracts go both ways they can't you know the lab can't train on the corporate information that's standard in all of the enterprise contracts and i and i would imagine that the opposite is true as well although it is this fuzzy ship of theseus world where if you're using coding agents to upgrade your infrastructure, and then you want to run and train some model on your infrastructure, do you have to pull out the tokens that were revised by the AI lab that you don't have the right to train on?

11:59John Coogan:It's all very interesting. Apparently, startups that have gone out of business are able to sell their corporate histories for something like a million dollars to data brokerage firms and AI labs now. Have you heard about this? Yeah, I heard about it. Are you skeptical? I'm skeptical. I mean, certainly there's a market for it, but it's going to need some more data. All the code that a company built over a few years, maybe they raised it. Code, but also usage within different enterprise applications. Yeah, all sorts of different stuff. In other news, Intel is joining TerraFab. Yes, let's read through this.

12:34Intel is proud to join the TerraFab project with SpaceX, XAI, and Tesla to help refactor silicon fab technology. technology. Intel says our ability to design, fabricate, and package ultra high performance chips at scale will help accelerate TeraFab's aim to produce one terawatt a year of compute to power future advances in AI and robotics. And throwing up a post of hanging with Mr. Musk himself.

12:58John Coogan:Let's go through the Wall Street Journal's coverage of this. Elon Musk is partnering with Intel on his ambitious TeraFab project, which aims to build specifically designed chips for SpaceX and XAI, as well as for Tesla. In an announcement Tuesday, Intel said it would work with the companies to design, fabricate, and package ultra high performance computing chips at scale. The company shared a photo of chief executive Lip Bhutan shaking hands with Musk, CEO of SpaceX and Tesla. The partnership is a win for Tesla, which has struggled in recent years, Intel, which has struggled in recent years, leading the company to cut production capacity when demand was surging for data center chips and when competitors like NVIDIA and AMD have That was always just such a tough pill to swallow when you would talk to the ASIC companies like Cerebris and you'd say, Hey, you're doing something new.

13:47John Coogan:You're not doing NVIDIA chips. Is there any way you could get off of TSMC? And they're like, no, we still need to be in Taiwan. Obviously, there's a huge geopolitical component here. We can get into all that. But last year, the Trump administration reached a deal to acquire an equity stake in Intel for around$9 billion to help secure the American chip makers business. The U.S. government held 8.4 % of Intel's shares outstanding as of March 20th, according to securities filings. The figure doesn't include warrants that could increase the government's equity stake in Intel. TerraFab represents a step change in how silicon logic, memory, and packaging will get built in the future.

14:23John Coogan:Tesla and SpaceX confirmed the partnership in Poston X. Musk unveiled the plans for a single facility in Austin, Texas to make chips to be used by SpaceX and XAI, which merged in February as well as by the publicly traded Tesla. He pitched the project as an opportunity to quickly experiment on chip design by designing and manufacturing the chips in one facility. The fab will make chips for use in Tesla's robo-taxis, which they're already fabbing, I believe, at Samsung, although they do have NVIDIA Dojo chips, I think, that are TSMC. Optimus will also need chips, and they are planning to use Intel for that as well.

14:58John Coogan:So these These are two areas of priority for the electrical vehicle maker as it shifts its focus to artificial intelligence enabled products. It will also make chips optimized for use in space where SpaceX is planning to deploy huge numbers of satellites capable of handling AI computing tasks. Who else do you think they need to get involved here? Because just the two of these, the two of these guys, you know, Intel and Tesla coming together. It's good to have more involvement, but still, I think the entire project. We've seen a few of those like AI leader gatherings in DC where you see Tim Cook and Sundar and Sam Altman and Dario and all the leaders are together.

15:39John Coogan:And I was always hoping that at one of those dinners, they would say, okay, everyone's going to try and say the biggest number. But this time, it's going to be how much you're committing to Intel and how much you'll buy from them if they come online with a competitive product. because the demand side has always been a big problem for Intel, that they have the capability, they have the plans to build a 2-nanometer, 3-nanometer plant, like a frontier plant, leading edge fab. But every other company has been so tied to TSMC. But I think everyone now acknowledges that TSMC is not investing super heavily in CapEx.

16:16John Coogan:They're not scaling up as much as the industry would like them to. And so lots of folks have sort of signaled towards a chip bottleneck coming in the next few years. And Intel has the opportunity to communicate that. This seems like the first step in that chain. So companies, including Tesla, often design their own semiconductors but need a supplier to actually make them in a so-called chip fab. Musk's companies have sourced chips from a wide range of suppliers, including NVIDIA, Samsung, Taiwan Semiconductor. Oh, I got it. Musk said that TerraFab is needed because his company's demand for chips is slated to far outstrip the supply it gets from partners.

16:56John Coogan:I was listening to Chuck Robbins from Cisco talk about data centers in space, and the heating issue came up. And he was like, I don't really have a solid answer for that yet. But I do think that if you are bullish on data centers in space, you have to start with the fact that Starlink works in space currently because it is doing compute. You couldn't possibly put, let's be honest, John. We couldn't possibly put a computer up there. Yeah, like there are computers with, like, they can't inference frontier models. They can't, you know, it's not gigawatts in space yet. But there are, I believe, across the entire Starlink cluster, megawatts of compute in space with solar panels.

17:45John Coogan:And they do heat up because you are running a chip that routes packets across the Internet from one satellite to the next to get you your Internet via Starlink. And so it's not that it's a solved problem. It's that we are on a path to deploy some level of compute in space. Tyler? Yeah. I mean, we've seen, like, Philip Johnson, like, there are chips in space. right now, like their GPUs, I think, he said there were like five or six H100s, right? Yeah, yeah. So like they do work. It's like, I think most people's problem with space data centers is that it's like, economically, it doesn't make any sense.

18:19John Coogan:Well, so yes, that is the correct angle, but a lot of people are getting hung up on it. No, no, there is a whole conversation about like, it is impossible. And you need to like move past that into the economic equation, which then gets you into timelines and actually thinking about what needs to happen to dissipate that heat. But clearly, yes, you can. I mean, you can put humans in space on the ISS and cool that. We have created ways to move heat around in space for decades. It's obviously a new challenge. But I think starting with the baseline of like there is compute happening in space right now.

18:57John Coogan:We're going to try and I mean, Elon wants to like a thousand exit, hundred thousand exit, million exit. I don't even know what the scale is, but orders of magnitude. And so there's new engineering challenges. Speaking of space, it looks like Elon is going to use SPCX as a ticker for the SpaceX IPO, which he had to acquire from Matt Tuttle. Hence the ETFs ticker change shown below. Eric from Bloomberg says, we predicted this could happen in a December note. Nice catch by Will, who famously gave the meta ticker to Zuck. I did not know that Will Hershey had the meta ticker previously. So we know somebody that squats on.

19:35Who had the meta ticker? A guy named Will Hershey. Oh, interesting. There's a company called Roundhill. But we know somebody who's been.

19:41John Coogan:I thought it was Matt Ball. We had somebody here come in outside of show hours and say that they were squatting on a bunch of tickers, and the idea seems so. I think what might be the reality is that you actually, it needs to be further along than just reserved. I don't know. I think so. You can go. If you're a startup today, you can go reserve. your ticker today. But I'm not sure that that actually gives you enough leverage to when when Elon comes knocking ready for an IPO, you actually have priority over it. All right, we got to talk about a corporate retreat that went badly wrong. Okay. Technology company Plex took its 120 employees to Honduras for a week-long bonding experience.

20:28It was a disaster from the moment they arrived. Senior executives at the tech company Plex were eager to treat their 120 fully remote staffers to a week-long corporate getaway in a tropical paradise pop quiz tyler do you know what plex is i don't know about no have we seen plex i don't know either so we all failed but now it's your job to figure it out i will continue the plan for the honduras trip was simple company meetings and team uh building streaming company by powdery soft beaches during the day and island fun at night at a cost of roughly half a million to the company They'd build the trip around a survivor theme with teams and challenges, but it'd be fun, not too physically grueling.

21:05The CEO of Plex, a free streaming platform, would play a role similar to that of survivor host Jeff. Perhaps the executive should have taken it as a sign that just as the first bus of staffers pulled up to the resort, the chief executive was already in his hotel bathroom experiencing the initial waves of a violent stomach infection. What followed was a comedy of errors, including military drills that outpaced anything this group of office workers had in mind. a rogue porcupine stranded airplanes and one syringe to the butt of an employee corporate retreats are generally assumed to be torture or at least a semi-stressful chore what with their forced fun activities and hybrid work play environments that leave workers confused about boundaries is that is that like the industry standard that seems wild i don't know i don't

21:49John Coogan:think i've ever been on a corporate retreat i've been on some like founders fund events but those Those aren't really retreats. Those are more just like conferences, but I don't know. Corporate retreat seems, I don't know, unexplored territory for me. It's no wonder the new season of Jury Duty, a comedy series that tricks an unsuspecting non-actor into believing his off the wall fictional circumstances are actually happening is set at a corporate offsite. But in real life, PlexCon 2017 beats anything on TV. Here's the story of an all staff company getaway told by six people who were there, a trip where most everything that could go wrong did go wrong.

22:21nearly a decade later they're still working together and still talking about it it's crazy that they act it was bonding experience yeah well yeah it's crazy that this is now now coming out so sean hoff 42 founder of moniker partners an independent corporate retreat agency that planned the trip about three weeks before we arrived in honduras we got an email from the hotel's general manager that said i will be departing i wish you the best with your retreat i knew something was off three days later another email the head chef was no longer going to be at the hotel uh scott 52, chief product officer and Plex co-founder.

22:53We get there. We've got to take the bus from the airport. Dirt roads, you start getting closer, and there are guard towers around the property. People with machine guns and stuff. A lot of people were like, where are we going? Keith, the CEO of Plex 54. We usually go a day early, and we set up. If there's any little thing, we have to get it right just so the employees have the best experience possible. Keith woke up the day that people were coming in Sunday morning, and he is sick as a dog. Everyone there is fried. Basically, people are telling me, don't eat the vegetables, don't eat the vegetables.

23:18John Coogan:Don't eat the vegetables. No, no, no, because they wash it in water. It's usually not filtered water, right? Because it would just be kind of crazy to... Yeah, yeah, here it is. I've got to have a salad, just one salad. So I got E. coli, which may be the worst thing you could get possibly ever. Just as people were arriving on the buses, I was like, I had lost eight or ten pounds. They had a doctor come to me, which apparently is pretty standard. They nailed an IV bag to the bedpost. People are arriving for a party that night. The next day is survivor-themed kickoff. There's not one person on the planet more excited about Survivor than Keith and his wife.

23:52They have watched every single episode. My wife and I met Jeff, the host of Survivor. What I wanted is when everybody shows up, I do a Jeff. Welcome to the island. Here's the theme for the week. But Scott got to do it. The opening Survivor thing was a contest where people on their different teams open up a platter. You have to eat what's on the platter. Sean. Sean, who's the Plex head of business development. Are you going to call me? Yeah, somebody is cold texting me, pitching me their startup, and they've called me a bunch of times today.

24:26John Coogan:Wait, is it actually them or is it their AI agent? I wish I could pick up. It's just like a little bit too much. But yeah, cold texting somebody, like getting their number, I don't think that's the new meta. No. It's bold. It's bold. We heard from an executive in tech that they are getting dozens of emails every single day, trying to recruit them. And every email comes from a new Gmail account that's unregistered, brand new. But it's all LLM written, very different, doesn't really do all the research, but has a few keywords in there. And it's clear that someone is building sort of like a next-gen recruiting agency that's basically just a lot of spam.

25:16John Coogan:It feels like the end result will be like a return to relationship building and not like broad top of funnel. I should read the cold text from this morning. And I have nothing against cold email and just being bold. But I did read this out loud to you, John, so I'll read it to everyone. So I got a text from an unknown number today at 7 a.m. All right, Jordy, good news or bad news first? This is blank, and I'll leave the name out. And then I just get a PDF of a deck and then a text. All right, Jordy, the bad news is this was an unplanned introduction, and on the surface, probably lukewarm outreach.