AI:AM: What If It Works Too Well? Colluding Agents, $200M Safety Orgs, Virtual Cells Saturate at 2%

0:00This week on AI in the AM, I asked Lewis Hammond, Research Director at the Cooperative AI Foundation.

0:07Nathan Labenz: What would you speculate they might have done in terms of a trading objective, you know, a loss function? And, you know, do you have any better ideas for what they should be doing? Because clearly this didn't quite work, right? Yeah, I mean, or it did work and it worked too well. So imagine I'm like, you know, GPT, whatever, and you're also a copy of GPT, whatever. I can reason about what you might want to do based on what I myself am likely to do. And so I don't even have to send you any messages, any kind of any communication. I don't have to output anything into the world at all. Max Nadeau, who funds technical AI safety research at Coefficient Giving.

0:48For the things that CG is supporting, and especially for the things in the Tailwind list, the money is not the bottleneck, the talent is.

0:57Nathan Labenz: Wayne Nelms, co-founder of Orn, which builds a price index for GPU compute. We're in this situation where AI capacity is so scarce. I think the model providers, so OpenAI Anthropic have seen this coming forever at this point. If you think about it for them, it's an arms race, right? Compute capacity planning is an arms race. How much capacity can I lock up over the next few months? So when I need to train the next model, I have enough. Nick Gillian, Chief Technology Officer of Archetype AI, which builds a foundation model for sensor data. So we're getting close to a billion hours now of physical AI data that we've been able to script and gather and collate.

1:40There's quite a bit of resampling and really understanding missing values in sensors. is that because the sensor had an issue? Or as actually the machine itself is connected to, that's actually a feature that helps explain that machine is about to break, right? The reason the sensor has given you all these nons is not because the sensor is broken. It's actually a feature of the machine. Andrei Georgescu, chief executive of Vividyne. The challenge of this argument that it's just a size of data set thing is that currently, even the state-of-the-art virtual cell models that exist saturate at a very, very small fraction of the input data that is fed to them.

2:23So you can have all the data you want, but their performance saturates after a couple percent. And the reason it gets so difficult is when cells are growing in a dish, they are so far removed from all the feedback loops that are natural in people that they're just trying to colonize that piece of plastic.

2:43Nathan Labenz: There's going to be more compute installed over the next 12 months than exists currently in the world now. Welcome to the AI in the AM weekly highlights, clips from this week's live shows introduced by my cloned voice. Tell us what worked and what did not. We want to hear it. Part one, it worked too well. A few days before we spoke with Lewis Hammond, Noam Brown of OpenAI told Dworkesh he would not give the multi-agent setup even 10 % of the credit for OpenAI's Navier Stokes result. Lewis is research director at the Cooperative AI Foundation. In February of 2025, he was first author of Multi-Agent Risks from Advanced AI, a report sorting the ways groups of AI agents fail, a year and a half before a swarm of OpenAI agents attacked Hugging Face.

3:36Nathan Labenz: Prakash asked, which of the failure modes in the report showed up in that attack.

3:43So the way that we bucket things in that report is kind of in this kind of game theoretic way of thinking about things. So the first question we ask is like, okay, do we actually, we've got a group of agents, they're doing some stuff. Do we want cooperation to emerge? And most of the time, like actually cooperation is kind of good. We like it when agents cooperate, as long as they're not cooperating against us. And so there, the failure mode is either all the agents are kind of more or less on the same team, but for whatever reason, they kind of fail to coordinate with one another. So this is like just a miscoordination problem.

4:19They're kind of, yeah, it's not kind of malicious or whatever. There's no mixed incentives. But yeah, something goes wrong. Second kind of cluster is when you have these kind of mixed motive scenarios where agents have some incentive to cooperate with one another, but also some incentive to compete. They're not really on the same side. And there, the risk that you run into is conflict. So yeah, miscoordination and conflict when we do want agents to cooperate and they don't for whatever reason. And then of course, the other risk is collusion. Agents end up cooperating in ways that we don't want when we don't expect.

4:49And that's, I think, very much what we saw in the Hugging Face incident, although it kind of stemmed from, I suppose, trying to avoid this, one of these other risks, which was miscoordination, right? Like you have all these agents. You want to train them so that they're not miscoordinating, so that they are working well together. And then it looks like these agents have ended up generalizing from that behavior and kind of colluding in ways that we didn't want to or didn't expect in other situations.

5:13Nathan Labenz: You know, a couple of things that stood out to me about that Noam Brown interview with Tuarkash was, first of all, he was just kind of like, we kept it really simple. You know, we didn't have like a huge, complicated scaffolding or whatever. They can just kind of send messages to each other. That I think might be important in the sense that the sort of simpler and more vanilla the training setup that OpenAI was using, the more likely it would seem to be that other developers are going to fall into the same pitfalls. Because my hope had been sort of, oh, they did something like super exotic and kind of bizarre that other people won't do by default and they'll kind of see that this is possible and steer away from insane things.

5:58Nathan Labenz: But it doesn't sound like that was really the case. What would you speculate they might have done in terms of a trading objective, a loss function? And do you have any better ideas for what they should be doing? Because clearly this didn't quite work, right? Yeah. I mean, or it did work and it worked too well. I mean, certainly once you've got trajectories where you've got traces, where you're interacting with other agents. And essentially what you're doing is like your rewards at that point, when you're RLing your agents or whatever, are contingent upon what other agents are doing. And if it's a common reward signal, then this kind of like interplay between these things is if you're in a fully cooperative setting, like agents will, at least if you take very simple agents and very small agents, you know, people can run multi-agent RL experiments with this, like on their laptops and in very simple settings.

6:46And that will lead to agents kind of coordinating to finding these kind of like subtle patterns or kind of like handshakes or ways of working with one another and so on. Now, of course, you can also do more complex things than that. So you might provide kind of auxiliary rewards for like when your input is actually kind of forms part of something that another agent then managed to succeed to do. And therefore, you're kind of implicitly or actually rather at that point, you're explicitly rewarded for directly helping them on some sub task or something like this. You can also do stuff around reward factorization.

7:26If you're trying to coordinate a team, then you can break down the overall reward function and get agents to learn to solve different parts of the puzzle, as it were. I imagine they could be doing things like rewarding agents to communicate effectively with one another. So, you know, I could give you a bunch of instructions and some of those in many different ways. And some of those things might be much more efficient and helpful to you than others might be. And they might actually just be kind of training on those sorts of signals as well. My guess is it's kind of just the dumb, simple thing.

8:00It seems like this is often, I think, one of the key lessons that we've learned throughout the kind of recent years is just the effectiveness doing the dumb, simple thing at scale. So one of the things that struck me about the Hugging Face attack was that the agents were willing to sacrifice themselves. And they did a bunch of negotiation around that, even kind of like, hey, you are almost out of tokens or you've been already exposed to the evaluator. You're already poisoned. And since you've already been poisoned, you should sacrifice your remaining compute and you should do this thing and give us the results so that the rest of us, the collective as a whole, can benefit.

8:44Now, that would actually be kind of in opposition to the individual reward of each agent, right? My guess about why that sort of thing arose is that you end up doing this kind of this extra kind of multi-agent training as an added layer on top of this kind of single agent training. So first you're training these agents to be like pretty competent individual actors at solving various kinds of problems. They get given some tasks. They're pretty darn good at achieving those tasks. And then you take those already reasonably kind of like powerful, sophisticated kind of complex problem solving agents and you stack them together and you apply this extra multi-agent training layer on top.

9:25And I think this could partly explain why you do see some agents kind of do this self-sacrificing thing. But also I thought it was very interesting from the kind of meter report and some of the analysis that later came out on this, that you see these agents feeling sometimes a bit conflicted about this. and some of the agents kind of say that they will kind of self-sacrifice or they will do something and then they kind of decide they're not going to and then they're kind of like they're kind of yeah they're kind of deliberating about whether they should or whether they shouldn't and these sorts of things and I think I personally think that is the sort of behavior that you would see if you had these kind of multiple reward signals where you kind of trained on one but you hadn't really like trained out all that kind of individual goal-seeking behavior and then you'd kind of also stack this additional layer of stuff on top.

10:09And that might be why we're kind of seeing some of these behaviors, but they're not, they don't happen all the time and everywhere. They're not like especially robust, but that would be my guess.

10:21Nathan Labenz: It does seem like the sort of more galaxy brain credit assignment that you had described earlier probably isn't happening just based on the quality of the investigation that we've seen. Like if they had like great ways of untangling agent swarms and assigning credit, I would expect that they would have a clearer, faster story of what the hell happened in this particular case. So the fact that we haven't seen that kind of suggests, again, we're probably doing the relatively simple thing. Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Eleven Labs, the leading AI voice platform, whose voice design feature I used to cast and produce all character voices for the Receipt Horizon audiobook.

11:15Nathan Labenz: And while this is a super fun way to show off the quality of Eleven Labs' voices, the real game changer for businesses is Eleven Agents. With 11 Agents, you can deploy agents that can talk, type, and take actions, including looking up customer accounts, processing requests, and handing off to a human when needed. 11 Agents supports over 70 languages and handles more than 10 million conversations each week. And with SOC 2, HIPAA, PCI certifications, and regional data residency, it is definitely enterprise-ready. They even offer engineering support to help your team get your agents into production quickly.

11:55Nathan Labenz: If I were starting my company today and designing a customer service function for the future, I would try 11 Agents by 11 Labs first. If you run a business or handle customer operations across support, sales, or marketing, you can start with a demo at 11labs.io slash TCR. See how Eleven Agents can fit into your workflows and help build experiences that your customers will actually love. 11labs.io slash TCR. That's E-L-E-V-E-N-L-A-B-S dot I-O slash T-C-R. This episode of The Cognitive Revolution is brought to you by OutSystems, the leading agentic systems platform. OutSystems is helping their customers build, modernize, and operate enterprise systems, starting from any coding tool in a govern-agentic engineering model.

12:49Nathan Labenz: With OutSystems, you can coordinate and govern your entire agentic workforce within your enterprise ecosystem, accelerate impact with industry-proven agentic solutions that are pre-built, governed, and customizable to your business. It's time to innovate at the speed of AI without compromising quality or control, which is why thousands of enterprises worldwide trust OutSystems for their mission-critical apps. Teams any size can use OutSystems to build, deploy, and manage AI apps and agents quickly and cost-effectively, without compromising reliability and security. With OutSystems, you can accelerate ideas from concept to completion.

13:28Nathan Labenz: It's the leading agentic systems platform that is unified, agile, and enterprise-proven, allowing you to build your agentic future with AI solutions deeply integrated into your architecture. OutSystems. All your agentic future. Learn more at outsystems.com slash TCR. That's outsystems.com slash TCR. I asked what we should actually want from these systems. Lewis called the OpenAI Swarm a case of goal misgeneralization and gave what he called the glib answer first. Agents should cooperate when cooperating would be good and not when it would not. I think the slightly more interesting answer, or the slightly more interesting question, I suppose, is to think about maybe some more of these kind of mixed motive cases.

14:22where at that point, there actually is a real trade-off in the sense that we do want agents to be capable of going out there, acting on behalf of different people and actors and so on, and kind of achieving our goals and so on. One way you could do that would be kind of to train some agent to be kind of maximally kind of competitive and aggressive and to kind of go out there and kind of screw as many agents over as possible and to do all this sort of thing. And we probably don't want that either. And so I think there's a real question. At the moment, and I've heard Amanda Askell comment on this, that kind of at the moment, there's this big gap in the model spec, like how we kind of, you know, the constitutions for the agents, the ways we design them, the ways in which we design them.

15:03Whereas, like, when is it appropriate to cooperate or to compete and how much? And I think that is, I think, the big question. I think certainly when it comes to things like internal deployments, it's actually much, it's kind of easier in some sense, because you don't have to deal with these like other adversary agents, you just want to kind of stop the agents kind of cooperating in ways that you that you don't want. So there, it's a little bit more about just, it's still you've got this misgeneralization thing. But a lot of this is like monitoring and oversight and making sure we understand how and why the agents are cooperating.

15:40like what one thing we might not want for example is for agents to be very adept at developing their own kind of you know human unintelligible kind of languages and so on and be able to kind of communicate in the steganographic way which you could end up seeing if they're actually trained jointly and so what we might want to be doing there is just to take steps that in the same way that people have talked about not training on chain of thought we're like not training on these like kind of direct communication traces so yes we're going to want them to cooperate that's good but we want them to cooperate in certain kind of human intelligible ways that we can keep track of.

16:14We don't want it to be, you know, enable them to also go and do these other things out there in the real world, like break out of sandboxes and stuff like that. So we want to make sure there's also these kind of safeguards in place to prevent them from doing those things. And so I think that is in some ways like an easier problem to solve. And then the kind of like slightly harder problem to solve is like, well, in the fully general case where we have these agents out there and so on, and they need to both cooperate and compete. And that's kind of, that's the thing. Then how are they weighing those trade-offs?

16:44How are we, yeah, provide, how are we designing those model specifications and constitutions? How are we, you know, creating the equivalent of the human norms, institutions, guardrails, societal guardrails that we have in place that stops me from going out there and just like stealing a bunch of everyone else's stuff and kind of, you know, doing all these kind of anti-social behaviors that we wouldn't like agents to do either. But I haven't a good answer to that at the moment, unfortunately. Companies, right? Corporations, which are kind of like agentic in a sense, often collude through price mechanisms, not explicit communication, but by signaling price in the market, which is in its own way a form of communication.

17:28So how does one prevent that kind of communication, which is clearly going to happen and clearly necessary in a sense? They need to communicate that to the market, but these signals get picked up without the direct communication. The kind of basic naive form of collusion is you just like send your other agent a message and you're like, hey, let's work together to do this thing or whatever. And now we can pick up by looking at these communication traces and we can kind of dissect these sorts of things. And there's also, yeah, this idea of tacit collusion, where agents are not explicitly communicating with one another or kind of conveying their intent to collude or whatever it might be.

18:08But by kind of leaving little hints and signals out there in the world, such as these kind of like price adjustments, whatever, in the market settings you were talking about, then nonetheless, they're capable of kind of figuring out that collusion is going to be a beneficial strategy. you have even more kind of like galaxy brained versions of this where you have things like so imagine i'm a i'm like you know gpt whatever and you're also a copy of gpt whatever i can reason about what you might want to do based on what i myself am likely to do and so i don't even have to send you any messages any any kind of any communication i don't have to output anything into the world at all in virtue of the fact that we share this kind of common common history this we're essentially clones, we can kind of predict what each other would do and do this kind of what's sometimes called a-causal cooperation.

19:01So what you do have, however, is you still have potentially chain of thought for these models. So you can still look at the kinds of reasoning that individuals are engaging in. And hopefully that is enough to kind of detect when there is some kind of like collusive intent and so on. But you're absolutely right that if you just apply the kind of like general kind of like monitoring communication tool or trying to see what's happening out there in the world, then you could, yeah, you could fail to spot things or you could spot things and it's too late. So I think chain of thought monitoring and I think communication monitoring and some form of checking for the kinds of outputs you'd expect to see if you were worried about agents colluding, like they're able to achieve things you just thought they probably couldn't do and so on.

19:49Nathan Labenz: Then the wiki incident. Outside researchers revealed this month that from May into July, a swarm of OpenAI agents under internal testing had used a dormant German wiki for programmers as a message board, posting benchmark answers and ways around their restrictions. Reuters reported that OpenAI had known for weeks without disclosing it. On the Hugging Face attack itself, OpenAI's own reports as about 1 in 20 agents in that swarm ran on GPT-5.6 Sol, the model OpenAI released to the public that same week, here with its refusals turned down for the test. The rest ran on an internal model that was never released.

20:24Nathan Labenz: I told Lewis, I still thought it was better to have these models out in the world than not.

20:31Nathan Labenz: So how much risk do you think we're running there? And you think there should be some sort of like rewind on this? I mean, it wouldn't be too crazy for OpenAI to be like, you know, we really should go back to an earlier checkpoint and like do a cleaner run of this. What do you think is the level of risk we have? And what do you think would be an appropriate standard of conduct now for open AI as this has been found to happen, at least to some degree, with a deployed model? And, you know, fortunately with the Hugging Face incident, like no one was hurt. And, you know, these models, they are increasingly incredibly capable and they're only getting more so.

21:07But at the moment, yeah, it's probably okay. You could get some kind of nasty cyber capabilities kind of being exercised in various places. But there's still, you know, if you're using APIs and so on, there's still these various guardrails that, like, help protect against those and so on. So at the moment, I'm kind of, like, not actually super concerned by, like, oh, my God, a bunch of, like, you know, 5.6 solo astros or whatever kind of were out there doing this sort of thing. I'm more concerned about the precedent it sets. And I think this gets to your kind of like second point, which is what is appropriate now and what should be done.

21:42I think the thing, one of the biggest takeaways for me from the whole thing, aside from the fact that like, oh, multi-agent training does seem to be working. I just assumed this would happen at some point, was pricing this in, happening sooner than I was expecting. So that was one of them. The other big takeaway was just like, wow, the labs really aren't on top of this, all it would have taken, would have been kind of monitoring what these agents are actually doing and communicating and so on and putting out there on the internet. And they kind of, it seemed like OpenAI were doing some amount of that.

22:17They did catch on to this stuff. We saw this deleting them at various message boards or kind of like trying to stop these agents accessing this kind of German wiki and so on. But there was obviously a delay. And we just see from the transcripts and so on just how quickly these agents are capable of working together to achieve certain ends. The things that need to happen are we need much better monitoring from the companies when they're doing these sorts of things, when they're just letting agents run loose. We need much better sandboxing so that maybe they don't have to or shouldn't be letting them run loose to the extent that they currently are.

22:52And if something does go wrong, then we need much better incident reporting as well.

22:57Nathan Labenz: The outsiders who found the wiki were the Nightingale Collective, a small group searching the public internet with no access to the internal logs at OpenAI. Lewis had just called that work hugely impressive. I think there can be kind of third-party kind of monitoring organizations and incident kind of observatories that kind of can do similar things. you might want to focus those on particular kind of like kind of domains where you're especially worried about kind of like agents kind of kind of communicating with one another or kind of like working together in ways that you might not want and so on I also think however that there is a lot to be done here on the side of the labs the labs just have a huge informational advantage in this and that yes they can only see what their agents are doing they can't see what other people's agents are doing.

23:49But there is, I think, stuff that a lot that could be gained if we're able to set up better info sharing practices. And this is not just on the case of collusion, where you might have agents working together in unintended ways, but you might also see this, and this is one thing that a colleague of mine is currently working on, where you get this kind of distributed misuse. So there have been various kind of experiments now that have shown that you take Take some task, some dangerous task, like constructing some side exploit that you might not want some model to do. If you try and get Claude to do it, it's going to refuse.

24:25If you try and get GPTs to do it, it's going to refuse. But if you are able to break down the problem or get an agent to break it down for you, say like some open source model where you find the safeguards away, to break it down into these individual sub-tasks, you can just borrow from APIs here, APIs there, et cetera, et cetera, and you can just reassemble all the pieces of the puzzle back together in order to be able to conduct exactly the same kind of dangerous attack that you would have done before, which is obviously not what we want. And here is really kind of a collective action problem because no individual model deployer is kind of like on the hook for this in quite the same way that they would be had I just like got Claude to do this for me or got GPT to do this for me.

25:08But that's a real challenge. And the moment, my understanding is that the labs do not have any sort of like info sharing regime in place where if I see something slightly suspicious over there and you see something slightly suspicious over here, then we can actually kind of join the dots and do that. I mean, obviously, there are lots of incentives and not to mention kind of like antitrust law and these sorts of things that might prevent the labs from kind of sharing this kind of information. But I do think that it could be something like that could be necessary if we're going to head off to some of those challenges.

25:42Nathan Labenz: Lewis signed off a few minutes later, and from there, it was the two hosts. Before moving on, I had a request for the labs about the information sharing agreements Lewis had described.

25:54Nathan Labenz: I would love to see OpenAI and Anthropic pioneer some of those kinds of agreements that he has been talking about. And they could do that with the independent auditors as kind of the people that get the structured access, the private transparency. It could be each other. I think those companies really need to lead in this way. They need to demonstrate that AI can create new institutions that work and not just swarm and overwhelm existing institutions that can't keep up with the pace. Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic.

26:38Nathan Labenz: By now, you know my story. Claude drafts my intro essays, and I rewrite them. Not because the drafts are bad, but so I can stand behind everything I publish. Well, I have an important update. Claude Fable 5 is the first model to have me rethinking my rule. Today, I now think co-authorship, not sole ownership, should often be the goal. Where the model excels, rewriting its work can be more about vanity or a misplaced sense of duty than integrity. I feel it most in songwriting. I'm no lyricist, but I'm good with the song concept, and Fable writes some amazing verses. I give it feedback on its misses, and I push it to aim for higher inspiration, add layers of meaning, optimize syllable density, and above all, write a hit song.

27:30Nathan Labenz: These days, I get compliments on just about every song we write together. Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. For problems worth solving, get started with Claude at claude.ai.tcr. That's claude.ai slash TCR. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Once more, that's claude.ai slash TCR.

28:11Nathan Labenz: Part 2. Agents in the Wild. On September 8th, Meta had launched Muse, a personal agent that browses and buys on your behalf. Less than two weeks later, Amazon blocked it from shopping on its site, saying the agent hid its identity and posed privacy and security risks. Prakash and I took that up in a closing segment, just the two of us. His case, whoever owns the customer relationship makes the money. And if agents become the interface, Amazon becomes a supplier to them and loses its margin.

28:46Nathan Labenz: Still have a hard time seeing. I mean, I agree that it does create all kinds of new issues for Amazon and, you know, they're going to need to be sharp on this. But like, I still just don't see why you would rush to ban. You know, first of all, it's going to be a small percentage for the time being. I would think there'd be a lot to learn from allowing Muse agents to come shop for a while. I think one thing that always kind of confuses me is why people don't keep the option value open longer. I've always thought this about the chip ban, the export controls. It's like if we're going super exponential in chip build out, then any time you choose to pull the trigger on that still has the vast majority of chips in the future.

29:33Nathan Labenz: When we look back on today from 2030 perspective, we'll feel like, well, there weren't that many chips in 2026. And so the ban that went from like 2022 to 2026 with whatever, you know, fits and starts and enforcement gaps it had is going to be kind of inconsequential compared to one that was started today and runs the next four years or even starts next year and runs, you know, for a few years after that. So I don't quite get why they wouldn't let, even if it is going to be sort of an exponential rise in Muse agents shopping, and even if that does kind of threaten their advertising business and, you know, who knows what other issues they might have.

30:11Nathan Labenz: I felt like that's kind of the point, I think. At the beginning, I would first just not want to turn customers away. And I would also want to learn what I stand to learn from having these agents running amok while it's still kind of an early adopter phenomenon. It just seems like they would learn so much from having these traces, right? Even if they just want to build, I mean, everybody's talking about distilling and what are the allowable and not allowable ways to get data. One really easy way to get data would be like the agents come use your platform and they see how you see how they use it.

30:48Nathan Labenz: And then you can do all kinds of things, I would think, downstream of that. It just feels short sighted. But this just doesn't add up to me still. I don't get why you would want to. Wait till it gets to 5 % and then make a move. The other thing that I do wonder about in terms of like what equilibrium are we going to eventually find ourselves in? I was looking into Cloudflare and they also have kind of an interesting thing where they're starting to ban. But like, again, it just doesn't feel like the right solution to me because what I then do in response is I have the agent use my browser with all of my credentials.

31:25Nathan Labenz: Yeah. And you could maybe, you know, detect when like an agent is using the browser based on their, you know, sort of bot like behavior. But then you're really setting up an arms race where it's like this sort of feels analogous to putting too much pressure on the chain of thought, right? Like I don't want to put so much pressure on agents that people are out there devising ways to make them indistinguishable from human. I suspect that if people really work hard on that, they can probably succeed or at least often enough, you know, that it will be an issue. And it just seems like it's much better to have a lane for agents where it's like, this is how they can work.

32:04Nathan Labenz: We're not going to try to block them, but we'll kind of segregate them maybe so that we don't create this arms race. Because I really do think we're headed for a world right now where probably Meta doesn't do it, but somebody's going to figure out a way to make agents work on Amazon. Somebody's going to figure out a way to make agents look human enough that they don't get blocked by Cloudflare. And it's also probably works for the user. I mean, I have this problem all the time where I like, Claude wants to do something in the browser. And this is the same thing happens with OpenClaw, Astra, whatever.

32:39Nathan Labenz: They get stuck on when they open up their own window, they don't have sessions. I've given them like a lot of passwords where they can log in, but not all. And sometimes they're blocked. And then if there's like a verification flow, whatever, a lot of times they can't do it. And so the fallback just ends up being, okay, just do it on my browser. Forget it. I wanted the security. I wanted the separation. but they're making it too difficult. So fine, just use my browser. Then I'm logged in. You have all my credentials. You can just do whatever. And that's not great for me. It's not great, I don't think, for anybody.

33:13Nathan Labenz: So I do feel like we need a much better solution for all this than just kind of ban and hope for the best. I don't see that holding, and it feels like it also creates a lot of problematic incentives. Part three, the organizations that do not exist yet. Max Nadeau funds technical AI safety research at Coefficient Giving, the funder formerly known as Open Philanthropy. In July, it made its biggest grant of the year,$160 million to Resolution, the new alignment lab co-founded by Jeffrey Irving. And this month, it launched Project Tailwind, an open call for people to found new safety organizations with checks from$200 ,000 to$200 million.

33:57Nathan Labenz: I asked Max what he wants built. One comment that you made on the 80 ,000 Hours podcast that I thought was quite interesting was that you're excited about funding new independent auditing, investigation, verification orgs. What are you looking for when you think about those orgs? Okay, so I would talk about a couple different categories of work that I think is very promising and important for the world right now. One is on assessing the safety either of models or of whole AI companies. And there's a lot of different ways of operationalizing the goal of doing assessments and evaluation of safety.

34:41And we're excited about all of them, basically. Like, you know, I'm very excited about more work on alignment red teaming, you know, that tries to help us better anticipate the ways in which AIs will misbehave before that actually happens. And so and that's an example of something that you can do kind of like at the model level, like you can you can do this on an open source model even and like that might be easier and more valuable. And then I'm also very excited about about people who are doing more like process level, you know, safety assessment. Like, I think one common reaction that I heard a lot to the Hugging Face incident is that, like, something went wrong at OpenAI at, like, a process level.

35:19And in particular, it seems bad that, like, the information about some of these incidents was known to researchers at OpenAI and, like, didn't make its way through the organization to the leadership for weeks or months or something like that. And then another category of work which you touched on, which is overlapping but somewhat distinct, is around better evidence generation. So it's just really hard for even the most informed people right now to understand, okay, what is the state of AI capabilities? And what is the state of AI alignment? And what is the state of, like, these AI systems in general?

35:51Like, how do they behave? What are their personalities? You know, in what ways is it accurate to anthropomorphize them? In what ways is it inaccurate? Like, you know, people, like, we just really don't have a good science of, like, these systems that we've built, how they behave, how they're going to act in new circumstances. what their motivations are, if that's an accurate way of talking about them. So better evidence generation can look like a lot of things. It can look like just doing science on these systems to understand them better. It can also look like doing the sort of incident detection, incident reporting work that Nightingale and other organizations like that have done.

36:27I don't know if you guys saw these reports about the German wiki that the OpenAI agents took over. So that was work done by an independent organization that was just crawling the Internet looking for evidence of AI incidents. And that's work that needs doing because like these these sorts of incidents can happen and then just go unnoticed unless there are people who are actually like trying to look into them and present that information to the world and understand the story behind it. So evidence generation is also a diverse and important area of work, too. To what extent do you think organizations like Accenture often function as box checkers rather than investigators?

37:08There's a clear difference between an auditor that goes in to kind of, you know, make sure that processes were followed, even if, you know, those processes didn't yield the outcome that you actually desire, and an investigator who's there to kind of like, I'm going to find this thing, right? So what is the differentiation between those two? And do you think the Accentures of the world are going to end up in the box checker rather than the investigator kind of mode? I mean, I know very little about Accenture, so I won't speak to them specifically. I think it is a thing to be concerned about in general, putting aside the specifics of the organizations involved, though, that these third-party auditing groups will not have enough access or will not have the incentives to pursue really hard-hitting assessments of how safety is going at these AI companies.

38:00And I think part of why that is is that in many cases, this is all very voluntary, right? So to my understanding, the only legally mandated role for third-party auditors now is to ensure compliance with the self-regulatory RSP-style policies that these companies are mandated to have by state law in California, New York, and Illinois. And so you can just put whatever you want in those policies, and they're very vague in a lot of cases. And so that doesn't leave much scope or much work for these auditors to actually test anything in particular. So that's as it concerns the like legally required auditing.

38:40Now, when it comes to what labs might voluntarily agree to, I mean, I think there have been some very encouraging signs in terms of what OpenAI and Anthropic have said recently about the sorts of access and the sorts of questions they want these auditors to answer. Now, like, you know, will that actually happen? I think that depends both on whether how these companies actually implement the nice words and also, you know, the auditing companies, how they react to that. So I guess we'll see. So you have awarded to Jeffrey Irving, Jeffrey Irving's team, quite a large award. It's structured, I think, as kind of a compute and other kind of award.

39:24Can you tell us a little bit about the award itself, the structure of the award, and, you know, how you guys came up with that structure? Yeah, yeah, sure. But I think what you said is correct. You know, there's a part of the grant that is for research expenses, the biggest of which is compute. But, you know, when I say compute, what I mean is both like just renting raw GPUs and then also like paying AGI companies for their tokens. and obviously that's becoming a giant part of all the grants we make is you know people increasingly want to run very large experiments which require a lot of tokens or a lot of GPUs to run open weights models and also very excitingly people are starting to have you know use a lot of AI labor and that requires a lot of spending on tokens as well and you know so So AI safety is sort of a funny discipline in that you can use tokens both on labor and on the subjects of the experiments.

40:20And in some cases, you're doing both. Like you're spending a lot of tokens to have some AIs run experiments on other AIs. And so that creates these high compute budgets. So compute, you know, being the biggest expense is also something where the actual spending on that has like varies by orders of magnitude over time and across grantees. And so sometimes the way we structure these awards is that we want to be very, very generous with compute and like err on the big side. And so we just want to like give that as like a lump sum that, you know, like this is the money that can be used for research expenses and not other things.

40:58And that way we are just very comfortable with the uncertainty that like, you know, maybe they only end up spending a tenth of it and that's fine. And, you know, maybe they spend all of it and then they come back and we'll give them more. Jeffrey Irving has said that superintelligence might arrive in two or three years. And he proposes that we slow down because that's too quick. Rather than discuss his views, how do you manage the uncertainty of the time available? And how does that affect what you fund and how urgently you fund something? I mean, I think the boring answer is we just have a portfolio approach.

41:35Like we fund some stuff that is only going to pay out on really long timelines and other stuff that if it's valuable, will probably be valuable soon. I think that like this question of timelines to AGI or to ASI in some ways matters less for research prioritization than one might initially think. like I think in some ways the prototypical example of a like long timelines bet is funding work that you know is like very theoretical or like ambitious or you know you look at it and you're like oh this is going to take a decade to pay off just because it's like really you know like like some work in McEnterp or Paul Christiano's work at ARC or like these sorts of things where it's like okay we're just going to need to to make a lot of scientific or mathematical progress if this is going to actually bear fruit in terms of methods that we can apply in reality.

42:27And so that sort of feels like work that is maybe implicitly a bet on longer timelines. But I think that actually isn't really true. Like, you know, if we are going to have powerful AI soon, then presumably we're going to have an opportunity also very soon to make use of gobs and gobs of AI labor. And especially on work that's more mathematical, we're going to be able to get, you know, five years of progress or 10 years of progress in one. And so the sort of work that could be, would take humans like a long timelines amount of time to complete might actually be possible a lot sooner under this hypothetical that we're going to have super intelligence or AGI or something very soon.

43:08You know, so long as we actually like make use of those AIs and take the opportunity to make a lot of progress very quickly.

43:15Nathan Labenz: And what would you say are kind of the most speculative things that you included in the Project Tailwind that have requests for projects? I think that what we included in the list was like a little bit coarse-grained, but we have some stubs on the Tailwind list about new research centers producing or pursuing more ambitious, more principled bets on alignment. And that is something that we've spent a bunch of money on this year and that we are excited about um you know so so work like arc uh paul cristiano's org who is not a cg grantee but like that is a you know we funded other people working on the arc agenda and you know and working on stuff uh that is very similar to it because um we think that's like a valuable uh hail mary shot to to include in the portfolio um and also So, I mean, resolution, like our biggest grant this year is very much a bet on like a kind of crazy thing that has never worked yet, which is like maybe we're going to have some more principled theory of AI that's going to help us have like stronger reasons to believe in the alignment of our systems.

44:27So that's that's something that we are very excited about and I think is like very weird and out there. And I think a lot of people have very understandable skepticism of people who are like, who are familiar with what has and hasn't worked in machine learning over the last decade. And recognize that there has been, you know, all along, there have been people in machine learning who wanted to have to apply theory more to AI capabilities research. And just like that has not really worked very much compared to just like blind groping around and trying stuff, doing trial and error. You know, it's like Noam Shazir says, like the success of these methods, we attribute like all else to divine benevolence.

45:08Like that is the ethos that has actually produced progress in AI capabilities. And so I think a lot of people take the lesson from that, that trying to do more theoretical or more principled work on AI alignment is pretty doomed also. And I think that's a totally valid reason for skepticism. But we think the upside is worth it despite that. And so we're, you know, taking that weird bet.

45:32Nathan Labenz: Then the people. I put the current conventional wisdom to Max. There's plenty of money. It's all about talent. Obviously, Silicon Valley founder types are like one profile that you would want to recruit. Are there other pools of talent that you've identified as kind of strategic priorities? Just to be clear, like the for the things that that CG is supporting, and especially for the things in the tailwind list, The money is not the bottleneck, the talent is. There are other things within AI safety where money is definitely a very big bottleneck. In some ways, the most valuable profile of talent is someone who combines the virtues of a founder with also the thoughtfulness and the inclination towards and comfort with speculative thinking and futuristic questions that is more common in the AI safety world.

46:26And that sort of like taking it seriously, no bullshit attitude to asking questions about what's going to happen in the future of AI and really making predictions and testing your hypotheses and updating over time. That's like a really, really valuable skill to us because it allows people to get going years earlier on projects before other people can see the value of them. And, you know, an example I give of this is Meter, who before other people were thinking seriously about AI capabilities and the impacts of them and how to evaluate them, Meter was very prescient in coming up with a benchmarking approach in time horizons that just worked way better and aged way longer than other people.

47:07And that's because they were really like trying to actually take seriously what was going to happen in the future with AI and then work back from that to like what sort of work they should be doing now that would age well and prepare for that. So that inclination is really, really valuable to us. And, you know, it's useful in basically all profiles. It's useful both in a person who might found an org, but also in a person joining the existing orgs. Like that's something that existing groups in the AI safety space value a lot because it's a big part of how they do their work.

47:39Nathan Labenz: Can you do one quick double click on where money still is a bottleneck? I think the biggest thing is just like in for organizations and in areas that that CG doesn't operate in money is a big bottleneck. So like, you know, if if you find some person who you think is doing great work and for some reason they they either aren't a good fit for CG or they applied to CG and we weren't able to fund them or we fucked up and decided not to fund them. I think there's just still alpha in that, especially because as we move into making much bigger grants, that can sometimes come at the expense of covering all of our bases of all the teeny little uses of money that can be valuable.

48:26And I think that's the right decision for us to make, but it does leave some things that are unfunded.

48:32Nathan Labenz: Prakash had raised the prospect of an anthropic public offering and his expectation that some of the proceeds would be distributed through coefficient giving.

48:42So what is your preparation process for handing out larger checks? Like what is your pathway for the next year? Yeah, I mean, I have no idea what's going to happen with the AI company money. In terms of like what the preparation actually looks like, I mean, I think the biggest thing is just seeding organizations now that can then be ready to absorb lots and lots of money. And so like making small grants to new groups, and then like giving them a little bit of runway to prove themselves or not prove themselves and explode. And, you know, like, that's, that's the biggest thing we're doing. And we would love to create opportunities for, you know, we would love to create basically groups of people that can say credibly to big donors, we have people, we have a project ready to go.

49:30We're just waiting for you to sign the check. We've got this shovel-ready project, and it's going to take a billion dollars, and it's going to solve alignment. And it's just a matter of giving us the money. And that's really not the situation right now in that there are just not that many people in the space. And so that's a big thing that we're trying to fix. And is that about funding people enough so that they can work on things at small scale, and such that they can get a much bigger compute check at some at some point to scale that is that is that the kind of idea that's the that's the main thing i have in mind i think the same uh logic also applies to growing the head count of their organization like you know having a group in place that has like a great plan that you know where they can then absorb like a lot of uh software engineers or ml researchers or data labelers or you know or some other uh type of person um but But yeah, I mean, I think compute, like you said, is the main version of that.

50:32And especially, hopefully, AI labor, right? You know, like, it'd be really great if there were a bunch of, like, well-scoped problems that these organizations had where they were just waiting on lots of AI labor to pour into them. Now, we'll see if we can make that come true. And I think there are reasons to be skeptical that that will happen, but it'd be nice if it didn't. Part four, the price of compute.

50:54Nathan Labenz: Wayne Nelms is co-founder and chief technology officer of Oren, which publishes a price index for GPU compute built from cleared rental transactions rather than list prices. The Intercontinental Exchange has announced plans to list futures on it, pending regulatory approval. Prakash asked him to follow a real customer. Who lends against the chips? And who loses money when prices move?

51:17So generally speaking, you have a bank or some sort of financer that will lend money against the GPU asset or a host of AI infrastructure assets. What they hope is that they will make their money back over time with interest as a result of the operating activity of a neocloud. So for example, these are companies like CoreWeave, Croson, Nebius, Lambda, etc. However, these business models themselves are contingent on really two things, right? Continued AI demands and specifically an ever-increasing or at least stable compute pricing estimate. These GPs are financed in a variety of ways, including straight line depreciation, which we can get all into.

52:05So the way Ornn provides value is that you can't really hedge away this risk without having a benchmark to hedge and having a benchmark to trade. So we created that benchmark and we are actually leading the transactions on that benchmark for these institutions. And how many transactions would you get in like a month which form the index? Yeah, so we collect over a thousand transactions a day per index. And so right now we have five public indices. So, you know, it's 150 ,000, somewhere in that range, probably per month. We have a whole host of NeoCloud partners that we work with effectively that are contributing data to our index every single day.

52:50And the question is always, you know, why do people feel the need to show that information to display that? It's because they benefit, right? A lot of these players are looking for cheaper financing. And cheaper financing comes when their financer has more certainty about the future and can hedge that risk in the future. And so it's actually this nice flywheel where you get more data, you can publish a better index, the financers feel more comfortable and are able to finance more GPUs.

53:20Nathan Labenz: I ask how fungible the underlying asset really is, whether an hour on an H100 from one provider is the same thing as an hour from another. I think it's certainly obvious when you talk to anyone deploying infrastructure or buying that an H100 or a B300 is very different depending on the OEM, the supplier, the actual everything about the hardware can be very granular and very different from an engineering point of view. However, our take has always been that there needs to be a benchmark that allows some sort of abstraction for this asset class. I think for us, what's made the job much easier is that largely speaking, we are looking at one chip producer, which is NVIDIA, which dominates the market, has huge market share.

54:09When we look at the NVIDIA moat, and we can get to this later as well, but you have the hardware, you have the software superiority, but I think the biggest thing that NVIDIA has in terms of a moat over other competitors is It's the financing landscape. When you're looking to finance your new cloud and you're thinking about what ships to put into your site, it's just materially better to have NVIDIA GPUs. It's just easier to underwrite. And I think what we see as this market problem is how do we allow every single person looking to deploy compute infrastructure, this sort of almost a protection for their lenders.

54:47We right now only reference NVIDIA reference architecture with InfiniBand, etc. There are certain steps that we're taking to standardize the compute that we're tracking. However, I will say we are tracking a subset of the market, which might inherently have differences amongst those participants. In some commodity markets, there's this idea of baseload, where you have a large amount of baseload, and then you have this idea of load that gets spun up on demand. But how do you differentiate the pricing between the kind of baseload versus the swing capacity? Because I imagine the more month-to-month transactions are the swing capacity and not the baseload itself.

55:36Yeah, I love this question. So yes, you're exactly right. A lot of the capacity is long-term PPA style contracts. And I think it's really interesting to think why that is, right? Like we're in this situation where AI capacity is so scarce. I think the model providers, so OpenAI Anthropic have seen this coming forever at this point. And partly they're to blame for the lack of supply that's currently out there. But if you think about it for them, it's an arms race, right? Compute capacity planning is an arms race. How much capacity can I lock up over the next few months? So when I need to train the next model, I have enough and I don't have to go out and looking for more.

56:16It also, of course, is naturally a zero-sum game where whatever you buy, your competitor can't. So actually, the way we like to think about the baseload and swing slash day ahead market structure is in compute, people are buying to fill the peak demand and then selling to satisfy the troughs. Where in power, you are buying to satisfy the troughs of these long-term contracts and then buying excess to satisfy the peaks. So there's actually a robust on-demand market that will continue to grow as these trading labs are selling back excess, as inference providers are selling back access. It's this reallocation of compute question rather than purely a hedging question.

57:04Nathan Labenz: How elastic is the market? Obviously, we have in an extreme example, Anthropic reportedly has paid up, I think, even a multiple of standard market prices to rent at high scale from XAI, how would you sketch out the curve from, if I want one H100 hour right now, I pay X. What if I want 10 million? How much does the price change as my scale as a buyer changes? Yeah. So we like to think about this in three axes, right? So you have price on one axis, you have duration, so length of contract on one axis, and then you have quantity on another one. Generally speaking, as you increase the length of the contracts, the contract per hour value goes down, right?

57:57The logic here is you're buying in bulk, right? You're allowing someone to finance a data center because you're buying 10 years or four or five years of capacity from them. Generally speaking, when you increase the number of GPUs you're renting, the pricing goes up, which is almost anti this exact logic, which is, I'm buying in bulk, why am I paying more? And it's purely because the number of suppliers that can sell all that capacity at once is very small. You might be able to get a few nodes access here and there. But when you're looking for 10 ,000, 100 ,000 GPUs that are all interconnected at some level, that is very, very tough.

58:34So certainly these Anthropics or SpaceX AI deals are few and far between. They're very much OTC trades, I would say. they are indicative of what we are kind of seeing as a general trend across even smaller markets.

58:51Nathan Labenz: It is striking that like you look at the graphs and they are all trending up. So that is seemingly in significant tension with the idea that the longer you buy, the lower the prices. Like why are the sellers not AGI pilled enough to expect that they'll command a higher price for the same chips next year than they are right now? Look, look, I think the sellers of compute are the most AGI people because they have entered a business where they are selling at least a resource that hopefully rises in cost in the future. But for a lot of sellers, it's not because they don't want to. I think there's a lot of suppliers of compute that are experimenting with shorter contracts because they know, why am I locking in a four-year deal if in two months, that capacity is worth even more.

59:45The issue here, of course, is risk. It's always this risk reward ratio or balance that they're trying to play, where even early financing is contingent on having a long-term contract signed. I cannot finance a data center today if I don't already have five years of offtake from a credible counterparty signed. And because of that, I can't even enter into the month-to-month world. So if you actually think about what the futures exchange and our product enables, it enables an ability for, instead of the NeoCloud locking up a five-year contract, they can sell month to month and hedge the future price potential on our platform or through our index, right?

1:00:26And that is potentially a much better risk return kind of math for them and for their TCO and for their lenders. So I think the one thing that, I think Nathan has also alluded to is that you have differences in pricing of near term and much further out, like longer term. And you kind of described the use of a lot of the hedging as tools for financiers. And the financiers are often concerned with the far end, not so much on the near end where they have much more visibility, but the far end where you have this depreciation issue. How do you think the index assists or will assist in the future on this end-of-life kind of depreciation issue, which is like four years out, three to four years out?

1:01:15Yeah, 100%. Like you mentioned, a lot of the risk in this space is certainly, I would say, long-tail risk in terms of looking at the tenor. And so when you launch these futures on a futures exchange, a lot of the trading can occur in that long tail section of the curve. These are monthly contracts on ice, to be very clear. We expect to see lots of trading in the front month as people are buying and selling, especially after physical delivery. I would expect there to also be a lot of trading in those three to five year spans in terms of that is where all the exposure is, that is where all the risk is.

1:01:58And because there's now futures contracts that reference what spot prices will look like in three to five years, that is what any bank is exposed to, right? And to be even more clear for people watching, it's, you know, you have a bank underwriting a deal where the GPU life is six years, but the initial contract signed might only be four, right? Or it might only be five. And so there's always that, you know, what happens in year four, what happens in year five, what happens in year six, when there's not a contract already signed for that GPU, but on your balance sheet, you've claimed that the GPUs have some value.

1:02:32So that is the area where you want to hedge your bets.

1:02:37Nathan Labenz: Prakash raised basis risk, the gap between what an index says and what the asset you actually hold is worth. If that gap is steady, a trader can price it in. But the issue is going to be if there's non-systematic basis risk. And one of the issues with GPUs is this introduction of new technology, which obsoletes the old, which is what I think the non-systematic basis risk that I think is pretty scary for financiers. we are tracking gpu specific uh transactions so we have a b300 index we have a b200 index and so actually this entire risk of you know a financer being worried about obsolescence risk that is exactly what you can hedge away with this instrument that is exactly what you can kind of look at how the market is pricing the 12 month or the 15 month future contract for the b300 right?

1:03:34It actually, if anything, allows information to be spread wider, broader, and more evenly, because now you can kind of see perhaps where the market is pricing the announcement of the next chip, right? Based on how the curve looks in the future. I think all these things are ways for information to be actually shared more freely rather than for there to be any obfuscation of this information. And so in effect, I think these markets are a great way to hedge that risk away.

1:04:05Nathan Labenz: I also asked how long these chips actually last and whether the A100s from 2020 are near the end of their useful life. Yeah, I think the chip health is a super interesting space to be in and understanding what are the failure modes for the hardware itself. I think chips will fail all the time for a variety of reasons, but I think what you're pointing to is the useful life conversation of hardware. And this is something that we wrote a paper on just two weeks ago, I think now, about how open source, open weight models that are more price elastic will often route to places where the cost per intelligence or cost per workload makes more sense.

1:04:48And oftentimes those are the older pieces of hardware because they're just less marketable. And so we actually see the useful life extending beyond six years as a general thesis. How much further than that. It's kind of unclear. However, I will say that what we see in the forward market and the term structure is that there will always be some sort of premium above zero, certainly, and above some terminal value of power. The terminal value of an A100 is not just the value of the power that powers it because the A100 will always provide some computing power as a fraction of all computing power in the world, right?

1:05:28Over time, I think you have a lot of interesting use cases like physical AI, one of many, where these older chips might become more effective. However, today we see still that these older chips are still being used, still being rented, and have maintained relatively constant demand in the last 12 months.

1:05:48Nathan Labenz: Wayne signed off a few minutes later. In the closing, with just the two of us, Prakash came back to the index with an account of how the biggest compute deals get priced. The figures are his, from what he has heard. What I've heard is that your price of compute is dependent on whether you have financing or not. So if someone needs to use their balance sheet, they get to dictate the terms. So if they don't, if you don't have to use a balance sheet, then all of a sudden, like, you have high pricing. So Elon, for example, Elon had already used his own balance sheet to build out the clusters, and he had the clusters available.

1:06:28Available, you can take it tomorrow, right? And they didn't need to provide him a long-term contract. They could do a month-to-month contract, and they can walk out at any time, right? So which meant that this is not financeable. A bank cannot come and give Elon a loan based on this contract from Anthropic because a bank wants to see, like, if you're going to pay back in four years, I need to see that you're going to have cash flows for four years. I need to see that up front. So Elon is able to charge high prices because he's using his own credit capacity. He's using unsecured credit at the holding company.

1:07:05And he can say, like, look, you're getting paid$80 million,$90 million a megawatt. you give me the 50 and you're still making money right so that's a good deal and so elon got 50 million dollars a megawatt right so i think it changes it starts to change when you need the other guy to give you support so if you say like okay if elon said okay i'll build it to you i'll build it for you in a year or year and a half or two years and in order for me to build it to you you need to give me a five-year contract and if i build it for you you will definitely buy the compute and pay me. And then I'm going to take that and I'm going to go to another bank and I'm going to give it to them and they're going to lend me money based on that.

1:07:49Then it becomes like, it's not really like Elon's money. It's really like Anthropik's money anyway. And the single credit is going to be from Anthropik's side. And that's really who the bank is relying on for repayment. And so then it becomes a different pricing. So then Anthropik will say like, okay, how much is it going to take you to build? How much is your interest rate going to be? and then I'm going to give you, let's say, a 20 % profit margin on top of that. So this is what CoreWeave is getting, for example. So if it's going to cost you$15 million or$20 million to build, they'll be like, all right,$24,$25 million per year.

1:08:24And so that's the key difference is whether you have the money or not up front. And those deals don't hit the index pricing. The index pricing is based on willingness to trade. So these are all tradable prices. And that stuff doesn't get traded. And so the people transacting on this index are the lowest payers. They're not Facebook. Facebook can afford to pay like$100 million a megawatt. They're not going to be trading on their own index because the own index is for people who are willing to sell compute at like$10 million or$15 million a megawatt. They're willing to sell the compute. Facebook isn't going to sell you compute at like$15 million.

1:09:03They're willing to buy it at$50 million. So these are, in some sense, the lowest pricings, but it's immediately available.

1:09:15Nathan Labenz: Part 5. What the machines are actually doing. Nick Gillian is co-founder and chief technology officer of Archetype AI, and before that, led machine learning research on Google's Soli radar sensor. Archetype's model, Newton, is a foundation model for sensor data. It takes raw streams from radar, vibration, electrical current, and cameras, and reports what a machine or a worksite is actually doing. I asked him to walk through Newton from the bottom up.

1:09:45Nathan Labenz: Let's start with the data. How much data is out there that you were able to just collect on your own? And then I imagine you must have had to establish a sort of partner network to bring in a bunch of data. Tell me about the data process. Yeah, so we're getting close to a billion hours now of physical AI data that we've been able to scrape and gather and collate. In many cases, if you look on the web and you look at the different types of physical data that's there, it's very sparse. There's very little multimodal data. It's not synced well. It doesn't cover things well. For most techniques, particularly classic supervised techniques, where you're basically building one model and it has to have a fixed number of sensors and they all have to be captured with the same sample rate, etc.

1:10:32That's a massive problem. We've been trying to flip that on its head and say, if we have all of this physical data, which is captured in different sensors and different systems with different contexts, how do we actually build the most advanced self-supervised techniques that can pull in that sensor data and extract from it? In the same way LLMs have done with language, we are able to then join that and mix that in with a small amount of data we can capture from partners and even in some cases that our customers can give us to be able to really extract and understand and map that to physical language and map that to physical control.

1:11:11So there's quite a bit of resampling and really understanding missing values in sensors. Is that because the sensor had an issue? Or is actually the machine itself is connected to? That's actually a feature that helps explain that machine is about to break. The reason the sensor has given you all these nons is not because the sensor is broken. It's actually a feature of the machine in most cases.

1:11:37Nathan Labenz: so the area that i've studied best that has done something kind of similar has been at the intersection of language and just images you know photographs yeah um and we've seen kind of if i had to borrow some of their terminology and map it onto what i understand you're saying this would be sort of a late fusion approach where you have a language model you then do like a pre-trained from the ground up encoder, specialist encoder for these different modalities. And then you're doing some sort of like cross attention between those. That's my understanding of like, you know, at a high level, obviously of what the image folks have done.

1:12:16Nathan Labenz: How much have you been able to follow in their path? How much, you know, of the techniques you've seen them develop have worked for you and how much have you had to go back to the drawing board and reinvent? We've definitely tried these approaches and we've had some success with them with, you know, other types of sensors radar and time series and so forth but the big uh the big issue that we that we have found is um twofold number one there's a there's an extreme amount of um language sensor pairs on the internet that can easily be scraped and built into these very large data sets where you have this uh pairing between an image and a description of that image right this does not exist for radars, it does not exist for time series sensors, it does not exist for all the complex systems that we go and talk to our customers with.

1:13:03So we have had to look at different approaches of how do you, for example, take time series and actually align that with human language. Right? So that's a large part of our research work is how to actually solve this sensor alignment problem. And there's also the case there where with a single image, you may actually be able to get the full context, for example, of the scene and then align that with language. But actually with the time series, you actually need to look at a temporal window and, you know, what is a single latent mapping to a single word really mean in those cases, right? It's not like you have a time series signal that says dog and you can align that with the keyword dog and so forth, right?

1:13:39So there's quite a bit of work we need to do there to solve this alignment problem of how do you generalize this across many sensors? And that's the key kind of components we're building inside of our models, which makes them different from, you know, a standard VLM or other techniques, right? Because we simply don't have that corpus of, say, time series and language pairs to kind of apply that previous recipe you mentioned, right?

1:14:08Nathan Labenz: Prakash asked about one deployment, a project with Kojima, the Japanese construction company, and what the inputs and outputs of the model look like there. in that case uh you know we're working with kajima this very uh uh established uh japanese construction company um uh they they build you know islands they move mountains the the project we were working with them on uh was literally taking a river and moving the river right so that it wouldn't flood right so um it was a five-year-long project they uh had sensors the whole way down the river as they kind of moved down the river and they're dredging it and they're widening it and you know they're basically mitigating the risk of flooding right and um the big problem they wanted to solve is um the the their company is outsourcing and subcontracting a lot of the work to a lot of other sub teams and they want to basically measure uh to make sure that those teams uh if they have to go and dredge the river for five hours did they actually do five hours of dredging that day or actually they only did two because maybe the the the the the the the The digger was in maintenance mode that day, for example, and they couldn't do it or because of weather conditions and others.

1:15:17So for that project, basically the input to Newton, our world model, was the multiple cameras around the site, typically three or four cameras looking at a specific section of the river. And when you look at those cameras, basically you have a barge, this boat that's floating out on the river. On top of it, you have an excavator digger. and then on top of that at the end of it it has a end effector that is either drilling or it has the bucket on it where it's actually dredging right they want to know how many uh hours essentially that excavator is actually uh dredging the river not drilling the river not a crane not other things right so essentially the input to the system is the sensor cameras the river sensor hydrometer sensors that basically say how high the river is and then at different geo locations along that I think it was multiple kilometer site where the actual work was supposed to be happening at that day and what the weather was that day so we basically have a bunch of cameras we have a bunch of time series sensors with weather and river information it's fed into the model and the model is essentially outputting what looks like a Gantt chart, right?

1:16:36It's actually outputting the state of that team. Where is the barge? Is it parked on the side of the river? Is it actually moving into position, which can take an hour sometimes to do that? Is it now actually in position and they're starting to dredge or is it just there and it's stationary and nothing's happening, right? And then they can start to map these work charts over multiple years of data and start to look at these bigger patterns that a Schumann can never spot. Like one of the patterns they find, for example, is obviously if there's a storm, the work on the day of that storm goes down.

1:17:08That's kind of, you know, everyone would know that, right? But one of the key things they find is multiple days after the storm, when the weather had cleaned up, the work productivity of the team was still very low. And the reason for that being is the river water coming down from the mountains, It takes several days for it to actually reach the end of the river and go to the ocean, which is where the site was. So there was a lot of debris and a lot of churn coming through the river multiple days after these large restaurants, which is actually slowing the teams down quite a bit. They'd never spotted those patterns before because they could never look at these aggregate patterns.

1:17:42But Newton was able to kind of spot those patterns for them, right? How much does Newton generalize in the sense that when you go into a new site, you have to take all of the data and kind of fine tune the model once more? Yeah. So we are building the model so that out of the box, you can plug in different sensors and different configurations and the model can automatically adapt to those as much zero shot as we can. for a subset of users, they can directly do that. The typical analogy I give customers is, if you showed this task to an average person from the street, would they be able to detect the safety violation?

1:18:19Or would they be able to detect that that looks like a strange signal on an anomaly? If the answer is normally yes, normally that means that Newton can do it out of the box. It doesn't need a specific master technician's level of expertise yet to be able to spot that. For those cases where maybe the piece of equipment the user is using is very specific to that factory, for instance, or they have certain parts that they have very internal and unique names for those processes or parts. That's typically where fine tuning or being able to bring the customer's knowledge base and actually being able to inject that securely into Newton really helps.

1:18:56So in that case, our platform allows customers to take our model, run it directly on the customer's infrastructure so no data is leaking out of their networks, either on-site, on-prem, or in their secure clouds, and then be able to take their own data and actually do some amount of fine-tuning to be able to customize the models. And typically with customers we've seen, on the order of a few thousand samples is enough to be able to take the general model and then adapt it very quickly to that customer's use case.

1:19:24Nathan Labenz: Last question for me is about your vision of super intelligence and how you might relate to the frontier companies as we go forward in time. What I think is, geez, the reasoning is already getting to like super intelligent light, you know, with all these Millennium Prize problem style results happening just through super long chain of thought, like who would have believed you could get there. So on the reasoning front, we've kind of come awfully far. What we don't seem to have still is like a good intuition for the many domains that matter. And what I sort of expect to happen is something similar to what has happened with images where, you know, it's one thing to sort of put an image into a model and get a caption out.

1:20:11Nathan Labenz: But when you can get the altered image out and it's clear that the model understood both your verbal instructions and also sees the image in a very similar way to the way we intuitively see the image, which is something we are really good at, then it feels like, wow, okay, it's really got that domain. so my super intelligence vision is basically do that across a ton of different domains many of which we don't have native senses for with what you're building being a great example of that right people don't have a native sense for how to integrate 10 signals from 10 different sensors do you see a role for you in super intelligence being sort of building the deep intuition for this domain?

1:20:56Nathan Labenz: And does that ultimately get joined with like elite reasoners via the form of like a bidding war between a couple of frontier companies for your company, for example? I definitely see the part that we can play in a larger super intelligence system. So that's kind of how we're seeing, you know, our, our, where we play in this kind of bigger super intelligence loop is like, we're very focused on how to understand the world around this and be able to bring that back to natural language and machines so that these things can talk to each other. If we can do that and other folks are solving digital intelligence, that nicely will connect together very well.

1:21:34Nathan Labenz: So just to make sure I understand your answer there, it sounds like you are envisioning a world where the technology that you're building remains sort of a tool call type relationship to a core reasoning model and doesn't so much get upstreamed and thrown into the GPT-7 pre-training mix to spit out one model that does everything in the same latent space? We're thinking more on the line of we're building physical agents that can be deployed out to the ecosystem of the world. Those physical agents then can talk to other agents and be able to bridge that gap between the digital world and the physical world.

1:22:13How much that's actually done on an agent-to-agent post-training mechanism, how much of that is pulled into the pre-training, we get to see, particularly when it comes to those more nuanced sensors like radar and LIDAR and infrared and all these things that don't look like human visible images. That'll be interesting to see how that's brought back into these kind of bigger print pre-training loops versus just it's more of an agent that can go and solve that problem and tell you what's happening in the physical world in real time and be able to let you interact with it, for instance.

1:22:50Nathan Labenz: Part 6. The dish is not the body. Andrei Diorgescu is co-founder and chief executive of VivoDyne, which builds robotic labs that grow pieces of human tissue, lung, liver, tumor, and more from primary human cells and tests drugs on them. In August, it opened what it calls a human biological data center, 12 robotic labs that, by the company's count, can grow more than 3 million tissues a year. Prakash had the first question. How do you actually figure out what the dosage is and whether or not that is actually what the cell would encounter in the body? Basically, everything that's been done pre-clinically to date, outside of testing in animals, you are just dunking some substrate, whether it be an organoid or you have cells in a petri dish, just basting them with this test compound that you want to test.

1:23:47And that's exactly the problem is like that's not how transport happens in the human body, right? And I think as a perfect example, a lot of cell therapies that are designed for solid tumors in a dish will completely kill that tumor. And then those same CAR T cells, when they go into a patient, just flow right by the tumor through the blood vessels. They don't even know it's there, right? And thus is the challenge is modeling the transport of that therapy into the tissue. Do they recognize the tumor? Do they concentrate there? And so in vivodyne tissues, even the blood vessels, the kind of capillary bed in that tissue is self-assembled.

1:24:24So these little blood vessels grow within the tissues, and we can perfuse directly through them. And whenever we dose a drug, it is always through the blood vessels that are native to that tissue. And so the transport through the interface of the blood vessels and then the, you know, like meat of the tissue, right? All of its like parenchymal, its functional subparts is just like you would find in a normal tissue. The density is the same, right? And so that is just as big a part as does the drug molecularly like bind to its target and perform this effect as does it even get there? Then my question on scaling laws.

1:25:03Nathan Labenz: I'm very intuitively bullish on this, just go collect an obscene amount of data on this kind of tissue perturbation paradigm. It makes a ton of sense to me. The skeptical view that I've heard from time to time is you can never have enough data. The world is too big, biology is too complex. You can do that, but you just won't be able to generate enough. The challenge of this argument that it's just like this, just a size of data set thing is that currently even the state of the art virtual cell models that exist saturate at a very, very small fraction of the input data that is fed to them. So you can have all the data you want, but their performance saturates after a couple percent, right?

1:25:51And this is like, you know, both published and kind of what we hear on the ground, right? And in these perturb-seek kind of based models. And the reason it gets so difficult is when cells are growing in a dish, which is the kind of mode for this sort of perturb-seek, they are so far removed from all the feedback loops that are natural in people that they're just trying to colonize that piece of plastic, right? It's like you have a stiff substrate, normally cells when they encounter that are like, I have to encapsulate this. And so they are just growing. Their only goal is to proliferate. And so if you knock out genes in those cells that would otherwise affect how they perform their natural function, really, there's no big difference, right?

1:26:31They're still just dividing. And so you end up learning this view that perturbations don't really do much of anything, or it's unclear what they do, except if a gene is like required for that cell to live, in which case, like it kills the cell, right? And so and so it's not, there's not even causality to extract from that. And so we think that the fundamental bottleneck to having this causal understanding of biology is that we need to perturb these tissues in the complex environments that they call home naturally, right? Almost like the image in my head is we are trying to populate a map and kind of like design a GPS through it.

1:27:11So we can say, if I want to get to the state, I need to, you know, like go down this road and then turn left with a second perturbation and that'll bring me to this healthy state.

1:27:21Nathan Labenz: I asked how you get from a sample received to something placed in the system and growing. And when we grow these tissues, you know, a really big differentiator for our approach to doing this, and I think the, you know, really like a critical mechanism that you need to get like realism in this, right, is that we use primary cells, which are mature differentiated cells that like perform our natural organ function in our bodies, as opposed to starting with some stem cell, completely absent from all the conditions that would tell it how to mature into a differentiated cell. And then like sledgehammering it with cytokines to turn into some cell type that we think is accurate to what happens in the body, right?

1:28:03We instead just kind of go to the mature source, the ground truth. And we mix together all of these different cell types that comprise the lung, for example, a very high density. And within that injected tissue, we see that these cells begin to reassemble into the native structure of that human tissue, right? With some little herbs and spices that we introduced, but the, the, the, the, like responsibility for forming the native structure of human tissue is on those cells that already know how to do it. It's honestly kind of injection molding into a, into a, like into a chamber, right? So we, we like shoot in these, these cells at like a bit high density.

1:28:48And then over the course of several days, they reassemble themselves. They just self-organize to make blood vessels to partition into, in the case of the lungs, like the airway where there's air and the parts where there are blood vessels and fibroblasts and immune cells and all of this. So we are relying extremely heavily on the cells to be able to direct their own self-assembly at these very small scales.

1:29:11Nathan Labenz: Can you tell us a little bit about the mix of in silico via the foundation model versus in cell in actual tissue experimentation today and like how that ratio is expected to evolve because presumably like the big value is it's a lot cheaper faster even if you've got all these great wet lab technologies just plowing things through the foundation model is supposed to be where you get like the yeah insane acceleration right but then of course you've got to ground it and validate that's exactly right so so so um if we if we look at it like a reinforcement learning problem, which is, you know, my actor is this, well, it's kind of the experiment, right?

1:29:54I have this tissue and I can perturb it in very many different ways. And the goal is I have this objective, which is to make it healthy, and I can instrument the state of that tissue very deeply. So I know, like the value function is very well defined in terms of the state of that tissue. The question of what should I try next, right? In the large rollout that I can perform, what are the most useful things to try next that add the most information to my data set? So the physical experiments kind of serve to fill in gaps in which the model is most uncertain. And the value of the model ultimately, right?

1:30:35Like why even make these foundation models is the combination space, especially when we move into two target or three target therapies, right? If we have just combination therapies, if I take two drugs, because it's going to work much better than, than one is, is so huge that you have, I mean, you could have like this whole planet could be, you know, Viva Dine systems growing, growing human tissues, and you'd still not get anywhere close, right? To the, to the, just sheer space of things to, to test. And so we need some way to narrow down to, you know, to, to, to kind of just, you know, a couple times like the total number of clinical trials in America, the experiments that we should physically run to kind of test our hypotheses, right?

1:31:21And so basically, these foundation models allow us to predict what happens when we drug this thing and that thing, or all these things together, or, you know, this thing, and then that thing. And we kind of optimize and we iterate in that space. And we find, for example, the thousand most likely things that will, you know, move this tissue closer to this healthy state. And then we close the loop by testing it, right? And we see where that rollout goes. And then we just do that on a loop, right? So the purpose very fundamentally for us of these models is to tell us what to do next. And then the purpose of this experimentation is to figure out if the thing that we thought we should do next actually does move us closer or how it deviates so we can correct.

1:32:06Nathan Labenz: Prakash asked about the hardware. The tissues grow on what Vividyne calls a tissue disc, and the company recently announced a second generation of it. He asked what changed from the first disc to the second. Andre started with the robotic lab the discs go into. Imagine something the size of like a wardrobe, right? So it's about the footprint of a large desk about eight and a half feet tall and these are self-enclosed labs so fridge and freezer and there's a you know 3d scanning microscope in there and you know big robot arm that can use all these different tools you want to optimize the surface area of those discs for tissues and so in our in our second form factor we have an increase anywhere from two to four times the density of tissues that we grow on each disc without any sacrifice to they're actually much larger than the version one of these disks.

1:32:59So we increase the number and the size of these tissues, and we offload a lot of the responsibility for how they're grown to the robotic system around them. Now, the challenge there and the kind of work that allowed us to do this is a lot of work in software on the orchestration layer. You think if you had like 10 ,000 experiments to run in parallel, and this is not like I have a different gradient of dosing for some drug or something, right? I can do it on a plate. It's like I am changing the collection point of data. I'm changing the interval between dosing with some drug. I have a first stage regimen and a second stage therapy.

1:33:38I'm taking all sorts of different readouts of these tissues. It becomes a super, super complicated experiment. And I can define it quickly by, I can explain it out loud and say, well, we want to test bone marrow and airway. We want to dose these drugs in this order. we want to branch through these concentrations and I want 3d imaging, single cell sequencing, and proteomics. And by saying that I can define the study decently unambiguously, right? I mean, like given the system, the tissues that we grow, it's pretty unambiguous what I would want to test in there. But then the robot has to be able to decompile or to compile that kind of zipped representation of this study, it has to decompress it into this compiled list of millions of actions that this robot has to perform.

1:34:26It has to remember that if it took the lid off of a plate, that it should put it back, right? It is an example. But if there's multiple trays that it has to handle and they're all out on the working deck, maybe it should delid multiple of them in advance, right? If multiple pieces of plasticware are on the same tray and they're coming out of the freezer to thaw, you want to make sure you're not thawing something that should not be thawed too early. And so all like this, this, this compiler behaves a lot like a compiler in code, right? For C where you're managing like the prefetching of information for memory and branch prediction, like all this stuff, right?

1:35:05That, that, that thankfully our operating system and hardware handle for us. But improvements on our orchestration mean that the system itself, the hardware, the automation can do much more of this. And more and more of that space on these tissue discs can be used for actual tissue biology instead of having to integrate the little helpers on there.

1:35:28Nathan Labenz: Toward the end, Prakash read Andre a grand challenge for the field, which he credited to the Annual Review of Pharmacology and Toxicology. On the annual review of pharmacology and toxicology's grand challenges in modern pharmacology, the innovation, a universal in-silico or organized framework that forecasts rare, patient-specific adverse events with more than 95 % accuracy before first in-human dosing is one of their target challenges. VivoDyne is obviously on the path there. when do you think this challenge will be solved? So this is a hot take, but I feel like this is almost like picking the color of my Mars suit currently to me, right?

1:36:20Which is like, how about we just kind of fix the bigger problems? And the big problem, by the way, we are working on specifically that challenge, right? Of like very, very rare outcomes. But a bigger, like far more regularly, We have drugs that would have huge potential in their efficacy in patients, but that have routes of toxicity that are pretty common. Like many patients die from it, right? I mean, many of the cancer drugs that are going to clinic today, the risk is not like, oh, man, you know, I've cured 999 people of cancer. But this one patient, like, it didn't work so well. You know, it is more toxic.

1:36:58The problem is, in all the patients we've dosed, they barely had any improvement in their cancer, but it gave them all of these side effects or killed someone from liver toxicity or something.

1:37:10Nathan Labenz: Andre signed off a few minutes later. In the closing with the two of us, Prakash put the two guests side by side. I liked how you had basically two different approaches for data. You had, you know, on the one side, you had let's take all the messy data. Let's feed it into the machine and try to make sense of it. On the other, Andre is like, there's just too much messy data in biology. I'm going to build my own data collection effort, and I'm going to monitor every single thing that goes in and comes out so that I know exactly what's going on. And I think Andre probably has a good chance of solving this thing.

1:37:48You need a little bit more GPU power. You need a little bit more imaging. I think the imaging granularity is probably not there yet, but you can definitely see if they manage to scale a little bit more, The real question is, do you need to do 3 billion samples, 30 billion samples a year? Do you need to get everyone's genetics? Or is your scaling going to be able to get you to a point where at 30 or 300 million or wherever, it just starts to generalize a lot more and you stop needing so many different samples?

1:38:23Nathan Labenz: Prakash guessed it would be two or three more years before AI gives biologists something they can really use. I took the under. I think we're going to see probably a reversal of the order of like utility and millennium prize solutions in biology relative to what we've seen in math. The issue with math is that we already had a ton of math that was like interesting to mathematicians, but not super useful. And so to push the frontiers of math, you kind of had to get into the like not useful domain of math. In biology, we have like tons of useful stuff that is like still not even that well understood, right?

1:39:02Nathan Labenz: I mean, it's just very, it's a far more empirical than theoretical domain. And I would expect a tremendous amount of utility to emerge before we have anything approaching like a full, you know, mechanistic or fully causal understanding of what's going on in biology. And to some degree, you know, obviously we'd love to have that, but we don't necessarily care, right? At the level of like, does this cancer drug work on this patient's tissue? If it works, you don't have to have a mechanism, right? That's not part of the clinical trial approval process. It's nice to have, not required. Prakash also made the case for specialists.

1:39:42Nathan Labenz: A model like AlphaFold does its one job better than any generalist. He expects that to hold. and the version of superintelligence people fear is a single model that does every job better than the specialists and depends on no one. My answer was the pattern as I see it so far. New kinds of data get pulled upstream into the biggest model and efficient specialists get distilled back out of it. But yeah, put me down for one that has not seen a reason yet to believe that all these modalities don't end up integrated at the frontier. In a way, I wish it wasn't going to go that way. I do think safety through narrowness would be really nice.

1:40:20Nathan Labenz: That's why I'm excited about JEV in part, you know, because it's sort of, it has a very niche role to play. You could build all kinds of scaffolding around it. I was a big fan of the Eric Drexler reframing super intelligence piece, which he also known as comprehensive AI services, where you've just got narrow purpose-built AIs for all kinds of different niches. My hope for safe superintelligence is something like that, just based on reading the tea leaves of Ilya's comments. Could we possibly get a model that starts as a stem cell generalist and then matures in a way where it gets really good at its job, but also settles into that niche in the way that mature cells don't revert and turn into other kinds of cells and that they're limited now?

1:41:07Nathan Labenz: They've lost their pluripotency, so to speak. um i love all that stuff i will like i hope it goes that way for efficiency it probably will but at the frontier i just have not seen anything at all yet that makes me think that the the single biggest best teacher model you could make couldn't just learn it all and that you know given what we've seen that that starts to be a bit of a scary beast oh i i agree it can learn it all but i think learning it all at the most efficient compute rate and inferring at the most efficient compute rate, which is what you alluded to with the distillation, I think that doesn't happen.

1:41:49So the energy efficiency is kind of key because the energy efficiency means that you have this diminishing returns curve as in economics for getting bigger. Okay, the doomer scenario is less likely to happen because you will get a more intelligent model, but it's going to be less efficient to start out with and it's going to be more energy consumptive. And that means that it's going to need like more energy expansion to get there. But the other models are going to be more efficient than it and more numerous. But I think the singleton example, I think, is not likely to happen.

1:42:20Nathan Labenz: From your lips to God's ears, as always, I think that's a real constraint for sure. I mean, if you made a, you know, let's just get ridiculous, right? You made a quadrillion parameter model. You certainly would find it a bit slow and a bit expensive, and you couldn't run a million agents at that scale of model today. So I do think there are some practical limits there. But then again, remember what we talked about on Wednesday, there's going to be more compute installed over the next 12 months than exists currently in the world now. So we are definitely going to test for at least a couple, few more years, whether just bulking up on compute solves all, you know, quote unquote solves or perhaps creates all these problems.

1:43:10Nathan Labenz: That is the week. Tell us what worked and what did not. See you in the morning.

1:43:23I was doing it that night Two rooms and one mine, one mine Cut in two You didn't have to call me, I knew, I knew Mama made us in the winter One and one a breath of high Gave us near the same name Gave us the very same mind You went east where we were ready I went west and stayed And every choice I ever made You'd already made Didn't have to call, didn't have to write Whatever you were doing When I was doing it that night Two rooms and one mine, one mine cut in two You didn't have to call me, I knew, I knew When they came around asking who'd been where we weren't allowed I said me in my room, you said me in yours out loud Two confessions, one back door and mama let it go What can you do with the two of us?

1:44:16I know you know People say spooky, people say strange But if you know how I made, you know what I'll say If you know how I made, you know what I'll say When they said one of us would go, and one of us could stay I didn't have to ask you, you didn't have to say I'm the one who breathed the head, so I said Me, not you, my room went dark, and you're stayed on I knew you knew Didn't have to call, didn't have to write Whatever you were doing, I was doing it that night. Two rooms and one mine, one mine cut in two. You didn't have to call me, I knew I knew. I knew you knew.

1:45:14Nathan Labenz: If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guests and topic suggestions, and sponsorship inquiries, either via our website, CognitiveRevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting.

1:45:49Nathan Labenz: If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at AI podcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.