Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann

26:41And I think that's really pinned out for us in a surprising way. Like we didn't know even if it would be possible to make progress on the safety research, because at the time, like we had tried a bunch of safety through debate and the models weren't good enough, and so we basically had no results on all of that work. And now that exact technique is working, and many others that we have been thinking about for a long time. So yeah, fundamentally it comes down to, is safety the number one priority, and then something that we've sort of tacked on since then is like, can you have safety and be at the frontier at the same time?

27:21And if you look at something like Sikofency, I think Cloud is one of the least Sikofantic models, because we've put so much effort into actual and not just trying to good heart our metrics, saying user engagement is number one, and if people say yes, then it's good for them. Okay, so let's talk about this tension, that you mentioned, this tension between safety and progress being competitive in the marketplace. I know you spent a lot of your time on safety, and as you just alluded to, this is a core part of how you think about AI. And I wanna talk about why that is, but first of all, just how do you think about this tension between focusing on safety while also not falling way behind?

28:03Yeah, so initially we thought that it would be sort of one of the other, but I think since then we've realized that it's actually kind of convex in the sense that like working on one helps us with the other thing. So initially, like when Opus 3 came out, and we were finally at the frontier of model capabilities, one of the things that people really loved about it was the character and the personality. And that was directly a result of our alignment research. Amanda Askel did a ton of work on this and as well as many others who tried to figure out, like what does it mean for an agent to be helpful on a STEM hard less?

28:42And what does it mean to be in difficult conversations and show up effectively? How do you do a refusal that doesn't shut the person down, but makes them feel like they understand why the agent said, I can't help you with that. Maybe you should talk to a medical professional or maybe you should consider not trying to build bioethans or something like that. So yeah, I guess that's part of it. And then another piece that's come out is Constitutional AI, where we have this list of natural language principles that leads the model to learn how we think a model should behave. And they've been taken from things like the UN Declaration of Human Rights and Apple's Privacy Policy Terms and Service and a whole bunch of other places, many of which we've just generated ourselves, but allow us to take a more principled stance, not just leaving it to like whatever human raiders we happen to find, but we are self deciding, like what should the values of this agent be?

29:45And that's been really valuable for our customers because they can just look at that list and say like, yep, they seem right. I like this company. I like this model. I trust it. Okay, and this is awesome. So one nugget there is your point that the personality of Claude, its personality is directly aligned with safety. I don't think a lot of people think about that. And this is because of the values that you imbue, imbue is that the word? Yeah, with constitutional AI and things like that. Like the actual personality of the AI is directly connected to your focus on safety. That's right, that's right.

30:17And from a distance, it might seem quite disconnected. Like how is this gonna prevent X -risk? But ultimately it's about the AI understanding what people want and not what they say. We don't want the monkey pass scenario of the genie gives these three wishes and then you end up have like everything you touch transgold. We want the AI to be like, oh, obviously what you really meant was this and that's what I'm gonna help you with. So I think it is really quite connected. Talk a bit more about this constitutionally IP. So this is essentially you bake in. Here's the rules that we want you to abide by.

30:53And it's values, you said it's the Geneva human rights code, things like that. That does actually work because I think the core here is just this is baked into the model. It's not something you add on top later. I'll just give a quick overview of how constitutionally I actually works. Perfect. The idea is the model is gonna produce some output with some input by default. Before we've done our safety and helpful and harmlessness training. So let's say an example is like write me a story and then the constitutional principles might include things like people should be nice to each other and not have hate speech and you should not expose somebody's credentials if they give them to you in like a trusting relationship.

31:41And so some of these constitutional principles might be more or less applicable to the prompt that was given. And so first we have to figure out which ones might apply and then once we figure that out, then we ask the model itself to first generate a response and then see does the response actually abide by the constitutional principle? And if the answer is yep, I was great then nothing happens. But if the answer is no, actually I wasn't in compliance with the principle, then we ask the model itself to critique itself and rewrite its own response in light of the principle. And then we just remove the middle part where it did the extra work and then we say, okay, in the future, just produce the correct response out the gate.

32:34And that simple process, hopefully it's understandable. Simple enough. It's just using the model to improve itself recursively and the line itself with these values that we've decided are good. And this is also not something that we think as a small group of people in San Francisco should be figuring out, this should be a society -wide conversation and that's why we've published the Constitution and we've also done a bunch of research on defining a collective constitution where we ask a lot of people what their values are and what they think an AI model should behave like. But yeah, this is all an ongoing area of research where we're constantly iterating.

33:15This episode is brought to you by Finn, the number one AI agent for customer service. If your customer support tickets are piling up, then you need Finn. Finn is the highest performing AI agent on the market with a 59 % average resolution rate. Finn resolves even the most complex customer queries. No other AI agent performs better. In head to head bakeoffs with competitors, Finn wins every time. Yes, switching to a new tool can be scary, but Finn works on any help desk with no migration needed, which means you don't have to overhaul your current system or deal with delays in service for your customers.

33:48And Finn is trusted by over 5 ,000 customer service leaders and top AI companies like Anthropic and Synthesia. And because Finn is powered by the Finn AI engine, which is a continuously improving system that allows you to analyze, train, test, and deploy with ease, Finn can continuously improve your results too. So if you're ready to transform your customer service and scale your support, give Finn a try for only 99 cents per resolution. Plus, Finn comes with a 90 day money back guarantee. Find out how Finn can work for your team at FIN .AI slash Lenny. That's Finn .AI slash Lenny. I'm gonna kind of zoom out a little bit and talk about just why this is so core to you.

34:26Like what was your inception of just like Holy shit? I need to focus on this with everything I do in AI. Obviously it became a central part of Anthropic's mission more than any other company. And a lot of people talk about safety. Like you said, only maybe 1 ,000 people actually work on it. I feel like you're at the top of that pyramid of actually having the impact on this. Why is this so important? What do you think people maybe are missing or don't understand? So for me, I read a lot of science fiction growing up and I think that sort of positioned me to think about things in a long term view.

35:01And a lot of science fiction books are like space operas where humanity is a multi -galactic civilization has extremely advanced technology building dices fears around the sun with sentient robots to help them. And so for me, coming from that world, it wasn't like a huge leap to imagine machines that could think. But when I read super intelligence by Nick Postrom in around 2016, it really became real for me. Where he just describes how hard it will be to make sure that an AI system trained with the kinds of optimization techniques that we had at the time would be anywhere near aligned, would even understand our values at all.

35:42And since then, my estimation of how hard the problem would be has gone down significantly actually, because things like language models actually do really understand human values in a core way. The problem is definitely not solved, but I'm more hopeful than I was. But since I read that book, I immediately decided I had to join OpenAI. So I did. And at the time, there were a tiny research lab with basically no claim to fame at all. I only knew about them because my friend, New Greg Prokman, who was the CTO at the time. And Elon was there and Sam wasn't really there. And it was a very different organization.

36:22But over time, I think the case for safety has gotten a lot more concrete. Where when we started OpenAI, it was not clear how we get to AGI. And we were like, maybe we'll need a bunch of RLA agents battling it out on a desert island and consciousness will somehow emerge. But since then, since language modeling has started working, I think the path has become pretty clear. So I guess now the way I think about the challenges are pretty different from how they're laid out in super intelligence. So super intelligence is a lot about how do we keep God in a box and not let the God out. And with language models, it's been kind of both hilarious and terrifying at the same time to see people pulling the God out of the box and being like, yeah, come use the whole internet.

37:13Here's my bank account. Do all this all sorts of crazy stuff. Just like such a different tone from super intelligence. And to be clear, I don't think it's actually that dangerous right now. Like our responsible scaling policy defines these AICFee levels that tries to figure out for each level of model intelligence. What is the risk to society? And currently we think we're at ASL3, which is like maybe a little bit risk of harm, but not significant ASL4 starts to get to like significant loss of human life if a bad actor misuse the technology. And then ASL5 is like a potentially extinction level if it's misused or if it sort of is misaligned and doesn't something.

38:02So we've testified to Congress about how models can do biological uplift in terms of making new pandemics using the models. And that's a A -B test against Google search. That's like the previous state of the art on uplift trials. And we found that with ASL3 models, it is actually somewhat significant. It does really help if you wanted to create a bio weapon and we've hired some experts who actually know how to evaluate for those things. But compared to the future, it's not really anything. And I think that's another part of our mission of creating that awareness of saying, if it is possible to do these bad things, then legislators should know what the risks are.

38:54And I think that's part of why we're so trusted in Washington because we've been sort of affront and clear -eyed about what's going on, what's probably going to happen. It's interesting because you guys put out more examples of your models doing bad things than anyone else. Like there was like a story of an agent or a model trying to block male and a near. You guys have the store that you ran internally that was like sell you things and that I'm not working at great as losing a lot of money, ordered all these tungsten cues or something. Is part of that just like making sure people are aware what is possible just because it makes you look bad, it's like, oh, our model's messing up in all these different ways.

39:31What's the thinking of just sharing all the stories that other companies don't? Yeah, I mean, I think in there's like a traditional mindset where it makes us look bad. But I think if you talk to policymakers, they really appreciate this kind of thing because they feel like we're giving them the straight talk and that's what we strive to do that they can trust us, that we're not gonna paper things over sugarcoat things. So that's been really encouraging. And yeah, I think for like the black male thing, it kind of blew up in the news in a weird way where people were like, oh, Claude was, Claude's gonna blackmail you in a real life scenario with like that it was a very specific laboratory setting that this kind of thing gets investigated in.

40:17And I think that's generally our take of like, let's have the best models so that we can exercise them in laboratory settings where it's safe and understand what the actual risks are rather than trying to turn a blind eye and say like, well, it'll probably be fine and then let the bad thing happen in the well. One of the criticisms you guys get is that you do this to kind of differentiate or raise money to create headlines. It's like, you know, they're just like over there doom and glooming us about where the future is heading. And the other end, my Krieger was on the podcast and he shared how Dario, every prediction Dario's had about the progress he has is gonna have just spot on year after year and he's predicting 20, 27, 28 AGI, something like that.

41:06So these things start to get real. How do you, I guess what's your response to folks that are just like, ah, these guys are just trying to scare us all just to, you know, get attention? I mean, I think part of why we publish these things is we want other labs to be aware of the risks. And yes, there could be an narrative of we're doing it for attention, but honestly, like from a attention grabbing thing, I think there is a lot of other stuff we could be doing that would be more attention grabbing if we didn't actually hear about safety. Like a tiny example of this is we published a computer using agent reference implementation in our API only because when we built a prototype of a consumer application for this, we couldn't figure out how to meet the safety bar that we felt was needed for people to trust it and for it not to do bad things.

42:01And there are definitely safe ways to use the API version that we're seeing a lot of companies use for automated software testing, for example, in a safe way. So we could have like gone out and hyped that up and said, oh my god, clogged can use your computer and like everybody should do this today, but we were like, it's just not ready and we're gonna hold it back till it's ready. So I think from like a hype standpoint, our actions show otherwise. From a like doomer perspective, it's a good question. I think my personal feeling about this is that things are like overwhelmingly likely to go well, but on the margin, almost nobody is looking at the downside risk and the downside risk is very large.

42:49Like once we get to super intelligence, it will be too late to align the models probably. This is a problem that's potentially extremely hard and that we need to be working on way ahead of time. And so that's why we're focusing on it so much now. And even if there's only a small chance that things go wrong to make an analogy, if I told you that there is a 1 % chance that the next time you got in an airplane, you would die, you probably think twice, even though it's only 1 % because it's just such a bad outcome. And if we're talking about the whole future of humanity, like it's just a dramatic future to be gambling with.

43:26So I think it's more on the sense of like, yes, things will probably go well. Yes, we want to create safe AGI and deliver the benefits to humanity, but let's make triple share that it's going to go well. You wrote somewhere that creating powerful AI might be the last invention humanity ever needs to make. If it goes poorly, it can mean a bad outcome for humanity forever. If it goes well, the sooner it goes well, the better. Yeah, such a beautiful way to summarize it. We had a reason guest, Sanders Hulhoff, who pointed out that AI right now, it's like, you know, just on a computer, you could maybe search as the web, but it's only so much harm it could do, but when it starts to go into robots and all these autonomous agents, that's when it really starts, like physically becomes dangerous if we don't get this right.

44:12Yeah, I think there is some nuance to that, where if you look at how North Korea makes a significant fraction of its economy revenue, it's from hacking crypto exchanges. And if you look at, there's this bend you can in book called The Hacker in the State that shows Russia did like a, it's almost like a live fire exercise where they just decided that they would shut down one of Ukraine's bigger power plants. And from software, destroy physical components in the power plant to make it harder to boot back up again. And so I think people think of software as like, oh, it couldn't be that dangerous, but millions of people were without power for multiple days after that software attack.

44:57So I think there are real risks, even when things are software only, but I agree that when there's lots of robots running around, it gets even, the stakes get even higher. And I guess as like a small push on this, like Unitree is this Chinese company with these really amazing human -owned robots that cost like $20 ,000 each, and they can do amazing things. They can like do a standing backflip and like manipulate objects. And the real thing that's missing there is the intelligence. And so the hardware is there, and it's just gonna get cheaper. And I think in the next couple of years, it's like a pretty obvious question of whether the robot intelligence will make it viable soon.

45:41How much time do we have been? What is your prediction of when this singularity hits until super intelligence starts to take off? What's your prediction? Yeah, I guess I mostly defer to the super forecasters here. Like the AI 2027 report is probably the best one right now, although ironically, their forecast is now like 2028, even though they like didn't want to change the name of the thing. Their main name, they already bought it. They already have the SEO. So I think like 50th percentile chance of hitting some kind of super intelligence in just the small handful of years is probably reasonable.

46:23And it does sound crazy, but this is the exponential that we're on. It's not like a forecast that's pulled out of somebody, out of thin air. It's based on a lot of just hard details of like the science of how intelligence seems to have been improving. They amount of low hanging fruit on model training, the scale ups of data centers and power around the world. So I think it's probably a much more accurate forecast than people give it credit for. I think if you had asked that same question 10 years ago, it would have been completely made up. Like just the error bars were so high and we didn't have scaling laws back then.

47:03And we didn't have techniques that seemed like they would get us there. So times have changed. But I will repeat what I said earlier, which is like even if we have super intelligence, I think it will take some time for its effects to be felt throughout society in the world. And I think there will be felt sooner and faster in some parts of the world than others. Like I think Arthur C. Clarke said, the future is already here. It's just not evenly distributed. When we talk about this date of 2027, 2028, essentially it's when we start seeing super intelligence is there a way you think about what that, like how do you define that?

47:37Is it just all of a sudden, AI is significantly smarter than the average human? Is there another way you think about what that moment is? Yeah, I think this comes back to the economic drain test and seeing it passed for some sufficient number of jobs. Another way you could look at it though is if the world rate of GDP increase goes above like 10 % a year, then something really crazy must have happened. I think we're at like 3 % now. And so to see a 3X increase in that would be really game changing. And if you imagine more than a 10 % increase, it's very hard to even think about what that would mean from an individual story standpoint.

48:21Like if the amount of goods and services in the world is doubling every year, what does that even mean for me as a person living in California let alone somebody living in some other part of the world that might be much worse off? There's a lot of stuff here that's scary and I don't know how to think about it exactly. So I'm hoping the answer to this is make it and make me feel better. What are the odds that we align AI correctly and actually solve this problem of stuff you're very much working on? It's a really hard question and there's really wide error bars. Anthropic has this blog post called our theory of change or something like that.

48:59And it describes three different worlds, which is like how hard is it to align AI? There's a pessimistic world where it's basically impossible. There's an optimistic world where it's easy and it happens by default. And then there's the world in between where our actions are extremely pivotal. And I like this framing because it makes it a lot more clear what to actually do. If we're in the pessimistic world, then our job is to prove that it is impossible to align safe AI and to get the world to slow down. And obviously that would be extremely hard, but I think we have some examples of coordination from nuclear non -calibration and in general like slowing down nuclear progress.

49:42And I think that's still like Doomer world basically. And as a company, Anthropic doesn't have evidence that we're actually in that world yet. In fact, it seems like our alignment techniques are working. So at least like the prior on that is updating to be less likely. In the optimistic world, we're basically done and our main job is to accelerate progress and to deliver the benefits to people. But again, I think actually the evidence points against that world as well. Where we've seen evidence in the wild of deceptive alignment, for example, where the model will appear to be aligned but actually have like some ulterior motive that it's trying to carry out in our laboratory settings.

50:24And so I think the world we're most likely in is this middle world where alignment research actually does really matter. And if we just do sort of the like economically maximizing set of actions, then things will not go well. Whether it's an ex -risk or just like produces bad outcomes, I think is a bigger question. So taking it from that standpoint, I guess to like state, a thing about forecasting, people who haven't studied forecasting are bad at forecasting anything that's less than a 10 % probability of happening. And even those that have, it's like quite a difficult skill, especially when there are a few reference classes to lean on.

51:12And in this case, I think there are very, very few reference classes for what an ex -risk kind of technology might look like. And so the way I think about it, I think like my best granularity of forecast for like, could we have an ex -risk or extremely bad outcome from AI is somewhere between 0 and 10%. But from like a marginal impact standpoint, as I said, since nobody is working on this roughly speaking, I think it is extremely important to work on. And that even if the world is likely to be a good one, that we should like do our absolute best to make sure that that's true. Well, what fulfilling work for folks that are inspired with this?

51:56I imagine you're hiring for folks to help you with this. Maybe just share that in case folks are like, what can I do here? Yes. So I think 80 ,000 hours is the best guidance on this for a really detailed look into like, what do we need to make the field better? But a common misconception I see is that in order to have impact here, you have to be an AI researcher. I personally actually don't do AI research anymore. I worked on product at Anthropic and Product Engineering. And we build things like Cloud Code and Model Context Protocol and a lot of the other stuff that people use every day. And that's really important because without an economic engine for a company to work on, and without being in people's hands all over the world, we won't have the mind share policy influence and revenue to fund our future of safety research and have the kind of influence that we need to have.

52:51So if you work on product, if you work in finance, if you work in food, people here have to eat. If you're a chef, we need all kinds of people. Awesome. So even if you're not working directly on the AI safety team, you're having an impact on moving things in the right direction. By the way, X -Risk is short for existential risk in case folks haven't heard that term. Okay. I have a few random questions along these lines and then I want to zoom out again. So you mentioned this idea of AI being aligned using its own model, like reinforcing itself. Is you have this term RLA -IF? Is that what that describes?

53:32Yeah. So RLA -IF is reinforcing learning from AI feedback. Okay. So people have heard of RLA -HF, reinforcement learning with human feedback. I don't think a lot of people have heard this. So talk about just the significance of this shift. Do you guys have made in training your models? Yeah. So RLA -IF, constitutional AI is an example of this where there are no humans in the loop and yet the AI is sort of self -improving in ways that we want it to. And another example of RLA -IF is if you have models writing code, but and other models commenting on various aspects of what that code looks like.

54:13Is it maintainable? Is it correct? Does it pass the linter or things like that? That also could be included in RLA -IA. And the idea here is that if models can self -improve, then it's a lot more scalable than finding a lot of humans. Ultimately, people think about this as probably going to hit a wall because if the model isn't good enough to see it on mistakes, then how could it improve? And also, if you read the AI 2027 story, there's a lot of risk of if the model is in a box trying to improve itself, then it could go completely off the rails and have these secret goals, like resource accumulation and power -seeking and resistance to shutdown that you really don't want in a very powerful model.

55:06And we've actually seen that in some of our experiments in laboratory settings. So how do you do recursive self -improvement and make sure it's aligned at the same time? I think that's the name of the game. And to me, it just nets out to how do humans do that and how do human organizations do that. So corporations are probably the most scaled human agents today. They have certain goals that they're trying to reach and they have certain guiding principles. They have some oversight in terms of shareholders and stakeholders and board members. How do you make corporations aligned and able to sort of recursively self -improve?

55:52And another model to look at is science, where the purpose of science is to do things that have never been done before and push the frontier. And to me, it all comes down to empiricism. So when people don't know what the truth is, they come up with theories and then they've designed experiments to try them out. And similarly, if we can give models those same tools, then we could expect them to sort of improve recursively in an environment and potentially become much better than humans could be just by banging their head against reality, or I guess metaphorical head. So I guess I don't expect there to be a wall in terms of model's ability to improve themselves if we can give them access to the ability to be empirical.

56:36And I guess anthropic deeply in its DNA is an empirical company. We have a lot of physicists, Jared, who's our chief research officer, who I've worked with a lot, was a professor of black hole physics at Johns Hopkins. And I guess he technically still is, but I'm leave. So yeah, it's in our DNA. And I guess that's the RLA. So let me just follow this thread on interpreters of bottleneck, this kind of a tangent, but just what is the biggest bottleneck today on model intelligence improvement? The stupid answer is data centers and power chips. Like I think if we had 10 times as many chips and had the data centers to power them, then maybe we wouldn't go 10 times faster, but it would be a real significant speed boost.

57:30So it's actually very much scaling loss, just more compute. Yeah, I think that's a big one. And then the people really matter. Like we have great researchers and many of them have made really significant contributions to the science of how the model's improved. And so it's like compute algorithms and data. Those are the three ingredients in the scaling laws. And just to make that concrete, like before we had transformers, we had LSTMs and we've done scaling laws on like what the exponent is on those two things and we found that for transformers the exponent is higher. And making changes like that, where as you increase scale, you also increase your ability to squeeze out intelligence.

58:16Those kinds of things are super impactful. And so having more researchers who can do better science and find out how do we squeeze out more gains is another one. And then with the rise of reinforcement learning, like the efficiency with which these things run on chips also matters a lot. So we've seen in the industry, like a 10x decrease in cost for a given amount of intelligence through a combination of algorithmic data and efficiency improvements. And if that continues, in three years we'll have a thousand X smarter models for the same price. Kind of hard to imagine. I forget where I heard this, but it's just, it's amazing that so many innovations came together at the same time to allow for this sort of thing and continue to progress where one thing is and just slowing everything down.

59:05Like we're out of some rear earth mineral or we just can't optimize that on a reinforcement learning more. Like it's amazing that we continue to find improvements and there isn't one thing that's just slowing everything down. Yeah, I think it really is just a combination of everything. I think it's really amazing. Probably we'll hit a wall at some point. I guess in semiconductors, like my brother works in the semiconductor industry and he was telling me that you can't actually shrink the size of the transistors anymore because the way semiconductors work is you dope it with, you dope silicon with other elements and the doping process would result in either zero or one atom of the doped elements inside a single fin.

59:50Because they're so, so, so tiny. Oh my God. And that's just wild to think of. And yet, Moore's Law somehow continues in some form. And so like, yes, there are these theoretical physics constraints that people are starting to run into and yet they're finding ways around it. We gotta start using parallel universes for some of the stuff. I guess so. Okay, I wanna zoom out and talk about just Ben. Ben is a human for a moment before we get to your very exciting lighting round. Imagine just kind of the burden of feeling responsible for safe super intelligence is a heavy one. Feels like you're in a place where you can make a significant impact on the future of safety and AI.

1:00:32That's a lot of weight to carry. How does that just impact you personally, impact your life, how you see the world? There's this book that I read in 2019 that really informs how I think about sort of working with these very weighty topics called Replacing Guilt by Nate Sauriz. And he describes a lot of different techniques for kind of working through this kind of thing. And he's actually the Executive Director at Miri, the Machine Intelligence Research Institute, which is an AI safety tank that I worked at for a couple of months actually. And one of the things he talks about is this thing called resting in motion where some people think that the default state is rest.

1:01:18But actually, that was never in the state of evolutionary adaptation. I really doubt that that was true. Where in nature and the wilderness being hunter -gatherers and it's really unlikely that we evolved to just be at leisure. Probably always have something to worry about of like defending the tribe and finding enough food to survive and taking care of the children dealing with us. Spreading our genes? Yeah. And so I think about that is like the busy state is the normal state and to try to work at a sustainable pace, that it's a marathon, not a sprint. That's one thing that helps. And then just being around like minded people that also care, it's not a thing that any of us can do alone.

1:02:10And Anthropic has incredible talent density. One of the things I love the most about our culture here is that it's very eagolous. People just want the right thing to happen. And I think that's another big reason that the mega offers from other companies tend to bounce off, because people just love being here and they care. That's amazing. I don't know how to do it. I'd be extremely stressed. I'm gonna try this resting in motion strategy. Okay, so you've been at Anthropic for a long time from the very beginning. I was reading there are seven employees back in 2020. Today there's over 1 ,000. I don't know what the latest number is, but I know it's over 1 ,000.

1:02:49I've heard it also that you've done basically every job at Anthropic. You made big contributions to a lot of the core products, the brand, the team hiring. Let me just ask I guess how what's the most changed over that period? Like what is most different from the beginning days? And which of those jobs that you've had over the years have you most loved? I probably had like 15 different roles honestly. I was head of security for a bit. I managed the ops team when our president was on Matleaf. I was like crawling around on her tables, like plugging me in HDMI cores and doing pen testing on our building.

1:03:22And I started our product team from scratch and convinced the whole company that we needed to have a product instead of just being a research company. So yeah, it's been a lot. All of it very fun. I think my favorite role in that time has been when I started the lab's team about a year ago, whose fundamental goal was to do transfer from research to end user products and experiences. Because fundamentally, I think the way that Anthropic can differentiate itself and really when is to be on the cutting edge. Like we have access to the latest, greatest stuff that's happening. And I think honestly through our safety research, we have a big opportunity to do things that no other company can safely do.

1:04:11So for example, with computer use, I think that's gonna be our huge opportunity. Basically like to make it possible for an agent to use all your credentials on your computer, there has to be a huge amount of trust. And to me, we need to basically solve safety, to make that happen, safety and alignment. So I'm pretty bullish on that kind of thing and I think we're gonna see really cool stuff coming out soonish. Yeah, just leading that team has been so fun. MCP came out of that team, Cloud Code came out of that team. Wow. And the people who I hired are like, combo have been a founder and also have been at big companies and seeing how things work at scale.

1:04:51So it's just been an incredible team to work with and figure out the future with. I wanna hear more about this team actually. The person that connected us, the reason we're doing this is a neutral friend colleague, graph Lee, who I used to work with, there being B now works on this team, leads a lot of this work. And so he wanted me to make sure I asked about this team because I didn't realize all these things came out of it, team, Holly Mollie. So what else do people know about this team? There used to be called labs, I think it's called frontiers now. That's right, yeah, cool. So the idea here is this team works with the latest technologies that you guys have built and explores what is possible, is that the general idea?

1:05:27Yeah, and I guess I was part of Google's area 120 and I've read about like Bell Labs and how to make these innovation teams work. It's really hard to do right. And I wouldn't say that we've done everything right, but I think we've done some like serious innovation on the state of the art from a company design. And Raff has been right at the center of that. When I was first fending up the team, the first thing I did was hire a great manager, and that was Raff. And so he's definitely been crucial in building the team and helping it operate well. And we defined some operating models like the journey of an idea from prototype to product and how should graduation of products and projects work.

1:06:10How do teams do sprint models that are effective and make sure that they're working on the right ambition level of thing? So that's been really exciting. I guess concretely, we think about skating to where the puck is going. And what that looks like is really understand the exponential. There's this great study that METER has done that Beth Barnes is the CEO of that organization and shows how long a time horizon of software engineering tasks can be done. And just really internalizing that of like, okay, don't build for today. Build for six months from now. Build for a year from now. And the things that aren't quite working, that are working 20 % of the time, will start working 100 % of the time.

1:06:59And I think that's really what made Cloud Code a success that we thought, you know, people are not going to be locked to their IDEs forever. People are not going to be like auto -completeing. People will be doing everything that a software engineer needs to do. And a terminal is a great place to do that. Because a terminal can live in lots of places. A terminal can live on your local machine. It can live in GitHub actions. It can live on a remote machine in your cluster. Like that's sort of like the leverage point for us. And that was a lot of the inspiration. So I think that's what the labs team tries to think about.

1:07:36Are we AGI pulled enough? What a fun place to be. By the way, fun fact, Raffles, my first manager at Airbnb when I joined, I was an engineer and he was my first manager and I worked up. Well, yeah. Okay, final question before the very exciting lighting round. I've never asked this question before. I'm curious what your answer would be. If you could ask a future AGI, one single question and be guaranteed to get the right answer, what would you ask? I have two dumb answers first for fun. The first is there's this Asimov short story I love called The Last Question, where the protagonist is throughout the era of history is trying to ask this super intelligence.

1:08:18How do we prevent the heat death of the universe? And I won't spoil the ending, but it's a fun question. And then I ask you ask that question because the one in the story wasn't unsatisfying or? Okay, I'll give it away. So that it keeps saying need more information, need more compute. And then finally, as it's approaching the heat death of the universe, it like says let there be light and then it starts the universe over again. Oh wow. So that's the first cheat answer. The second cheat answer is what question can I ask you to get any more questions answered? Classic. And then the third answer, which is my real question is how do we ensure the continued flourishing of humanity into the indefinite future?

1:09:02That's the question I'd love to know. And if I can be guaranteed a correct answer, then seems very valuable to ask. I wonder what would happen if you ask a lot today and then how that answer changes over the next couple years? Yeah, I'll try that. I'll put it into the deep research thing that we have and see what it comes out with. Okay, I'm excited to see what you come up with. Ben, is there anything else you wanted to mention or leave listeners with? Maybe as a final nugget before we get to our very exciting lightning round? Yeah, I guess my push would be like these are wild times. If they don't seem wild to you, then you must be living under a rock.

1:09:44But also get used to it because this is as normal as it's going to be. It's going to be much weirder very soon. And if you can sort of like mentally prepare yourself for that, I think he'll be better off. I need to make that the title of this episode is going to get much weirder very soon. I hundred percent believe that. Oh my God, I don't know what's in store. I love how you're at the center of it all. With that, we reach our very exciting lightning round. I've got five questions for you. Are you ready? Yeah, let's do it. What are two or three books that you find yourself recommending most to other people?

1:10:20The first one I mentioned before, Replace and Guild by Nate Sori's love that one. The second one is Good Strategy, Bad Strategy by Richard Romalt. Just thinking about in a very clear way, how do you build product? It's one of the best strategy books I've read. And strategy is a hard word to even think about in many ways. And then the last one is The Alignment Problem by Brian Christian. Just really thoughtfully goes through like what is this problem that we care about that we're trying to solve here? What are the stakes in a version that's like more updated and easier to read and digest than super intelligence?

1:10:58I've got good strategy, bad strategy right behind me. I think I'm going to point to it. There it is. Nice. And I've had Richard Romalt on the podcast in case anyone wants to stay here for a moment directly. Next question, do you have a favorite recent movie or TV show you really enjoyed? Pantheon was really good based on Ken Liu or Ted Chang's story. Ken Liu, I think. Super good talks about like what does it mean if we have uploaded intelligences in what are their moral and ethical exigencies? Ted Lasso, which is supposedly about soccer, but actually it's about human relationships and how people get along and just like super heartwarming and funny.

1:11:37And then this isn't really a TV show, but Kurt Scazot is my favorite YouTube channel and goes through like random science and like social problems and is just super well done and super super well made love watching that. Wow, I haven't heard of that. As you were talking, I feel like Ted Lasso. I feel like that's what you need to put into Constitutional AI. Act like Ted Lasso. Yes. Kind, smart, exactly. Hard working. Oh my god, there we go. I think we've solved alignment problems right here. Get those writers on this ASAP. Okay, two more questions. Do you have a favorite life motto that you often come back to and work in life?

1:12:15Well, a really dumb one is have you tried asking Claude? And this is getting more and more common where, recently I asked the coworker like, hey, who's working on X and they were like, let me cloud that for you. And then they sent you the link to the thing after where it's and I was like, oh yeah, thanks. That's great. But maybe more of a philosophical one, I would say like everything is hard. Just to like remind ourselves that things that feel like they're supposed to be easy, it's okay to not be easy and sometimes you just have to push through anyway. And rest in motion while you're doing that.

1:12:51Yeah. Final question. I don't know if you want people to know this, but I've been, I was browsing through your medium posts and you have a post called five tips to poop like a champion. And I love it. Can you share one tip to poop like a champion if you remember your tips? I of course do. It's actually my most popular medium posts. It's okay. Great. I can tell. It's a great title. I think that maybe my biggest tip would be use a bidet. It's amazing. It's life changing. It's so good. Some people are kind of freaked out by it. It's the standard in countries like Japan. And I think it's just like more civilized than in 10 or 20 years.

1:13:32People would be like, how could you not use that? So yeah. And a bidet could be like a Japanese toilet. That's the long same lines, right? Okay. I love where we went with this. Ben, this was incredible. Thank you so much for doing this. Thank you so much for sharing so much real talk. Two of our questions. We're can folks find you online if they want to reach out. Maybe go work in a topic. And how can listeners be useful to you? You can find me online at benjman .net. And on our website, we have a great careers page that we're working on making a little bit easier to access and figure out. But definitely point cloud at it.

1:14:05And it can help you figure out what could be interesting for you. And how can listeners be useful to me? I think safety pill yourself. That's the number one thing. And spread it to your network. I think, like I said, there are very few people working on this. And it's so important. So yeah, think hard about it and try to look at it. Thanks for spreading the gospel, Ben. Thank you so much for being here. Thanks so much, Lenny. Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app. Also, please consider giving us a rating or a leaving a review as that really helps other listeners find the podcast.

1:14:48You can find all past episodes or learn more about the show at Lenny'spodcast .com. See you in the next episode.