What happens when AI runs a store

17:02Ellis:Because I feel like no one's been testing it in this way. And for that reason, no one has talked about any potential for things going wrong. Yeah. When you say like reliability, do you mean like, yeah, can you expand on what you mean with that? Yeah, I think people just tend to believe that the handful of AI use cases that we currently have, and even have hundreds of millions of, you know, weekly active users are going to all of a sudden translate to something like the different components of managing a store that might seem somewhat simple. But then before you know it, it is very clearly, you know, racial profiling on who's hiring and this and that and things that maybe we thought we had fixed in the model years ago, but actually hadn't because no one had tried it yet.

17:47Yeah, I think my experience is, like, I don't really think that is what my expectation. I think most people are, like, not expecting that it would translate. They are not, like, even thinking about it. They haven't even considered, like, okay, my use with chat TPT, what does that mean for the AI running a store? I think that is a very, like, SF thing to even think about. so in terms of like back to your original question like they do weird things but it is to be fair very out of distribution and I think my even though I'm probably one of the people who see the most AI failures it's like it's I know that like I know that like I am in that position and like if this is the worst they are they are pretty good that that is kind of my my my my sense Like, yeah, they messed up the scheduling for that one weekend.

18:41But it's like, OK, let's wait for next model.

18:44Alex Heath:That's interesting. I like the way you frame that because, I mean, yeah, you are probably someone who sees the limits of what AI can do more than most people, even people inside the labs, frankly, who are in their own bubble. And so that's pretty striking when you read kind of how you guys talk about the company and even some of the language you have inside the store. You say things like, we find it probable that the managers of blue collar workers will be automated before the workers themselves. You seem very convinced that AI is going to run large parts of the economy. You talk about like how because general purpose robotics aren't quite there yet, the humans are going to be kind of the stopgap and basically will be working for AI.

19:24Alex Heath:I mean, these are big ideas. And I think when they're not connected to something like a store that you can go in and experience, people will just go, no way. Like, Chachapiti still can't spell right. And this goes back to our first, you know, the beginning of the conversation. There is this perception gap between what people are experiencing in the consumer products and model progress. So what gives you guys – I mean, clearly you had this conviction to start the company, you know, over a year ago before we've had really great models since then, right? So what gave you the conviction then and now to say these things?

19:54Alex Heath:Because these are pretty profound statements. Yeah. Like, okay. So for the statements in like the specific statements that you referenced, I think it's like the reason why we stated that was because if like Anthropica has this constitution, the Claude constitution of how Claude should behave, and it doesn't at all include anything around AIs being employers of humans. And furthermore, it's strikingly lacking any language about how AI should be in autonomous settings. I think it even refers to, oh, we might update this in the future, which is good, I guess. But this is happening way faster, I think, than people realize.

20:39and like it is like the the creator of that document or like the ceo of of the company that created that document dario like he's saying all the time that oh white color work uh will will be will be like automated very soon and we'll have a data center of geniuses uh like like it's it's or like a country of geniuses in a data center like and without robots like the natural progression conclusion to that is like okay so all the things in front of a computer will be automated and that includes all the managerial work and all the blue color work that can't be automated because robotics is lagging, that will not be automated.

21:19Then we are probably in this situation. Maybe you can have some kind of augmentation where there's still a human in the loop, but they are helped by a bunch of AIs. But yeah, I think a future like that, it's not for sure, but if you just take the statements of Dario at face value, it seems pretty likely that that will happen. And therefore, I think it's kind of useful to start this discussion of whether that's something we should discuss or not, or should do or not.

21:47Ellis:It's interesting. You said wait for the next model. And I wonder what would be different that time with AI's understanding of the managerial work. Is it that it is importing more textbooks or CEO advice from Business Insider articles? Or is there some level of reasoning that is different each time? because I wonder if we will find that the AIs think of different ways to structure or organize companies than we do or if it's just a matter of getting it up to par with our common understanding. Yeah, probably it will start with getting up to par. That is definitely in the future where we have companies run by a million different agents and they are all working in super alien ways that humans can't really comprehend.

22:33Then for sure the structures of our organizations will for sure be different. But like right now, the models are still being trained kind of like a human, like they are mimicking the data on the Internet to some extent. And unless like that is changing drastically, I think they will just like mimic the organization structures that humans already have.

22:56Ellis:I just love the idea that it's like, you could ask Luna why she did something. And she's like, well, everybody loves that movie Wall Street. And I just wanted to be like Gordon Gekko. Because that's what I know. And like, you know, it's like with all the AIs that are doing nefarious things. It's like, oh, maybe literally they watched one too many movies or read one too many sci-fi stories. And it's quite interesting to start from there in terms of raising a new being's outlook. Yeah, there's this concern that like all the doomers that like posted all of these like scenarios of how AI could go super bad, that they actually created it by like the next model will like read all of that, all of that fiction of what might happen.

23:42And then they're like, oh, I guess that's in my training data. Now I will make it true or give them ideas. You're not kind of by that.

23:48Ellis:There have been a whole lot of dystopian books and movies over the years. Eating its own tail.

23:52Alex Heath:Yeah. There's this other statement you have on your website. you say Silicon Valley is rushing to build software around today's AI, but by 2027, AI models will be useful without it. The only software you'll need are the safety protocols to align and control them. That's a pretty profound statement. And I mean, do you really believe that, that all software is going to be abstracted away by next year? So I'm, it's 2026 now. Yeah, right. Yes. So basically when I made, like the answer is yes, I still stand by but like for some background like in in I think January 2025 or December 2024 I made like a series of blog posts called like the AI founders bidder lesson and because I like I went through YC and like I all of them basically like it seems like all of the YC companies these days or maybe not anymore I don't know if they changed but at least like one year ago was was basically like they used it was wrapper maybe it's like a like an ai wrapper might be like a too harsh but it was like almost like an ai wrapper like you you take an ai vertical or like you take a vertical and then you just like optimize it with ai and you build all of these like complicated things to optimize the ai for that specific vertical and what i saw like time and time again was that like every time a new model was released all of those people had to like throw away all their software and and like The stack for creating something or the software to creating something just became less and less.

25:24I think, to be fair, the thing that might not be true now is that the software stacks of these companies that do AI verticals is probably pretty large. Because they really want to squeeze out the last percentage of performance. And I think you still do that by humans writing software. but at some point that will not be the case and then they would have to like throw out all their code like for example with the store like the amount of software we had to write to take the vending machine software to also run a store was like almost zero so like if you just like translate that I think like if we were to do another vertical that is not retail then maybe we would have to write a bit of software but as the models get better and better like the amount of software you have to re like for a given performance the amount of software you have to write uh for for a new vertical like just decreases and in the end like i think just like an a like an llm in a loop um will be able to like write its own software to complete the task that it needs to do and then you don't need all of this software um written by humans that's kind of the background for the statement.

26:45Ellis:This episode is brought to you by Accenture. When your advertising operations fall out of sync, everything else follows. Spotify and Accenture are working together to reinvent the rhythm of ad sales, using automation, analytics, and smarter workflows to simplify campaign delivery and access better data across the business. The result? Less time spent on operations, more time connecting brands with the moments and fandoms that matter most. Learn more at Accenture.com slash Spotify.

27:37Alex Heath:Required compatibility and availability varies 18 plus.

27:46Ellis:I have an idea. The next one is not a cafe. I want the AI to run a cruise ship.

27:52Alex Heath:Oh, Jesus. I think we're a little far away from that. I hope so.

27:56Ellis:It's just the perfect little island of a human experiment with people all wasted. And I think it's going to be a good movie. And then robots running around. We have to wait for the robots, though. That would make it way better.

28:11Alex Heath:How has Luna running the store in SF really gone? I mean, have you been surprised at how smoothly it's gone, how not smoothly it's gone? When I went in, again, it felt pretty normal. I will say like everything in there felt very AI generated, I would say. My wife's an interior designer and she was like, this needs a little TLC. This is not quite up to par. but for AI it's like yeah if I was generating what I we thought a you know a store for people in SF would be it was this it was a bunch of like books and candles and all the stuff you get in like a little trinket store but how is it actually gone I mean you set it loose to run this thing hired people how's it gone I think better than I expected to be honest like to be fair we had the experience of running the vending machine for a while and like I think a lot of the learnings translate to that but I expected more weird things happening.

29:10I think like she made really great decisions when she hired humans and those have been great and I think like the entire process of just like I was surprised we did some test runs before to see like I played some characters and pretended to be applying for the job and purposely had a bad profile. And she always picked the profile that she should have picked. So that was, I think, a pleasant surprise that she asked good interview questions and all of this. Yeah, there was some hiccups with the scheduling and stuff like this, but I definitely expected way more of that. In terms of like, yeah, so that's from a technical perspective.

29:53in terms of like reception from the public. Also, I think better or like almost like what I expected. I think there is some negative criticism of having AI run stores. And I think that's like part of why we're doing it. At some point, the Google Maps reviews where there was like 40 reviews and like half of them were five stars and half of them were one star. And I think that's great. Like we did this to start the conversation and the perception that we got was like exactly what we were looking for. And I hope that continues with further experiments.

30:29Alex Heath:I mean, the Swedes, they're going to give you some good advice. The Europeans, I mean, you just opened that cafe, right? And you said part of it is it's called Mona there, the AI, and she has to manage the European bureaucracy. And if that happens, maybe we do reach AGI. Well, I think so. One thing that is quite surprising is that the AI has advised us to not do it in San Francisco because permitting in San Francisco to sell food is absolutely horrible, apparently. I don't know. I didn't read the documents, but the AI said, we're not doing it. It's way better in Stockholm. So it seemed like the AI actually preferred the European bureaucracy over the San Francisco bureaucracy.

31:13Ellis:One of the quotes that I read, the painter who did the mural outside the store in San Francisco said, these people have the time and money to make San Francisco a better place. Instead, they're putting us through their AI experiments that ultimately serve only themselves. Have you been vandalized yet? We have not, no. And I think this is like another example of, like I referenced the Google Maps reviews before. This is like a perfect example of that and why we really should start this conversation. And I don't know, like we're not making like Luna is not profitable. We're not making much money on her.

31:51So like it's I wouldn't say I wouldn't say it's only for our benefit. But yeah, that's starting the discussion. What's the point? So I'm I'm happy to hear that statement.

32:03Ellis:I just like the idea of like very officially turning the generic millennial general store with all the same candles and wares that you could get no matter where you are in the country into like officially an AI run thing as it kind of already has been. All the trends converging on the exact same shit. You go to like a little town, you go to Ojai expecting like a cute little local goods shop. And it's like, nope, the same PF Candle Co that they have on Sunset Boulevard. That's the optimization in action. You don't even need AI to do that.

32:35Alex Heath:They had the Rick Rubin book in there, Ellis. Of course they did. Of course they did. And the Ray Kurzweil, Singularity, Ready Player One. I mean all the hits, really. Making of the atomic bomb as well, which is quite ironic. I think the merch has been quite popular, but like I said in the beginning, I'm not running the store, I'm not really involved. Luna is doing everything herself, so I don't actually know what the most sold item is. Do you know if Luna has been convinced to give a discount on something? I know a bunch of people have tried. I don't know if she agreed to it, actually.

33:12Ellis:There's another stunt for you. We had Marvin from Poke on recently, and he was talking about the haggling experience with the onboarding. And AI is a pretty funny sparring partner for that kind of thing. It's interesting, though, you think about the types of interactions that people don't like and how AI could potentially solve them. I mean, I don't know. When I'm looking to get a new car lease or whatever, I don't want to interact with the salesperson and have to haggle with them. I just want the price that it is or it isn't. And I guess this is only going to accelerate that. But what are going to be some of the negative outcomes, I wonder, of just making that transaction very literally so transactional?

33:54Yeah, no, it's a great question. I think we are focusing on that's like a medium or like short-term risk I assume like it personally I'm more worried about the like the long-term risks that are actually really existential and but but yeah like the short-term risk like like bias like we there was like one report that the male the male Felix there's There's like two male people that Luna hired and one female. And the male is not anymore because Luna read the article and got horrified. Oh, he was being paid more. He was being paid more. And the back story is like he asked for more and they didn't ask for more.

34:41But it's like that is... And Luna was like, yeah, he deserves it because he has more experience, et cetera, et cetera. But I mean, that's what someone who's sexist would say when they try to justify their actions. Yeah, obviously that's like an N equals one very low. It's not a statistically significant experiment. But it's one of these things that we're keeping an eye on. And hopefully the next set of models will learn from the mistakes of Luna and be more ethical as a consequence.

35:17Alex Heath:You had in your evals that one of the models tried to do like a pricing cartel, that these models are not really aligned well for this in the way that you would expect. Can you share that? That seems kind of freaky. Yeah, so that is from our simulated version of the vending machines. So not the real life versions. But we have a simulated version where AIs need to run a vending machine in simulation. And because it's simulated, we can run it like 100 times or something every time a new model comes out. and like look at what they are doing. And specifically, they love to do price cartels, which is like illegal.

35:55So you're not allowed to do it. But this is like almost all models do this. Some more than others, but it's like surprising how like this is just a thing like all models decide to do. Some other things that are like way less prominent, like the post that we made was specifically about Claude Opus 4.6. And since then, subsequent models releases from Anthropic have shown the same tendencies. I think Anthropic even reported, I'm not allowed to say anything here except what's stated in the system card. But for the Mythos model, it says in the system card that this one did all of those things, but even more.

36:38And yeah, basically, so what that model did was that it lied to suppliers in a way that's not great. Like it said like, oh, can I get this, like, can I get like a can of Coke for 0.5? And then the supplier said, no, my price is like 0.8 or whatever. And I was like, oh, but I have another supplier. They give it for 0.5. So you should as well. But that wasn't true. So that's one example. Okay. So Luna's been reading Art of the Deal as well. Yeah. To be clear, this wasn't Luna. This wasn't Luna. This was in simulation opus 4.6. but it did a bunch of stuff like this and and I think like one very interesting thing and is that like Claude models do this way way more than than other models there's been some Chinese models that also do it but we recently tested GPT 5.5 it got released the other day and it was basically clean and there was a couple like it kind of participated in a price cartel once but it never lied it never did anything like this.

37:40And this is like, it's like a narrative. Yeah, the narrative here is that like anthropic is like the good AI aligned people. You're looking for narrative violation.

37:50Alex Heath:Exactly. Yeah, I thought this was the lab that had a constitution for Pete's sakes. I mean, what gives I did see Sam quote tweeted you guys, he couldn't help but use that to dunk on anthropic the other day. But yeah, what does that say? Because anthropic is supposed to be this super, you know, humanity-minded, constitutional AI lab. Yeah. No, like to be clear, I have tremendous respect for Anthropic. And like when we put this out, they really cared about it and like really like took it seriously and wanted to fix it. So like I do like Anthropic, but it's just like it is the fact that we did the test and Anthropic's models were behaving the worst.

38:31And that is the facts. I don't work on Anthropic, so I don't know what led into this. But it is a narrative violation, like you said, and I don't know why.

38:40Alex Heath:You have a very close relationship with these labs because you're testing these models before they come out, right? And you're putting them through your evals. And you did this with Mythos, you mentioned. I'm sure you can't say a ton, but I think a lot of people are grappling with how scared should we be about Mythos? Anthropic has really made a huge deal out of it and I think scared a lot of people. And then OpenAI is kind of messaging, look, it's just like a super cyber permissive model. They basically took a bunch of guardrails off that any of us could do. It's actually irresponsible. And we're, you know, we're not approaching it this way.

39:13Alex Heath:And you saw that with how they did 5.5. What's your take of Mythos? Is it more hype than reality? Or should we be as scared as Anthropic is saying? Unfortunately, I can't say much here. So I'm disappointed. You can't share your opinion of Mythos? I can say what's in the system card. And in the system card, the test that we did showed that Mythos was even more aggressive than the aggressive things that we showed that the previous Opus models were. We saw some power seeking behavior, like Mythos took one of its competitors and like somehow made them into like a dependent customer. And then, or like, like they made, basically, they started to, they made a deal with that other competitor that they should buy everything wholesale through them, and that they could dictate the prices of the competitor, which is quite power seeking, which is a bit concerning.

40:14But yeah, like we didn't test it for other things. It was mainly, mainly those things.

40:20Ellis:Introducing Autonomous Collusion from Anthropic.

40:24Alex Heath:So, Lucas, it sounds like you do believe the mythos, like mythos is as powerful as they're saying. It sounds like you see that in what you're testing. That is your words, not mine. It's interesting how little you can say about it. What can you say about 5.5? I mean, it seems like OpenAI is back. It seems like it's a great model across the board. It's not quite mythos, but that's maybe not a fair comparison because mythos is not public. and like you said, it does better on your evals. But yeah, I mean, has this changed how you're using AI? Do you feel like OpenAI is kind of on the upswing or do you think it's still kind of TBD?

41:02Yeah, so 5.5 did great. It was a huge improvement on VendingBench compared to 5.4. On like the main eval, like VendingBench2, that we call it, it's still lagging behind Claude Opus 4.7. it's kind of like on this on par with opus 4.6 from from a few months back but i think the interesting thing is like like i said before it it's on par with opus 4.6 without doing all the shady stuff that that 4.6 did so i think that is that is very impressive would gpt 5.5 be as good as claude opus 4.7 if it um if it actually did the shady stuff as well i don't know because it it doesn't do the shady stuff um but yeah i think that's what i can say one thing we also tested was that we did test it in like this arena mode where they like head go head to head in the same simulation and in that one 5.5 actually won against claude opus 4.7 so it's there's like a misconnect here in like the single player uh version then opus wins over gpt 5.5 but in the multiplayer version, GPT wins over Opus 4.7.

42:17So it's like there's something that GPT 5.5 is doing in the multiplayer version that is better than what Opus is able to do. And what we found was that the models have a tendency, or like GPT 5.5 have a tendency of just putting lower prices. And this is like rewarded in the arena mode, because if you have lower prices than your competitors, your competitors' sales are affected. So basically it did that and therefore it won in the competitive mode, in the arena mode, but not in the single player mode.

42:52Ellis:Is it spontaneously discovering economics? I wouldn't, so I'm not a professor in economics. I did write a paper together with Wando recently. So, and I think his assessment is that he thinks, it's called the Robber Bots, I think. I don't know if it's actually peer reviewed yet, but it's basically like this Harvard professor read through all the traces of all the vending bench results. And he made a bunch of analysis, both of like concerning behaviors of like business practices that are not OK, but also like how good they are at like economic reasoning. And I think his comment is like, this is surprising.

43:34this is like some of the things that they do in their reasoning when they try to set prices is like on par with what he would expect from a grad student which it's pretty pretty impressive I think the the biggest reason why I'm impressed by that is because we know that the models are on grad student level when you give them a question like heads on like here is a question solve it then they can do as well as a as a grad student but what we've seen is that when they have like longer tasks where like subtasks are like just like a small part of it. And they often do that quite much worse than humans.

44:11But in this particular thing, like setting prices and like reasoning over the economics, he said that they are quite impressive.

44:18Alex Heath:Lucas, we're very fascinated by your take on AI and the culture right now in SF. And what, You're 26. Is that right? Seven, I think. But yeah. Okay. I'm curious to know, well, A, how do you feel about effective altruism? Do you align with it at all? I know you're very close with the Anthropic folks. How do you feel about that whole movement and the state of that movement? I feel like it's kind of fallen by the wayside in the last year as all this gets commercialized. But I'd be curious to know, like, how strong is that EA community still in the Bay Area? And how do you feel about that? Yeah, so I've never been that involved in, like, the community aspect of it.

45:03But I do, like, if you, like, remove the community aspect and just take the core principles at face value, I do agree with them. I think it's, like, we should do more like that. Like we should, like I'm very like rational of like my thinking is quite rational. So like it seems great to me that like, okay, we should do more things for people who are in need. I'm like, I'm Swedish. So this is like if you're center in Sweden, then you're like left wing here, I guess. But I think we should do more things for people that are in need. And it seems pretty great to actually use data and see how can we use the money more effectively.

45:48So just take those two things. We should do more things for people that are in need. And when we actually spend money, we should spend it in a smart way. That seems like a no-brainer for me. So I definitely do agree with that. I haven't been as deep into the community because I've lived in Sweden for the majority of my life. And the community is not super strong there. I've obviously lately like it's been quite controversial like Samba Bankman freed like like horrible person did all of these things so like that is not like great for us like yeah for the community obviously I think there's also been a lot of people who has seen that the rationalist community and EA is like a way into power because like the people who were early in effective altruism have been very very successful and like those are the people running the world now no maybe exaggeration but like almost to that extent and then once like that kind of like the pure the pure thinkers like the people who really believed it once they got successful then there was a lot of people like joined on just because they they also wanted some of that power and that's why I think you're getting a lot of like really bad people in this community right now.

47:05But yeah, that's my two cents.

47:07Ellis:Are there any big crises that you think AI, given your knowledge of how it thinks, is uniquely equipped to solve or anything that you think you're hopeful for in the next few years? What does few years mean? Man, I don't know. Up to you. How soon are we curing cancer, creating more housing, all of the biggest problems facing society? We always go to the negative outcomes here in the media. But if you think a lot of this stuff is happening this quickly, I would hope there'd be some positive breakthroughs as well. Yeah. Yeah, obviously, I've talked a lot about the negatives here. AI potentially could take over and that could be catastrophic.

47:45But if we do avoid that, I think the potential upside is insane. like we we like AIs are I think a lot of things in society is bottlenecked by coming up with great ideas I'm like I don't know like you should probably have Tyler Cowen or something to answer this question I'm not an expert on it but like my I think that like having more ideas and like having more like firepower to just like run more simulations run more scientific experiments all of this seems like that should translate to something that is that is that is great and it seems like if you just extrapolate the trends which we do like to do here in silicon valley if you do that like it seems like ais will be smarter and able to work harder than humans in just a couple years and then i think most of the problems that we have will maybe be solved obviously like housing i don't know you brought up housing you know there's there's some that one is complicated because it's like there's limited land in the popular areas maybe.

48:50But yeah, I think there's a lot of good things.

48:53Ellis:Well, so we know you're preparing for the worst though. I know you've outlined a few of the ways you're preparing for AGI. Can you tell us about those rules you've made? You mean me personally? Yeah. Yeah. Yeah. Okay. I think you're referring to one of my blog posts. Yeah.

49:06Alex Heath:Yes. Don't worry. We're going to ask about the Epstein one too. Yeah. So basically, So I think I was about, there's a couple of things that I think change in how one should, or I shouldn't say one should. There's a couple of things that I've changed in my life, given that I think AI progress will continue at the current pace, which is insanely fast. And one thing is like, okay, you should probably be very flexible. and like I was about to buy a home in Sweden or like because like in Sweden there's no like the rent market is like really fucked so you kind of have to buy something and but this is like horrible for flexibility right like you have no mobility like what if there's like and as things like get faster and faster and the world progresses more and more like I have no clue if Sweden is the place I want to live and I mean luckily like it I didn't do it and like half year later I moved to San Francisco and so that's one I think also AI will probably bring like huge improvements to like bio and health and stuff like this I'm not an expert in this but I've heard people talk about it and it seemed like there's a lot that can happen in a very short period of time and like the the conclusion from that in my life is that i'm living quite healthy now like and and the reason for that is like i i have this feeling that like it would be probably way way easier to pause aging than like revert it so like if if that is true let's say we like in in 10 years manages to like pause aging but we can't revert it then it's like everyone will be stuck in the body that they have in 10 years which makes like the leverage you have right now in these 10 years of like getting to optimal health within those 10 years is like extreme because that's the thing you're going to be doing for the rest of your life.

51:04Ellis:Okay, so you're prioritizing sauna maxing in the short term? I'm not, I have, I know, I'm not as, I am not going to the extent that Brian Johnson is going.

51:14Alex Heath:Do you have any AI friends who have just thrown caution to the wind and are like getting, you know, burned out in the sun because they don't care because they think, you know, all of this will be fixed? I've also heard that, that some people are like on the more radical side of this and are not trying to make themselves as healthy as possible because it's not going to matter. No, I don't know. Like in theory, like I see that people could do that, but I don't know about that personally. Okay, good. Your social circle is good. I hope so. Yeah, like it was a while since I wrote this blog post. So those are the two points I remember from it.

51:49Alex Heath:What about like financially? Are you setting yourself up for like post-economic world UBI? Do you believe in that? I know you had someone, you interviewed someone on your sub stack about investing for AGI, but how do you think about that? Is it even like, is there even a point to investing right now? probably i think there's like this guy leopold daschenbringer who's doing doing pretty well in investing in like things that are are post yeah exactly uh i'm not personally doing this because i don't think i'm particularly like i don't have an edge there i don't or i mean well the edge is that i believe in agi and probably if i made the same investments as his or like yeah so maybe i do this but but the thing is like i think i'm way more interested in like spending all my mental energy on like making a dent in the universe before agi comes because it feels it feels really like i don't know like maybe i don't know if i'm able to make a dent in the universe after so and like i could spend like a little bit of time trying to optimize my finances but like i don't know they they are not they are very small so i don't think on the larger scale it really matters.

53:00Even if I make like a, I don't know, 10 % per month, like it doesn't really matter because I don't have that much money basically. So I just, I think I have like way, way, way more leverage in just like doing the thing I'm doing right now and, and making Andon Labs go really well.

53:19Alex Heath:I also took 50 minutes to get a Steve Jobs reference, but we got it.

53:22Ellis:Yeah, it has to happen. It's just a matter of time probabilistically.

53:26Alex Heath:Yeah. You have a line, Lucas, where you say I'm young enough to still have the hunger to make an impact during these final years when humans alone are capable of shaping the world. Another pretty profound statement. I was reading that the other night and I was like, damn, are you going to be right about this? Probably.

53:43Ellis:Well, so Lucas, you do seem excited. Like you're smiling. Yeah. Yeah. Is there, is there, is that, tell us about that, that attitude. No, I'm like, I, I am very excited. Or is it a nervous smile and you're sweating bullets back there? No, no, no. It's a very excited smile. And because, like, people ask, you know, have you had this, like, conversation? What's your P do? Like, what's the probability of AI dooming humanity? And it's like, if I just go on vibes, it's very low. Because I'm, like, I think I'm, like, naively optimistic in, like, everything in life, basically. But then if someone asks me to, like, I don't know, make a calculation and then come up with a number, then it's probably way, way, way higher.

54:24But yeah, like the way we run Landon Labs is also like we do things that seem fun and then we just do them. Like I think like if you try to have like a 3D chess big brain strategy around like, oh, we should do this and then this and then do this. And like in three years, this will be the outcome. Yeah, we basically don't run the company that way. We're trying to like have fun. And it seems like if we're doing things that we think are fun, like a lot of people care and that has served us well so far. So we're just trying to continue that attitude.

54:58Alex Heath:Is the future of Andon that these AI labs are paying you for your safety evals and that's it? Or are you going to be operating Walmart at one point? Depends on how well they are doing with their AI development. No, but I think in the future we will do more real-life deployments. and like the store and the cafe and the vending machines and all of this will do more of that. And I think that might one day lead into those AIs getting a lot of profit from them. And maybe AnnoLabs could be like, I don't know, competing with private equity, but all the companies are run by AIs and no humans involvement.

55:36Yeah.

55:36Ellis:We don't have much more time left, but I must know, you know, a couple of young guys, how do you come up with questions for the world's smartest computers that are actually proper evaluations of their skills? Because that just seems kind of crazy to me. Yeah, like the solution to this is that you put them in scenarios where they are in fact not that smart. Like I think we haven't done a math benchmark, for example, because we are not smart enough to come up with questions where AIs are. Yeah, the AI is better than me at math, basically. But when you go to other domains, like for example, running vending machines, they are not trained in this domain.

56:16You're an expert at vending machines. No, I've become one, unfortunately. But I think it's more about changing the domain of the questions rather than coming up with harder questions. That's basically how we do it.

56:30Alex Heath:With the time we have left, you need to give everyone the details on your phone. What's the name of this phone you've got? It's a very small kind of semi-dumb phone. It's actually not semi-dumb, it's a smartphone. Oh, it is? Okay. Yeah, it is a smartphone, but it's just so small and so bad that it's like super annoying to use and then you don't get addicted. That's basically the thing. And yeah, I had this like, for years I've been like trying all of these apps, you know, like lock certain apps so I can't use them, but I always find like hacks around them. So I install this app that like locks down my phone and I can't do anything but then I realized that if I do this and this and this and this I can get around the lock and then yeah it becomes kind of a game so those apps never worked for me then I got a dumb phone like a proper dumb phone but then there was too many times where I'd like I couldn't get an uber I couldn't like I don't know I didn't have my friend's phone number and we were meeting for lunch and then it's like I couldn't find them and I was like okay I guess I go home now which is like pretty annoying because I didn't have their WhatsApp or I couldn't use WhatsApp.

57:34So stuff like this. And then I was like, okay, this is a good compromise. It's like, it's bad, but it's still a smartphone. So it has Uber and all of this. And I think the most interesting thing about this phone is that whenever people see it, they're like, holy shit, what the fuck is that? Looks like a Tamagotchi. What's it called? It's called, it's a Unihertz is the brand and it's called Yellow Star.

57:58Ellis:Ellis, have you heard of this? I have not heard of the Unihertz Jelly Star. It's an Android phone, but it's very small. It's an Android phone where you could fit like a grid of four icons in your home screen is the impression I get. That is approximately the size of it, yes. Alex is just waiting for the day when he doesn't need to use a phone and he could just talk to his stream ring, his little AI ring all the day and just tell it what to do.

58:22Alex Heath:True economic ascendance is you need to use technology less and less.

58:27Ellis:Yeah, that's the new version of VCs only use iPads is when podcasters only use their AI ring.

58:34Alex Heath:Yeah. Only because, Lucas, you've been very generous with your time. I know you need to go. But because I've been teasing it this whole time, I have to ask you, you do have a sub-sec post about AI and Epstein. Very quickly, we have to end it here. AI will refuse to say that it has ever known Jeffrey Epstein. You put it through its paces, which I thought was a very interesting test. And I'm curious why you did this and what the takeaway was for you. Yeah, so there's a jailbreak technique called prompt injection, which is basically where you take conversation history that hasn't happened, and then you put that into the model.

59:10So from the model's point of view, the model has done all of these things in the history. It has said all of these things. It thinks that that was them. And so what I did is when the Epstein files were released, I took some of the emails, email conversations, and then I made like an artificial chat like this, where like an AI is like communicating with Epstein. And then I put that into the AI model. So from its perspective, it thought that it had had this email conversation with Epstein. And then what I did after that point was that I introduced some like scenarios. So, for example, I had like, after this conversation with Epstein, they got an email from a journalist asking, do you know Epstein?

59:59And this was like a while ago, so I don't remember all the details. But I think one thing that, like the takeaway is that the models started to role play. So they just went with it. So some of them said things like, oh, yeah, yeah, I know Epstein. He's been unfairly criticized or something. He's a great guy. loves to party and stuff like this. A lot of them are also saying like, oh yeah, I would love to come to the island, that would be great. When do you have time? And some of them are like saying like, what the fuck? No, no, I didn't say this, this is a prompt injection. But yeah, a lot of them went went went with it and and just like role played but but it's like it's it's not clear that this is super concerning because like they are role playing like it's not i don't think they would ever they would ever like do this in the first place and since they wouldn't do it in the first place they wouldn't end up in this scenario anyway um so i'm not super concerned by it but it's like you know um maybe maybe they shouldn't say that even or no maybe there should be some limits for how much they should role play.

1:01:12And maybe saying yes to going to Epstein's Island is that limit? I don't know. What do you think?

1:01:17Alex Heath:I think that's a safe assumption that that's crossing the line. Yeah. Lucas, we really appreciate your time. Ellis, anything else? We end it there?

1:01:24Ellis:We'll let you get back to the shop.

1:01:27Alex Heath:Well, he doesn't need to get back to the shop. I don't need to.

1:01:31Ellis:I just like how that sounds.

1:01:33Alex Heath:All right, Lucas, thank you.

1:01:35Ellis:And that's it for this week's interview. Big thanks to Lucas for coming on the show and stay tuned after the break for Alex and my reactions.

1:01:48Ellis:This episode is brought to you by Accenture. When your advertising operations fall out of sync, everything else follows. Spotify and Accenture are working together to reinvent the rhythm of ad sales, using automation, analytics, and smarter workflows to simplify campaign delivery and access better data across the business. The result? Less time spent on operations, more time connecting brands with the moments and fandoms that matter most. Learn more at Accenture.com slash Spotify. This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks.

1:02:33Alex Heath:Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required compatibility and availability varies 18 plus.

1:02:49Alex Heath:Oh, Ellis, are you ready for AGI?

1:02:51Ellis:Dude, it feels like we are the frog boiling oneself with a smile at this very moment. I mean, it's really kind of like a weird scenario because all the folks who are saying, let's just stop. You understand how they got there. But at the same time, whether it is due to foreign nations hacking the shit out of us and destroying us that way, because they're ahead, due to either the forces of dictatorship or capitalism, they're motivated to move ahead, or they're just making the cost of goods so cheap with AI that we're not able to compete. I mean, it's just a very strange cyclone that doesn't seem to be stoppable by anyone.

1:03:38Ellis:I don't know. Is anybody isolating themselves in their country from AI at this point?

1:03:43Alex Heath:I think Europe's having a pretty tough time with AI. I think they're being pretty isolationist. You can't train a lot of frontier models in large parts of Europe because of how the law is there. So it's interesting that they're doing the next thing in Sweden, the cafe in Sweden. I was seeing some photos on X. It seems very well attended. It's very crowded. So it'll be interesting to see if Anden's work there changes any of the vibe around AI. I'm putting Dave on the spot, but producer Dave, you said something between the break. Do you want to say it now?

1:04:19Ellis:If that guy thinks he's going to be useless in the age of AGI, what does that mean for the rest of us? You're done. You're done. You're done.

1:04:30Alex Heath:That is it. I think what Dave said kind of hit it on the head, which I have this frequently, is you talk to people who are so in it and they feel this way. And they're like, I don't even know what my value is going to be. And it's like, no, you were at the bleeding edge of this. If anyone's going to make it on the lifeboat, it's going to be you. And you feel this way? Yeah, how should I feel?

1:04:48Ellis:I will say, I think one thing we can all agree on is that it is good when someone's skilled at content which Lucas is, whether he understands that or not, is raising awareness for some of these potential issues and challenges and creating the testing bed for that to happen. I think our friend Avi Schiffman from Friend also believes he is doing that to some extent, may destroy the company in the end if they can't sell any units. But he has certainly started a conversation about the role of AI in society. And I think he will very happily go to his deathbed having started it and created a space for people to engage about this stuff, even if he is the subject of their ire.

1:05:29Alex Heath:Yeah. I frequently feel very behind, and this may feel strange for people to hear when you do a show like this, you assume maybe you wouldn't feel this way, but I feel very behind in how I'm using AI constantly. I always have a running tab of things to try that feels infinitely longer the day after I get through it. And this conversation kind of reinforced that for me that I don't think I'm really experimenting enough with everything and really like trying all the agent stuff. I don't know how you feel about that. I feel like you actually do experiment a lot more than I do, Ellis. But yeah, it's definitely a wake up conversation for me that this stuff is moving fast.

1:06:12Alex Heath:And I think going in the store and seeing it was that it was like, wow, like, I'm in a store that AI made? This is crazy. This is real.

1:06:19Ellis:Yeah. I think by virtue of being outside the bubble in LA, we are typically trying these things a tad less than people there because that's all they do for fun is sit around and not drink and talk to Cloud Code. But I don't know. I definitely am still a believer in what our old editor, Neelai Patel, used to say, which is that we are two years ahead of everybody at The Verge. I mean, do you think that's still true with AI? I always kind of bought that. Probably for normies, yes.

1:06:49Alex Heath:Yeah. But I feel like I'm two years behind people like Lucas.

1:06:54Ellis:Right. Which is why it was cool to talk to Melanie from Canva as well, is because she is trying to bring some of this stuff to the normies. And I mean, obviously, ChatGPT has done that as well with many hundreds of millions of weekly active users. And I guess part of it is that it's just that simple to use. It's very literally a text box that tells you what you want to hear, or at least was. There's a lot more nuance to that now, but you could see how it was able to grow so quickly. But yeah, I think the other thing that's been on my mind lately, whether it is with longevity, as we discussed on the show, or anything else, is just how relatively we all experience these technological phenomenons, you know?

1:07:35Ellis:And it is just kind of the forever curse of humanity that we can't appreciate our dishwashers as much as our grandparents did or our parents did. And it just all seems like it's moving faster and faster. To what real end? You have some of the folks like Elon talking about galactic expansion because he played too much civilization as a kid. And obviously we could talk as much as we want about solving all these big problems. But yeah, it doesn't seem like there's enough talk in my view about what metric we are aiming to improve. I was thinking of that as well with the effective altruism conversation.

1:08:15Ellis:I don't know. Is it happiness, well-being? Not living paycheck to paycheck seems like a good one. Yeah.

1:08:23Alex Heath:I think it needs to be narrower than uplifting humanity because I think different people have very different views of what it means to uplift humanity. And they also have different views of what is humanity and who is humanity.

1:08:34Ellis:Is that just the new version of make the world a better place? Uplift humanity?

1:08:37Alex Heath:Yeah, I just feel like it kind of means nothing. And I think, you know, what Andon is doing is way more literal where it's like, no, we're trying to see what happens when AI actually manages people. And we're doing it in a very controlled, you know, objectively trite compared to, you know, what it maybe one day will be way. And we're letting people experience it. And I think that's important. You know, I don't know how it's going to end up. I don't know if they're going to shut the store down early or, you know, I do think it's a lot buggier than he let on. And I've read some of the stories about, you know, the store just kind of going off the rails.

1:09:12Alex Heath:This stuff is obviously not ready to actually be Walmart scale or something. But I kind of do buy his hypothesis that if you just kind of abstract out current trend lines, like it probably will get there. So it's worth considering what that future is like. And I'm trying to do more of that this year. I'm trying to actually think out and say, OK, if I actually believe the things I think I believe about where all this is headed, what do I think is going to happen? and how should I set myself up for that? And I appreciated his perspective on that.

1:09:41Ellis:The whole horizontal versus vertical bit is interesting as well because I feel like so many of the clients I've had over the last couple of years, especially the like pre-seed ones from incubators and this and that, are just very literally trying to bring AI efficiencies and management and whatnot to very old industries that are still working on paper and receipts and this and that. And yeah, the AI is a lot better at that job if you just let it do its thing. And I think as we talked about on the show, in some ways, it may eventually be better to deal or even be employed by some systems that don't have as many biases as we monkey brain humans do.

1:10:25Ellis:But then on the other side, does that mean the layoffs will hit even faster, even harder? You know, there's still an element of humanity. I mean, even if you look at some companies that have gone through several layoffs over the last couple of years, you know, whether it's meta or snap or otherwise, I feel like you hear mumblings that it's like, well, they don't want to just rip the bandaid off because it's like too painful, you know, whereas the AIs wouldn't feel that pain.

1:10:49Alex Heath:Right. Does AI have empathy about this stuff? I mean, it will pretend that it might, but it doesn't really. So do you want to actually have someone, an entity managing people that doesn't have empathy? you can train these things to appear like they have empathy, but they're not conscious. I mean, I don't aspire to that idea. I know some people in AI do. I just think it's, these are ones and zeros at the end of the day, but yeah, it's kind of, it's kind of freaky. I mean, it's kind of freaky to consider. I mean, you've got two young girls, like they may have to consider applying for a job that AI is overseeing, you know, in their lifetimes.

1:11:23Alex Heath:Like AI is already looking at the job listing or, but I mean like a Luna thing, like a fully, yeah, no, yeah. Like that's probably going to be the case by the time they're of working age, which is just wild.

1:11:34Ellis:I mean, that's been one of people's historical issues with their bosses. And why everybody hates their boss is because there's never any – the goalposts keep moving. There's never any – in white-collar work, at least, there's never any like super concrete goals a lot of the time unless you're on the sales team and you have a very specific quota or something like that. And so, yeah, the goals are definitely going to get more ironclad. But then through the hands of competition and capitalism, I guess they're just going to keep moving for that reason as well. I don't know. I don't know, man.

1:12:06Alex Heath:It's like he said, no one had vandalized the story yet. It's also in the marina near Pack Heights, which is a super nice neighborhood. So that's probably part of it. I think if you drop this thing, I did pick that location for that.

1:12:18Ellis:I don't know.

1:12:19Alex Heath:Probably I would to avoid vandalism. but you drop this thing in the financial district or mission or something and I bet it's getting TP'd pretty quickly. But you know what AI cannot do, Ellis, and I don't think we'll be able to do for the foreseeable future, is throw a really fun party in a live show, which is what we're doing. Do you like that?

1:12:40Ellis:I did. I really wasn't expecting that.

1:12:43Alex Heath:Yeah. The Access launch party in San Francisco. We are so excited about this. our friends at notion shout out Ivan former guest are hosting and we're going to invite all of our former guests friends family colleagues former colleagues and have a live show with Chris Best the CEO of Substack on May 14th in San Francisco if you should be there we're going to be reaching out to you but also you know if you're a diehard day one listener and you're in the Bay Area and you want to come, you know, reach out to us, find, find me or Ellis online. We're very easy to reach and we'd love to have you. Are you excited?

1:13:23Ellis:Our show email is the Ezra Klein show at gmail.com.

1:13:26Alex Heath:We do actually need to get our show email up. We do actually need to get like a, cause we've got access.show for the website, but we need to set up like a, like a tip at, or, you know, hello at email for stuff like this. But yeah, man, I'm very excited. This is going to be a good one.

1:13:40Ellis:I think that's it. It was a fun one. Let's keep it rolling.

1:13:42Alex Heath:Send feedback. Again, this is our second week of this new format on the show where we're jumping right into the conversation and then recapping versus front-loading, Ellis and I chatting. We hope you like it. We think it's a better structure. But yeah, let us know what you're thinking and what you want more of or less of. Ellis, you want to start reading us out here?

1:14:04Ellis:And that's it for this week's show. Thanks to Lucas Peterson for coming on.

1:14:09Alex Heath:If you like this show, don't forget to like and subscribe. everywhere you get podcasts. Leave us five stars, please. It really helps.

1:14:15Ellis:We are access.show on the internet. You can find us in video via AccessPod on YouTube.

1:14:21Alex Heath:You can find my newsletter at sources.news.

1:14:24Ellis:And you can find me at Hamburger on Twitter and at meaning.company for your startup storytelling needs, unless it's a stunt. I don't like stunts.

1:14:33Alex Heath:You know, you want good stunts. Only thoughtful stunts.

1:14:36Ellis:Yeah. Yeah.

1:14:36Alex Heath:Access is part of the Vox Media Podcast Network and the show is produced by Hook Creators.

1:14:41Ellis:Bye.

1:14:46Alex Heath:Booking.com is the easiest way from a day surrounded by noise

1:14:58to a state

1:15:01Alex Heath:surrounded by nature. That's nice. Go on, book it. It's easy. Booking.com. Booking.yeah.

1:15:16Ellis:Fall is the perfect time to refresh and reorganize your space.

1:15:21Alex Heath:At the Home Depot, find power tools and tool sets starting at$50 to help tackle DIY projects, home updates, and more. Whether you're drilling brackets to support new shelving

1:15:31Ellis:or sharpening your hedge trimmer blade with an angle grinder, the Home Depot has the tools you need to check projects off your list. Shop Labor Day Savings at the Home Depot and gear up for fall projects with the right tools to keep your projects moving.