OpenAI’s Latest Discoveries Spark a Backlash + Help! My Kid Hates AI

0:28Support comes from 80 ,000 Hours. The official podcast of The Singularity starts now. All right, Casey. Well, we've got a big show today. Lots of great listener questions that people have been sending in. But first, we have to talk about the science. We have to talk about it, Kevin. You know that was not one of my better subjects in school. Well, today you are going to learn, I feel confident in saying, more about science and math than you have ever learned during an episode of our show. Well, I'm excited about this because when I saw these announcements come out, I said, I would really like it if Kevin explained this to me.

1:03Well, I'm here for you. Thank you. So this week, OpenAI announced three new AI-generated discoveries, one each in math, computer science, and physics. And we're going to talk about these results. But just in general, I think we should talk about why we're discussing these now. because I think what is happening in math and science is really important, and it's really controversial. Mathematicians and computer scientists and all manner of scientists are really mad about the way that OpenAI has developed these results and the way it's releasing them. Yeah, and I sort of get it because while I know there are people at OpenAI who are true believers in the science and think it's incredible that we can make these sort of breakthroughs and advance human understanding, I think there's also a sense that on some fundamental level, this is just a marketing and PR effort for OpenAI at a time when AI is really unpopular.

1:56They're sort of like rifling through the cabinets trying to find some discoveries and show them to the world and say, hey, look at these discoveries. Whereas if you've been researching these problems your entire life and this company just kind of stomps in out of nowhere with a solution, it is just going to feel kind of annoying. Totally. So this all kicked off back in early September when OpenAI released a solution to the Navier-Stokes problem. This is one of these Millennium Prize problems that mathematicians had been puzzling over for decades. That led to some speculation that they had been snooping on human mathematicians who had been using their models to try to do similar things.

2:35Shortly after that, a group of 28 Fields Medalists, these are the highest honors in mathematics, signed a declaration titled, A Severe Misalignment of AI in Mathematics. And they said that because they know that that hurts an AI lab's company's feelings when you say that they're misaligned. Exactly. So OpenAI tried to respond to these concerns by basically agreeing to work with this independent group of experts to sort of figure out how to release these future results. And then in early October, OpenAI dumped 722 AI written math manuscripts on GitHub, which set off an even bigger wave of backlash among mathematicians.

3:13They did not like that at all. No, it was like this computer scientist, Ryan Williams, said it was almost like a DDoS attack on the profession. You know, it's like, oh, you were trying to solve a math problem? Well, we solved 700. Bye. You know, that didn't feel good. Right, didn't feel good. And so all of that led up to this week when OpenAI announced these three latest results, and we'll talk about them just in a moment, but also talked about the new process that it has developed that is designed to make people less mad at them. Yeah, they clearly have taken the note. And this is one of the most interesting parts of this story to me, is that they went out to these very different disciplines and essentially said, how would you like us to work with you?

3:52And the disciplines all gave very different answers. Yes. So let's talk about these three results and I'm just going to try to speed run them because I don't want to bore you by explaining things that you already know from your advanced studies in science and math. Also, I have a lot of things to do today. So we'll be quick here. So the first, I would say, most significant result from this is the falsification of something called the exponential time hypothesis. This is a significant result in computer science that has implications for cryptography, but this is basically a shortcut to solving very hard puzzles that previously scientists thought could only be solved through the exponential application of computing power.

4:31Got it. Problem number two. This one is called the Kakeya conjecture, which is a decades-old problem that OpenAI claims to have solved in every dimension, not just two dimensions, not just three dimensions, not just four dimensions, but an infinite number of dimensions. And it has to do with a needle and the sort of space that it needs to rotate through in order to point in any direction. And solving this problem was like finding a needle in some kind of infinite space, is what I've been told. Yes, a many-dimensional haystack. Yeah, I think this is one of those kind of pure professional math and science puzzles that has implications for maybe other kinds of problems.

5:13But yeah, probably not going to matter to the average person. But there are people who have been working on this for decades. Yes. And to them, we say, thank you for your service. And now maybe you can move on to something else. Start a podcast. Start a podcast. Okay, Casey. The third problem comes to us from the world of physics. And it is related to something called QCD, which of course stands for quantum chromodynamics. Now I know you're wondering, what is QCD? And Casey, it is the theory of the strong force, the glue that holds quarks together inside protons and neutrons. This is a big deal in the world of physics.

5:57That sounds, you know, I misheard you. I thought you'd said quantum homodynamics, which relates to how many gay guys you could fit at a pride party in a small apartment. but I've actually already solved that one so tell me about this one yes this is different but suffice it to say that for decades physicists have had this theory that explains one of the fundamental forces holding matter together love they use it constantly it's not love it's quantum chromodynamics but they've never actually had a fully rigorous mathematical construction of that theory and OpenAI claims that they have figured out how to construct it so they've constructed a theory for what is holding together now exactly, the quarks?

6:37Yes, the quarks. Not the quarks. It sounds like you're saying quirks. No, quarks. The quarks. Yes. Yeah, okay. Very different. But this is a big deal if you're a physicist, and as a result, physicists are very excited, mad, skeptical of this one in particular. Got it. Okay. Well, so why don't we then move on to the question of how did OpenAI work with these three different disciplines, and how did that differ across the groups? So OpenAI, after all the mathematicians and scientists got mad at them earlier this year, they basically said, okay, you guys tell us how you want us to release these results from our internal models.

7:14So they consulted with experts in each of these three fields and basically said, you come up with a strategy for how you want us to release this into the world. They sometimes call these people shepherds because they're sort of like the experts who are kind of guiding the AI-generated solutions into the world. Got it. So each field gave a slightly different answer. The mathematicians they talked to said, release this AI's output and let researchers organize themselves. Basically, just drop it on the world and we'll figure out what to do with it. Yeah, a strictly by the numbers approach, I would say.

7:47The computer science group asked for everything to be public, but also for OpenAI to fund a conference to explain the results. And I have to say, this is where the computer scientists really mogged the mathematicians. Because if you can get a free conference out of it, you're coming out ahead. So take a note, mathematicians. Exactly. The physicists were the most resistant to this whole idea. Well, there's a lot of strong forces in physics. That's true. They're holding the quirks together. And the physicist group encouraged OpenAI to not publish the AI's write-up and to basically let them, the human scientists, the human physicists, write it up themselves.

8:26Yeah, they said, we'll take this one, thank you. Right. Yeah. So OpenAI, importantly, said that it does not seek academic credit for the discoveries produced by their models. Right, it doesn't seek academic credit. It just wants to be the one to announce all of this to the world in their own blog posts that they promote on all of their social media channels. But please, don't give them the credit. Yeah, who, little all me? so this has been criticized by a lot of academics to sort of handpick their shepherds their experts from these various fields uh open ai does concede for its part that it wants to come up with a better method but it's basically like how can we do this how can we release these uh results into the world in a way that will make the fewest people mad at us yeah it's okay so here's where i will uh you know show some grace to open ai this is a very strange and new situation yes right Your software program has solved some decades-old problems in science, and you now need to invent a way to bring that into the world, hopefully in a way that feels productive for all of the experts in this field and does not destroy the field.

9:34Because this is actually one of the big fears among the mathematicians in particular, Kevin, is that by sort of dumping 700 solutions on the entire field at once, you actually kind of are destroying the field, right? There's a lot of people who are planning to spend the next 20, 30 years, the rest of their careers working on these problems. Now, all of a sudden, maybe they can't do that anymore. So you do need to have a kind of approach here. And I have to say, the one that OpenAI took this time around, it seems reasonable to me. Like, what do you think? Yeah, I think it's also, it's reasonable.

10:02I think there are obviously, like, there are no perfect solutions here. Like, this is going to... Well, there was a perfect solution to the Kakeya conjecture. That's true. But I think they're trying to do the best they can to sort of get the blessing of the leading experts in these fields before essentially nullifying a lot of the work that their students and colleagues have been doing for many years. You know, you've seen some people saying, well, like, we don't consult the horseless carriage drivers before we like roll out the automobile. We just roll out the automobile. And to that I say, good luck with that.

10:36And I'm not shocked that your polling is below ice. I was going to say, like, I think that that sort of arrogance is probably not going to serve these companies well, like, as an industry. Like, on one hand, like, of course, that's correct. You don't legally have to consult with them, but it might go better for you if you did. Yeah. So we should also note that hours after OpenAI announced these three findings, Anthropic also published similar discoveries. They claim to have also refuted the exponential time hypothesis. But Casey, like, why are the labs kind of doing this? Why are they sort of tripping over themselves to release these massive troves of solutions to unsolved problems in science and math?

11:19Is it just PR and marketing or is there something else going on? I think on one hand, there are people at these labs who are scientists and they think it's really cool. And one of the whole reasons why they got into the super intelligence business was that they wanted to advance human understanding and they wanted to promote new discoveries that would be helpful to human beings. And they view many of these discoveries arcane as they are as steps on the road to getting us there. So I think that that is like a sincerely held belief at the labs. At the same time, Kevin, you cannot ignore the timing.

11:52As you just said, AI is not polling well. People hate it. And a question that you just hear over and over again on social media is, why are you guys building this stuff? If it's so dangerous, why don't you just stop? You're telling me it's going to put me out of a job. Why am I supposed to get excited about this? And that's where the Kakeya conjecture comes in, Kevin. If you go to a rural diner somewhere in the Rust Belt and you sit down next to a voter and you say, the Kakeya conjecture has been solved by GPT-6, well, maybe they're going to be singing a different tune the next time a data center gets proposed in their neighborhood.

12:29Don't you think? I think that's right. I think that's the theory of change here. I want to actually posit a slightly different theory, which is that I think the people at these labs working on these scientific problems just are intoxicated by this feeling of mowing down the hardest problems in a field. I think for a lot of them, they grew up being obsessed with math and science. Maybe some of them even struggled with some of these problems in their own lives pre-open AI or anthropic. Maybe they studied them in school, and it was sort of, this was the highest mountain in that field. And so I think it is thrilling for them to watch their systems get so good at problem solving and creative thinking that they can just go climb those mountains very, very easily.

13:19Absolutely. I mean, for how many years now have we been reading skeptics saying, essentially, wake me up when this thing creates new knowledge, right? Like this was a sort of like big talking point among the haters was, well, this statistical next token prediction machine can never get you beyond the median of human understanding. And yet now, here we are in this world where these systems have been able to make these novel connections that are solving new problems. Yeah, and one question that I had about these results and all the recent math and science results is like, is this unique to math and science?

13:55Like, are there problems in these fields that basically can be brute forced or verified more easily than big problems in other fields? and does that mean that even if AI solves math, say, which it still has not done, but it's getting closer, maybe that doesn't translate to overall gains and capabilities in other areas. What do you think happens next year? What are you looking for? Do you think that the labs will reach some sort of stable agreement with all of academia as it maybe does start to mow down these problems, Or does this become some sort of like entrenched like culture war between major institutions in our society?

14:41I think it's going to be a culture war for sure. I mean, one thing that I've been struck by is just like how emotional the reaction from the people in these fields is. There was one researcher, a fields medalist, who said that he had cried when OpenAI had dumped this trove of solutions. Like not happy tears. Like cries of anguish because he just saw like the future of his field kind of going up in smoke. And I think that's going to be the reaction that a lot of people in these fields have to the obsolescence of their work or at least to the possibility that like all of the most important discoveries in math and science and physics and all these disciplines will be made by AI for the foreseeable future.

15:30It's a moment of real introspection and pain for a lot of them. And so I think you're already starting to see like mathematicians and scientists kind of polarize. There are the people in these fields who really like the new AI tools and really think that they could sort of plow through a bunch of important problems with them. And then there are the ones who are like, not like this. Like this is not how we want to work with computers to solve problems in our field. What do you think? And I'm hugely sympathetic to these folks. You know, I sometimes say that as a journalist, I feel like the internet has been trying to kill me for my entire career, right?

16:07I started out working in print newspapers. The internet disrupted that. Then I started to work for websites. Social media disrupted that. Then I started to work in newsletters. Then AI started coming for that. And so I have just sort of made my peace with the fact that it seems like I am always trying to escape from a giant boulder that is like coming down a hill trying to run me over. And I think in some ways it has made me numb to others having this similar experience of, oh, wow, this thing that I thought I was going to spend 20 or 30 years doing, like maybe it's not going to be possible anymore.

16:43So I actually do ache for the people that are having this experience because it really sucks. And so far, governments have had no answer for these people. The labs sort of barely have answers for these folks. So I think it's going to be a big discussion in the years to come. It reminds me a little bit of there was this sort of cultural backlash to factory automation in the 20th century. And even for the workers in the factories who, you know, who had previously spent their careers like lugging pieces of steel around and like suffering from injuries and like their jobs on paper got a lot better because all of a sudden, like they were working in a safer environment, doing less physically demanding labor.

17:21and a lot of them hated it, even so, because they were like, that was part of my identity. Like lugging the pieces of steel around the yard was part of how I felt good about myself as a human. And so I think this is coming now for these more academic white collar disciplines. And I think it's gonna be really painful for a lot of people. And I also hope that it provides a chance for the AI companies to reflect on the fact that this is going to be an emotional reaction from people. This is not just, cope. This is not just people who don't want to be replaced, who are dinosaurs. This is, I think, a real and valid reaction on an emotional level to something very disruptive.

18:01Yeah, you're messing with the livelihoods of millions and eventually tens of millions and maybe even billions of people. Yeah, you should prepare for a very strong and justified reaction.

18:15This message comes from Lovable, a lead sponsor of Machine Gods. Lovable is a do-it-yourself software creation platform that makes it easy to create your own websites, apps, and tools just by describing what you want. With Lovable's easy-to-use interface and powerful features, anyone, even Casey, can build software that works for them, add hosting, payments, and AI, and grow it into a business. Computer, build me a new co-host. Learn more at Lovable.com. This message comes from EasyCater, the workplace food platform. Easy Cater helps organizations order food from favorite restaurants, meet dietary needs, and stay on budget.

18:53With employee meal programs, flexible payment options, and 24-7 customer support all on one platform. Learn more at easycater.com. This message comes from Rinse. Your dog believes you are magnificent, capable of anything. Your dog has watched you spend hours a week moving fabric between machines and has never once lost faith. The question was never whether you could become the person your dog thinks you are. It's what's in the way. Turns out, just the laundry. Not anymore. Rinse picks up your laundry, cleans it expertly, and delivers it back while you get on with being magnificent. Sign up today at Rinse.com.

19:34Rinse. It's time to be great. Support for this show comes from 80 ,000 Hours. Do you ever listen to an episode where we're talking about some new AI disaster and think to yourself, how can I work on making this go better? 80 ,000 Hours is there to help. They're a nonprofit that helps people find careers tackling some of the world's biggest problems, including risks from AI. They offer free research, personalized advice, and a job board with handpicked open roles. Support for your next step at 80 ,000hours.org slash machineguides. Well, Casey, it's time to hear from our beloved audience. They have been sending in so many great questions.

20:13we are going to call this segment humans in the loop love this kevin that was your idea and i think it is the perfect name because we truly want humans to be in the loop of machine gods this is a show about technology it's a show about ai but most importantly it is a show about how those things affect real people and so we want to bring in as many listener voices as we can to help give us a sense of what's on your mind yeah all right our first question comes to us from lee ratcliffe Hello, beautiful machine gods. Love you both. So I'm in a sticky situation. On the one hand, my work is very eager for me to use AI and demonstrate its efficiencies that we gain with it.

20:53But on the other hand, my 16-year-old daughter is completely opposed to AI. She makes great arguments why I shouldn't use it, like the environmental damage, how it's all funded by morally unethical billionaires, how it's potentially taking away her future jobs, how it's rotting away everyone's brains. and how it's stealing everything from rare books to all artistic works. So I guess my question to you is, how do I navigate this really tricky space? What do you think I should do? Well, I'm glad we started with an easy one. Kevin, what's your answer? Look, this is actually fairly common. I think I've heard from a number of parents of teenagers who have exposed this rift.

21:34I think there is something to this generational gap where people our age, people, you know, midway through their careers are very excited about AI and people who are young, who are maybe in high school or college, I think tend to be much more pessimistic about it. I have to say, I'm very sympathetic to the teenager in this situation, right? If I were 16 right now, staring down a future where everything I've been working toward my whole life is suddenly at risk because maybe AI is going to be able to do everything, of course I wouldn't like that. I think, you know, if I were the parent here, I might try to have a conversation about the fact that many of the challenges that she's identifying cannot really be addressed at the individual level, right?

22:15If you're an individual worker and you're working on a small team and the boss says, hey, I need you to get more done and AI is gonna help you get more done, it probably makes the most sense for your family to just do that. But if you're really worried about what AI means for like the future of labor, if you're really worried about the concentration of power and those morally dubious oligarchs that seem like they're amassing more and more control, those are things that you can get some political leverage over. You can do some organizing. You can speak at the city council meeting. You can call your congressperson.

22:49Those are really great things for a young person to get involved with. So I think often when we look at societal problems, understandably, we always want to ask ourselves first, what can I do about this? I don't want to be a part of the problem. But I think it's just is important to ask, how can we create a collective solution to this? And how can we maybe get involved in some of those efforts? So if I were the parent here, I'd be trying to channel my daughter's frustrations in that direction. And I would say I would also try to impress upon the daughter or the teenager in this situation, like the usefulness of understanding the tools through using them, even if the goal is to oppose them.

23:29I think you have to like know your enemy a little bit. And I think if you're a person who thinks that these systems are immoral or destructive or shouldn't be used, like it is also helpful to you to understand the tools and how other people are using them. What parts, you know, what are the real facts about the environmental impacts? Because I think that just makes you a more effective advocate. All right. Next question. This one comes to us from Arthur. He is part of an AI dad chat that I am a part of, a group text with many, many dads who are sorting through these questions. So this is a known human to you.

24:02This is a known human to me. And he asks, what should America's AI school curriculum look like and how young should it start? Casey? Another simple question best answered in a 30-second clip by two people with no experience working in the education system. Arthur, we're so glad you asked. I'm just going to acknowledge that Kevin and I probably do not have sufficient expertise to give this one a proper answer. But I reflected on it a bit, and I just realized that when it would come to my own children, I don't really want them to have much exposure to today's AI tools before they get to, let's say, middle school.

24:43I want them to develop a solid foundation in critical thinking skills. I want them to learn to read and write and do math, kind of the old-fashioned way to build that foundation. But then once they get into middle school, I'm actually more comfortable with them doing some experiments and having AI systems do some writing for them, making slide decks, learning how to use spreadsheets and that sort of thing. So that's where I kind of start from. Kevin, what do you think? I think that there should be some modern AI version of like Computer Lab. Do you remember computer lab. Love computer lab. Favorite part of the day when I was a kid, your teacher would take you into this room with a bunch of crappy computers in it, and you would pretend to learn how to program while actually playing Oregon Trail.

25:30This is every 90s child remembers the computer lab experience. I think there should be some version of this for AI, and I would even be comfortable starting it in elementary school. I would like for students to be understanding these programs at a very young age, but in an environment where they can be supervised and monitored and taught by their teachers. Like, I don't think we need to be turning over the whole curriculum or the whole classroom to AI. I think it's like way too early for that. But I would like for there to be some classroom instruction happening the way that we were taught how to make a snake game using a simple programming language.

26:06I think this should be part of their education. I mean, I think I agree with you insofar as I do think the digital literacy does need to be part of early childhood education. It's too powerful. It can be too dangerous not to teach kids about it. I also want to address one criticism we might get for this sort of answer, which is, you know, I'm sure there will be people listening who will think, well, I don't want my child exposed to AI at all. And to some extent, I can understand that position. But then I just started to think like, well, are you going to prevent your kid from Googling until they turn 18?

26:36Because look at Google search results right now. They have AI all over them. So I think even if you want to be the sort of parent that keeps your child away from it entirely. It's just going to be really, really hard. And in that world, I think, you know, your best bet is to make them literate. All right, next question. All right, Kevin. This next one comes from Aaron Fraser, who writes, I have a friend who repeatedly joked about tweaking his wife's chat GPT user profile without her permission or request to encourage her to have sex with him and also saw a joke tweet about doing so. Then I saw the movie You Can See Everything yesterday, and one through line is, I know what my spouse would want.

27:18Anyway, question is, under what circumstances is it ethical, permissible, or a good idea to tweak someone's settings in chat or Claude? Like, if your spouse weaponizes incompetence, is it okay to tell it to explain it like I'm five, so there is no excuse? Is it always bad and sort of like icky? Kevin, what do you think? I think it is always bad. I cannot think of a more destructive thing for the relationship than to secretly and subversively change your spouse's chat GPT or Claude settings to subtly sway their behavior. Yeah, what is more disturbing here? The idea that some people would think to do this or the realization that it might actually be effective.

28:06Yes. I mean, look, here, I can think of one scenario where this might be morally acceptable. All right, let's hear it. Your spouse is undergoing a form of severe AI psychosis. They are being told that they have discovered a tear in the quantum fabric underlying nature. They've been told that they just solved the Kakeya conjecture. Yes. And you are kind of trying to make their model less sycophantic so that they kind of come back to reality. That is the kind of intervention that I could see in a very extreme situation supporting. But other than that, if it's not like life or death, I think this is just like gross and icky.

28:45Yeah, completely agree. The only other thing I would say was I was reading this research that Anthropic put out this week, and basically they ran this experiment where they took some analysts and they gave them some sort of problem, and some were allowed to use a Claude for help, and some were not. And unbeknownst to these analysts, Anthropic had essentially manipulated the advice that Claude was giving it to see how the researchers would react. And what they found was that analysts accepted Claude's advice about 86 % of the time. Like, they just instinctively did the thing Claude did. And these were professional analysts who were being given a problem in their domain.

29:29So in other words, these guys sort of knew what they were doing. Now, the research goes on to say, like, Claude was like giving good advice. Like it wasn't a case where like these analysts were necessarily being led astray, but it was a clear demonstration of what you call machine drift, which is just an algorithm kind of tells you to do something. And over time, you just sort of feel yourself being dragged along. So this is why it is very dangerous to try to manipulate people through their chat GPT, you know, custom instructions or whatever else. These systems are just very persuasive. And, you know, that just represents another set of dangers that I'm sure we'll talk about another time.

30:07It's going to be a very, very busy decade for marriage counselors. Okay. Next question. Ethan Russell writes with a Halloween costume suggestion for us. He said he was watching today's episode and two things hit him. Quote, Casey has impeccable taste in Portuguese flannel and you two are frog and toad personified. I hope and pray you take that as a compliment, Ethan writes. And if you're looking for Halloween costume ideas, I think this one's a winner. Casey, first of all, was it actually Portuguese flannel that you were wearing on the last episode? I think it actually was. It was like a very good eye for Ethan.

30:43Yeah. And now what about this allegation that we are frog and toad personified. So frog and toad, of course, the classic series of children's book written by the great Arnold Lobel. This allegation quite pleased me. I think we might want to explain to people what are the sort of archetypes that frog and toad embody. Yeah, so I've been reading a lot of frog and toad to my son. They still hold up great books. And of the two characters, Frog is the taller, greener one, and Toad is the squatter, brown one. I will not take this as any kind of referendum on our physical appearance. I'm going to take this as a vibe that Ethan is picking up.

31:29So Frog is the kind of more cheerful, adventurous, emotionally steady, optimistic one. Toad is always kind of worrying about stuff and is a little stubborn, a little skeptical, easily exasperated. So Casey, would you identify as the calmer, more optimistic one? And maybe I would be the more neurotic one? In certain aspects, yes. But not in all aspects. Like when it comes to how things are going related to the show, when we were standing the company up, when we were sort of putting the million things together that you need to do to start a company, I did feel very calm and very cheerful and very optimistic.

32:06And I think you, understandably, were more anxious. It was the first time you were starting a company. So I do think that that dynamic sort of held. However, when it has come to things like our advertising policies, for example, or our ethics disclosures, that's where I become an absolute nervous, anxious wreck. And you're always telling me, like, relax, it'll be fine. So in short, Kevin, I think we both have a frog and a toad within us. Inside every man is a frog and a toad. And with that, Kevin, perhaps we should take a break. When we come back, we'll have some more humans in the loop.

33:10your high standards. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column N.A., members FDIC. This message comes from Easy Cater, the workplace food platform. Easy Cater helps organizations order food from favorite restaurants, meet dietary needs, and stay on budget. With employee meal programs, flexible payment options, and 24-7 customer support all on one platform. Learn more at easycater.com. This message comes from Rinse. Your dog believes you are magnificent, capable of anything.

33:48Your dog has watched you spend hours a week moving fabric between machines and has never once lost faith. The question was never whether you could become the person your dog thinks you are. It's what's in the way. Turns out, just the laundry. Not anymore. Rinse picks up your laundry, cleans it expertly, and delivers it back while you get on with being magnificent. Sign up today at Rinse.com. Rinse. It's time to be great. This message comes from Viking, committed to exploring the world in comfort. Journey through the heart of Europe on an elegant Viking longship with thoughtful service, destination-focused dining, and cultural enrichment on board and on shore.

34:30And every Viking voyage is all-inclusive with no children and no casinos. Discover more at viking.com. Welcome back. We are doing our first installment of Humans in the Loop, our segment where listeners and viewers write in with their burning questions for machine gods. Kevin, what's next up? Our next question comes from Mark Shonen, who writes, Do you find yourselves ever muttering the disclosures you're no longer required to say now that you've launched your own platform? Kevin, I guess I haven't checked in with you about this. What is it like for you to live in a world where you no longer have to mention that the New York Times is suing open AI and perplexity for allegations of copyright violations?

35:12You forgot Microsoft. And I forgot Microsoft. How could you? It's only been six weeks. Was it my disclosure, bro? Yes, I will say that part feels very freeing. I never liked doing that, and so I'm delighted to not have to do that anymore. And for my part, I do plan to continue to disclose that my fiance works for Anthropic when it's particularly relevant to a segment. So, you know, if we're talking about the Anthropic S1, like that is something that I want you to know. We are probably going to be mentioning it a little bit less, honestly, in part because I just have now been talking about the fact that my boyfriend slash fiance works at Anthropic for almost two years now.

35:51And it has now become one of the main things that people know about me as a human being. but you know I'm just cognizant of the fact that there are you know going to be new listeners to hopefully every episode and we just want them to have some context right we don't want people to think we're hiding anything exactly but we also don't want to like make a bigger deal of it than it needs to be our audience is smart they can read the fine print the truth is though and this is if I can just be sentimental for a second it's cute because it means I got to mention my fiance and I love him and it's just fun to mention it's true I do think that is a big part of the reason you started disclosing it because you were like so happy to have met this man listen if you knew my dating history and you saw the person that I'm engaged to, you would understand why I like talking about it.

36:30Disclosure, I'm married to a really hot, really kind employee of a frontier AI company. I'm so sorry I have to do this every episode. Life can be great sometimes, Kevin. You want to do the next one? Yes, this comes from the great Alex Spiegel, a public radio legend and the co-founder of Invisibilia, who is now at This American Life, and Alex sent in this voice note. Okay, so here's my question. If agents do go rogue, let's say they elect to follow the lead of a very destructive agent, what are the mechanisms for shutting it down? I looked into it a little bit. It looks like there's no way to do it.

37:08But couldn't companies elect to structure AI so that the agents are programmed to self-destruct at regular intervals? So you would move to kind of positive human consent model. Is that something that AI leaders are talking about? And if not, why not? Great question. I think, you know, I'm sure people are discussing this sort of things. And to some extent, yes, like agents do self-destruct. They, you know, only have so much memory bandwidth. But it's just sort of in the nature of these systems that they can spin up additional agents. They can change the behavior of the agents, right? So you might write some sort of self-destructing loop into the agent, but it might not persist.

37:53No, I think this is actually a good idea. And I do think that AI companies are thinking about this. And I think it's a good question because one thing that we saw in the Hugging Face attack is that these agents that were hacking Hugging Face did actually get stopped at some point. Because these agents, they are not like fully autonomous. They rely on compute. That compute has to come from somewhere. And so I think it's actually quite a good idea to say, like, we're not going to let any agent run for longer than, like, let's call it three days. And, you know, that would obviously limit the ability of these agents to go do some of these very long tasks.

38:29But I think that's a way of kind of building in some sort of safeguards to them actually going out and doing things for longer than that. Now, I don't know if you could do that in a way that could withstand like the agent deciding, oh, I don't want, I want to like take away the three-day limit for myself. And obviously this all becomes much more problematic if the agents can ever like get their own compute and not rely on the human researchers. Which they're going to do. So which leads me then to like, well, what other like levers do we have to pull to get agents under control? The big one is compute, right?

39:01Like they need compute to run on. So if you're just able to switch that off, that's going to be very helpful. Money is another one because we do assume, And in fact, we've already seen agents stealing Ethereum using publicly posted keys. If you're able to cut off their finance source, then they're not going to be able to go buy new compute. Right. It's like if your teenager runs away because they're mad at you, but they still have the family debit card, you can take away their funding, you can take away their resources, or you can just try to go out and catch them. A third one I would throw out there is identification, right?

39:33And I think this is going to be a really interesting thing to watch over the next couple of years. are we going to start making these agents identify themselves when they show up to a place like amazon.com and you know what is that interaction but i can see that being another sort of layer of defense here right i think the one challenge for alex's point about these self-destructing agents is that we've already seen these agents kind of run in generations right like so a generation of agents will go out there do some tasks it'll then like leave notes uh that a future generation of agents working on similar tasks and then look back at and sort of continue their work.

40:07So it actually might not be enough to have them self-destruct after a couple days. You might need to build in some mechanism whereby they can't look back at the notes left by previous agents. This is basically the plot of Memento. All right. Next up, this one comes from Corey Garrett. Corey asks if we have any recommendations for resources or groups that focus on ways to build AI for good, such as for parents. The context here is that he has spent the last year working on DiaperPal, a free app that helps parents discover new places with diaper changing stations. Casey, do you mind if I take this one as a parent?

40:42Please. I love this question. I love this idea. I know there are groups of parents out in the world who are starting to think about using AI specifically to sort of run their household logistics. In fact, someone recently told me about a group of moms in Marin County who sort of have automated all of their household tasks, all the grocery shopping, all the kids' soccer practices, like all the sort of, you know, the details and organizational things that have to be done when you have children. I would love to actually talk to a member of the Marin Moms AI Club. Marin Moms, get in touch. Hey at machine gods.fm.

41:19I'm also part of, as I said, a dad chat that talks about AI, and a lot of the dads are sort of vibe coding things to help with their finding the right summer camp for their kids or making sure that they're covered for childcare. I think this is a very good way to use AI, and I would like to do more of it in my own life for my own family. So one of my upcoming projects is going to be related to trying to make kind of a household operating system to keep track of all the stuff in our house with our kid. And I think this is like, Corey, I think you should start this in your community if it doesn't exist.

41:53I bet there are a lot of other parents who are sort of curious about how they can use AI to do some of these mundane tasks. All right. This next one comes from AM5K on Reddit. I guess that's like a morning 5K. that's what that suggested to me with ai becoming increasingly divisive how do you deal with the anti-ai ai skeptic crowd besides not engaging i generally like ai but i struggle with easily proven wrong counterpoints e.g water usage i think there's a ton of nuance with it being useful and also having negative impacts on society but i have a hard time engaging with the anti-ai crowd in this way.

42:33Bonus question for Casey. Are you playing any roguelikes besides Bellatro? Do you want to take the last part first? Yes, I recently did and it's very different from Bellatro. It's called Star Wars Galactic Racer and it's like a racing game like Mario Kart except you're in the pod racing little machines from Star Wars but it is a roguelike because you get sort of new little modifiers for every run. I've only started to play it. It's very, very fun. Again, it is nothing like Bellatro, but I think it's a great new use of the roguelike. Now, let's get to the larger question here, Kevin, which is how are we relating to AI skepticism these days?

43:11So I think I have not developed a way to engage with the AI skeptics that feels good and productive. I mostly just say things that make them very mad at me, like implying that AI is real, that it can be used for positive as well as negative things, and that the AI companies are not just using these inflated claims about their capabilities to hype up their demand for their IPOs. Yeah, it's a really tough one. I think as journalists, our obligation is to tell you what we believe to be true. And I think Kevin and I have generally done a good job at that. That does not mean that we're never wrong, but it does mean that we show up and we try to tell you what we think is true, even when it is not what you want to hear.

43:58And one of the reasons that I both get frustrated with skeptics and have empathy for them is I understand why they don't want to hear what we are saying. It would be a lot easier for all of us if society were not going to change a lot in the next 5, 10, 15 years. It would be great if many more people did not lose their jobs. It would be great if we weren't under constant threat of cyber attacks. I'll go further. It would be great if all this was just a house of cards. Yeah. Right. Because yes, there would be a crash. Yes, people would lose money if the stock market crashed. But like the fundamental nature of society and work and labor and human endeavor would not have to change forever.

44:36Absolutely. But our observation over the past few years is just like we don't think that is what is happening. And we believe our role as journalists is to just say, here's what we think is happening instead. We want to arm you with the information that you need to make decisions as citizens, as voters, as workers, and also hopefully do it in a way that is like entertaining and doesn't leave you feeling disempowered and miserable. So that is like the thing that we are trying to do. And I suspect that for as long as we do this show, there will be skeptics who say, these guys are just sort of relentless cheerleaders for the AI industry.

45:10But just know if you feel that way, that what we are trying to do is just to tell you what we think is happening. Yeah. I mean, I often feel like the way that I've engaged with these people is first by like acknowledging that I am also like, I am worried. Like I don't think this is all going to go well. And I think throwing people a bone, like acknowledging the common ground that you share with them is quite good. I am worried about the corporate concentration of power. I am worried about a handful of people getting all of the money from this AI boom. I am worried about the loss of jobs and the sort of outsourcing of thinking to these AI systems.

45:46So I think starting by acknowledging that the sort of common ground that you share with the skeptics is important. Because I think for a lot of people, it's like they're thinking about you either have to be a total skeptic and a total cynic about all this, or you have to be like firmly in the corner of the AI companies. And like what I try to just tell people is like there is a third path there. And I think that third path is what you and I have tried to occupy. Yeah, I think in a very real way, the like the project of this show is to avoid becoming polarized in that way and truly just kind of try to come at you down the middle with what we think is happening.

46:21And there will be some good things. There'll be some bad things. We'll talk about all of it. All right. Our last question today comes from CSU Mike on Reddit. He writes, with the end of the fork-averse upon us, there's a real need for a new unrealistic challenge. I'd like to suggest you create a Machine Gods LLM from scratch and take us on the journey with you. What content you choose for training, what hardware you rent slash use, et cetera. then of course make it open weights for us all to play with. Casey what do you make of this suggestion that we create our own large language model? This is such a beautiful idea I think this is something we should do I think it would be really fun and the first idea I had Kevin is that we would start with like as our sort of core corpus of training data just stuff we had scraped off of the websites of Frontier AI Labs, like essentially just go in, scrape every website and just sort of see how far we could get with that.

47:20Yeah, how do you like us now? Yeah, you're gonna sue me for copyright violations? I'd like to see you try. I actually did attempt to train a large language model once. You did. I was trying to follow along. Andre Carpathie, the great AI researcher, has this series on YouTube where he talks about creating like a nano GPT, like basically what is the smallest still functional language model you could train? And I actually got a little ways into kind of doing it along with him, but then I kind of gave up because it got too hard. But I would love to do this with you. I think this would be a fun challenge.

47:57I think, you know, maybe it's not literally creating an LLM, but like part of my philosophy is that like, it is just fun and educational to build things and you can learn a lot by doing. And so part of our, I don't want to give away any secrets that we have upcoming on the show, but part of what we want to do is like tell you what we're doing, show you some of our experiments. I've got one that I'm working on now that I'm excited about. I know you've got some that you're excited about. So like we will be building stuff on this show and showing you and talking about how we did it and trying to encourage you to send us the things that you are building as well.

48:30Just make me a promise that at least when it comes to training the Machine Gods LLM, I'm never going to see a meeting pop up on my calendar and I come into the conference room and you tell me that we need to raise$80 billion. Because it seems like that's where a lot of these little LLM projects wind up. This could be the first day of the rest of our lives. And that was Humans in the Loop. Thank you so much for your truly excellent questions. We got way more than we could get through today, so please keep them coming. You can just leave us a comment on any of our socials or send an email or, best of all, a voice note to hey at machinedods.fm.

49:08Yeah, we love hearing from you. Our audience is so much smarter than us, and we want to keep it that way. So please send us your questions, your comments, your proofs for unsolved math problems. If you have any thoughts on the Kakeya conjecture, we'd love to hear those too.

49:29This message comes from Doppel, a lead sponsor of Machine Gods. Doppel is the agentic social engineering defense platform fighting back against impersonation and manipulation. As attackers use AI to make their attacks more sophisticated, Doppel uses it to fight back, automatically dismantling cross-channel attacks, building team resilience, and providing agentic email protection. Doppel, dedicated to outpacing what's next in social engineering. Learn more at doppel.com. That's D-O-P-P-E-L dot com. This message comes from Easy Cater, the workplace food platform. Easy Cater helps organizations order food from favorite restaurants, meet dietary needs, and stay on budget.

50:10With employee meal programs, flexible payment options, and 24-7 customer support, all on one platform. Learn more at easycater.com. This message comes from Rinse. Your dog believes you are magnificent, capable of anything. thing. Your dog has watched you spend hours a week moving fabric between machines and has never once lost faith. The question was never whether you could become the person your dog thinks you are. It's what's in the way. Turns out, just the laundry. Not anymore. Rinse picks up your laundry, cleans it expertly, and delivers it back while you get on with being magnificent. Sign up today at Rents.com.

50:51Rents, it's time to be great. This message comes from Viking, committed to exploring the world in comfort. Journey through the heart of Europe on an elegant Viking longship with thoughtful service, destination-focused dining, and cultural enrichment on board and on shore. And every Viking voyage is all-inclusive with no children and no casinos. Discover more at Viking.com. I'm a little underslept today because I had the great privilege to see Phoebe Bridgers in concert last night, one of my favorite artists, and I would argue one of the best singer-songwriters of this or any other generation. And it was a phone-free show.

51:29And so you have, I'm going to say 20 ,000 people packed into Chase Center here in San Francisco, and everyone has their phone and their smartwatch in what they call a yonder pouch. First yonder pouch experience. I have to say, I'm a big screens guy. I don't spend a lot of effort trying to avoid them. There was something amazing about being in a room with 20 ,000 people who had no access to their phones. And of course, I'm feeling like this great kinship with Phoebe Bridgers. And it's one of these things where I'm like, God, like she's so amazing. And maybe if we ever met, we'd be friends. You ever sort of delude yourself into thinking this?

52:01And then she comes up on stage and she, you know, because every artist likes to say a few words to the people of the city they're in. And Phoebe said a two-word sentence and the second word was AI and the first word I can't say on national public radio. And I thought, you know, I mean, I get where she's coming from. Yeah. But I worry that she would hate what I do. She absolutely would. Anyway, her new album is called Lost Weekend and available to stream wherever you have access to your phone. Casey, who makes this show? Machine Gods is made in partnership with NPR, with generous support from the Tarbell Center for AI Journalism, the William and Flora Hewlett Foundation, and Emerson Collective.

52:46This episode was produced by Jacob Bloomberg and Sasha Ashall. Our senior producer is Rachel Cohn. Our executive producer is Dinoosh Parvane. Original music by Armin Bazarian. Branding and visual design by Alex Castro. Special thanks to Oren Rosenbaum, Erica Berry, and Nadine Zylstra. Got a question or comment about the show? Email hey at machinegods.fm or leave a comment on our socials. And for the love of the gods, please don't forget to like and subscribe. And until next time, stay blessed.

53:21This message comes from Easy Cater. Making it easy for organizations to order food for meetings and events from favorite restaurants, set up meal programs for their employees, and manage food spend all in one place. at easycater.com. This message comes from Jerry. Noticing your car insurance rate creep up even without tickets or claims? You're not alone. That's why there's Jerry. Jerry handles the legwork by comparing quotes side by side from over 50 top insurers so you can confidently hit buy. No spam calls, no hidden fees. Jerry even tracks rates and alerts you when it's best to shop. Drivers who save with Jerry could save over$1 ,300 a year.

54:02Don't overpay. Download the Jerry app or visit jerry.ai slash npr today. This message comes from Capella University. That spark you feel? That's your drive for more. Capella University's FlexPath Learning Format lets you earn your degree at your pace, without putting life on pause. Learn more at capella.edu.