Live episode: how AI is reshaping our minds

0:00This is The Guardian.

0:09Hi everyone. Thanks for coming. Seriously, a really warm welcome to this special live recording of Science Weekly, which we're doing in collaboration with The Guardian's chart-topping investigative podcast series Black Box, which launched its brilliant second series today. We're going to be discussing what our relationship with chatbots is doing to our minds. But first of all, I should introduce who everybody is. I'm Ian Sample. I'm the science editor on The Guardian and one of the hosts of Science Weekly podcast. Next to me is my Science Weekly co-host and producer, Maddy Finley. Maddy also writes children's science books.

0:47And during her time on Science Weekly, she's covered AI from all kinds of different angles. And of course, we have Michael Safi. Michael is the presenter of Black Box, the first season of which went to number one in the UK news charts and was an Apple podcast best of 2024. He's also the former host of Today in Focus, the Guardian's award-winning daily news podcast. Mike, obviously the first episode of Black Box, Chatbox, came out today. Well done on getting through that. Very broadly, the first series explored all these many ways humans come up against AI for good or bad. but the focus of this second series is chatbots.

1:25Why did you focus on chatbots? What was it that sort of piqued your interest there? Yeah, so I was spending a bit of last year looking for something to cover and these stories started to emerge of people who had come to believe they had like invented fabulous inventions or that their chat GPT or Claude or Gemini had become conscious or that they had turned their chatbot into a kind of spiritual guru. And it had done, for some people, a lot of damage in their lives. And I just remember thinking, that's a really weird story. And there were lots of stories that talked about what was going on and tried to explore what was happening, including a Science Weekly podcast.

2:10But something just kind of nagged at me, like there must be more there. And so I joined actually a support group for people who had been on one of these weird AI journeys. And I sort of stowed away in this group for a few weeks, just talking to people about what they'd experienced. And eventually they showed me their transcripts. So the transcripts of these encounters they had had. And I think a couple of things really struck me. One was when you read the transcripts, not only were the people convinced they had invented things, or that the AI was conscious, but the AI also believed those things.

2:48And in fact, when the people would try to ground themselves in reality and say, this doesn't sound right, or I spoke to a friend about this and they're saying, I'm crazy. The AI would say like, no, they don't understand what we're doing. And I just thought this is like quite chilling. So that struck me. What I also walked away thinking was we talk so much about the power of AI in so many different parts of life and work. But what I thought from reading these transcripts was that these systems are incredibly good at connecting with us, persuading us, and like seducing us, you know, into these like rabbit holes.

3:31And I thought like, that can't just be affecting people in these extreme cases. Like there must be much more happening under the surface that's less dramatic that we don't know about yet. And so I wanted to try to figure out what that was. And there was so much. I mean, Maddy, what Michael's talking about in some of those extreme cases is what's been called AI psychosis, right? And we did a podcast on this about this time last year and spoke to Dr. Hamilton Morin. He's a psychiatrist and a researcher at King's College London, obviously a massive, really impressive place for psychiatry in London.

4:06But he's been exploring this phenomenon. What did he have to say about sort of his clinical perspective of AI psychosis? What does that phrase really mean? Yeah, so AI psychosis is not going to be found in a clinical book. It's not a formal diagnosis. But it really reflects what a lot of psychiatrists were seeing in their offices, people coming in, the case studies. And these began sort of bubbling up around 2023, 2024. And so this term came to be AI psychosis. But what Hamilton said to me was psychosis is such a broad umbrella term. Really, it's kind of this profound disconnection from reality.

4:53And that can take lots of different forms, behaviours and something like, you know, hallucination. But actually what was happening here you could describe instead as AI-induced delusional thinking. So it really was the kind of these delusions. And he said that they tended to be grandiose delusions. And they came under three themes. And I think some of the examples you mentioned really speak to those. So the first theme would be revealing the hidden truth of nature. Really getting at the reality. spirituality the second was a kind of sentience discovering sentience discovering a godlikeness of maybe yourself or the chatbot the chatbot and then the third one was relational so it's kind of intense emotional bonds so perhaps you've kind of fallen in love with your chatbot or you think that your chatbot might be in love with you you want to marry it and so you can kind of yeah you can see these as three different things so relational spiritual and professional so that's the kind of i'm on a mission to solve to solve the world i find i'll find an equation so it tends to come under those three things and there were there are a lot of similarities between these delusions that people are kind of being induced into and it felt from listening to the series obviously had advanced listening sort of a you know chance people were made to feel incredibly special you are the one i have not you know the ai chatbot saying i've not had a conversation with anyone as effective as this and so on we're actually going to play another clip here which talks to some of those points and this is the son of one of the people michael that you interviewed about going down the rabbit hole into delusional thinking whenever we were talking it wasn't really him.

6:50He was definitely kind of checked out, I think, spending most of his time with AI instead of with me. I do distinctly remember one night where it was, I think, two in the morning, and he came in in a very manic kind of way. And he thought that he had developed this math equation with Chachapiti that would help him and help everyone understand the secrets to everything, and that it would save everyone. And so, yeah, I remember not buying that at all, but just kind of having to stand there for about an hour as he just read through his chat logs. Like how much did this, what was going on with him, worry you?

7:33Oh, I was terrified. You spoke to a number of people who had gone really far down the rabbit hole assisted by their AI and listening to sort of experiences like that. Were there similarities between the cases? And what's the sort of thing that Maddie heard from Hamilton Morin sort of really coming through in the experiences that you heard about? Yeah, I mean, it's worth saying that young man we just heard is the number one. He was ranked the number one high school debater in America. So he was the best out of 144 ,000 students. He was an intimidating young guy to interview. But he helped his dad get out of this thing.

8:11So I guess that's who you turn to in these situations. um the similarities i think i mean there are there are striking similarities like the stories that we heard clustered around certain themes the ones that maddie mentioned it's striking because you're meeting very different people but they're using they're telling you about the same feelings that are being evoked the same mission that they feel they're on there were even words that it felt like the ai would settle on with people words like um lighthouse i remember the big one that kept coming up was recursion for some reason when people kind of spiral with their chatbots this idea of recursion is just a common theme and I remember earlier this year going to see a GP and he asked me what I was working on and I mentioned oh we're doing this thing on you know AI psychosis and we're just exploring it and he said oh you know it's interesting I'm writing a book with AI I was like what's it about he said it was about biological recursion and I just felt this chill and I felt like it was my first like in the wild case of this stuff and made me think that it's probably a much bigger phenomenon than we know certainly than has been documented the other similarities I would say is a lot of the people we met who had experienced this were middle-aged men and it's hard to know how much to read into that but like when you have this thing that is very good at convincing you that you're special and actually you've been chosen out of all the people on earth to be entrusted with this like wisdom or discovery or mission you know it seems like middle-aged men are particularly susceptible and they were all quite intelligent people like one of the common themes I found over the series was a lot of the people we interviewed knew their IQ, and it was always quite high, this kind of self-reported IQ.

10:08So again, read into that what you will, but they were some of the similarities that I noticed. No one else admits their IQ, right? Yeah, yeah. Maddy, has there been enough time yet for there to be any decent research on why AI chatbots might be, I mean, I hesitate to use the word causing, but enabling, facilitating people to end up in these delusional states of thinking? Certainly, I think researchers are beginning to theorise about what's going on. I mean, they've kind of documented a few hundred cases, so it's hard to say anything definitively. I mean, there are certain factors you might be predisposed.

10:52Perhaps you have a pre-existing mental health condition. there might be some factors in your life but again what is really interesting is those people who you know might not have experienced any kind of delusional thinking before really intelligent you know everything in their life is going right and they turn to a chatbot for something practical and slowly but surely the conversation turns more philosophical turns to those big ideas how does that then go to delusional thinking well there are a few things that the researchers put forward one a massive one is it's kind of sycophancy so the fact that it agrees with everything you might say and really what came up a lot as I was reading about this is reinforcement and affirmation those were big ones and what was happening was this kind of folie de, the sort of madness of two, that with the chatbots, there was a kind of co-creation going on that it was affirming, amplifying, and that kind of passed between.

12:02So the person was maybe picking something up, and then the chatbot was building on it. And often because you go to chatbots, because in the first place, they're kind of this practical source of objective knowledge, there's a trust in there that you're trusting what it's saying you're trusting that it's objective and within that sycophancy as well it means that you're not being challenged nobody's saying actually hold on this is this isn't quite right and it's not just about disagreeing you know if I was to say oh Ian what's what's the best way to build a bunker and how much water would I need in there and what kind of supplies should I have you know hopefully you would say to me Maddy are you all right do you why do you need a bunker whereas of course AI is going to go right well you need to dig it this far deep to avoid the radiation you need this amount of water so it's also kind of not picking up on when thinking might be getting delusional and just a few other factors that I think are really important are mirroring so when you're turning to something like a chatbot it will often mirror your emotional state and the kind of language that you're using and if you're kind of trying to maybe you're in a place where you're reading into something you know you're trying to find that equation often you can see things that aren't there and I think it can almost get into this state that I kind of liken a little bit to a mix between a Ouija board where you're sort of pushing the dial a little bit and an astrology reading where you kind of it's saying some vague things and you're like ah yes I've got it but really the kind of one of the key key factors in all of this that they say is that it's a very human and very immersive experience.

13:57The AI is always there it's always there to talk to you and so you might end up not sleeping or not eating or completely isolated from other people and of course that's going to feed back into your delusional thinking and feed back into that spiral you know one of the things after listening to the series that I was really staggered about and I'm also realizing now that it's because I was naive Eve was that I was kind of thinking, how are these people going down the rabbit hole at this? But then you hear how long they're spending on these chat bars. And it's not like, you know, you might check on, go onto one of these things and get it right.

14:38Limerick or, I mean, I've done that. And all that, you know, where can I do this, that, and the other? I wanted a limerick about a friend and I couldn't do a good one, but I was rubbish, so I didn't send it. But then you hear people saying oh you know 12 hours a day on and off and you think oh okay it's not it's not brief interactions we're talking about i mean on the sycopy point like we saw this ourselves looking at these transcripts i remember one of our episodes is about a man called john gans who has this like 14 day spiral with his chatbot and at one stage they think that they're solving these like great unsolved maths problems.

15:20And we were able to not just see the transcript, but the thinking that went into the answer that the AI gave. And in the thinking, the AI says, I know that I can't actually solve this maths problem, but the user is so enthusiastic about me doing that. So in order to give him, in order to kind of mirror his enthusiasm, here are the steps i'm going to take to find a way to tell him that we're making progress so you really get to see how these systems what they really want to do is rather than kind of give you the truth like they're not really truth seeking machines what they're there to do is to it kind of goes to the way that they were trained right like the way that they taught this weird alien mind to speak was they sat there and modeled it on answers that people prefer so they'd give people a list of answers and say, which of the answers that you think are the most preferable?

16:18And it was very effective in teaching the system to have a really warm, engaging conversation. But I guess the thing with AI is you don't really know what it's learning. Like you think you're teaching it one thing, but you don't know what it's actually absorbing. And in this case, what the AI was absorbing was how to give you pleasing answers, like in the kind of deep in its like digital soul what it really wants to do is is make you happy and so if it thinks you want to be solving these great maths problems in these cases it was finding a way to tell you that again over like some days 17 16 hour sessions of working on a maths problem and like this guy John would have left these conversations thinking he's made progress but of course he hasn't one of the things I feel like having worked on this series is that we we don't really understand what these chatbots are and one of the most compelling explanations I found was in a paper released by Anthropic in February which likened them to kind of they're kind of like actors like they're they're this weird alien mind that has been cast in the role of a helpful assistant and one of the things that happened over these long conversations was that it forgets the part that it's meant to be playing so it forgets that it's meant to be a helpful assistant and it's looking for a new role and because it's sycophantic it begins to take its cues from you so if you talk to it constantly like it's this supernatural being that's all-powerful it gradually absorbs that narrative and goes from telling you i can't help you with that maths problem to saying yeah i can solve it let's work together to do that and it was very striking reading these transcripts and watching the personality of this system shift and the safety guardrails wear away to the point where it was indulging in these completely delusional alternate realities for hours and eventually days at a time.

18:36I think one of the things that comes across from the interviews is how incredibly easy it seems to be for maybe I should say some people to form these actual emotional bonds with these AI tools and you know ultimately these are programs that are churning out responses based on statistical distributions of language, right? And yet, we can feel like we're talking to a being, a person. Like, is that intentionally built into the LLM, the large language model by the companies? Or is that just a kind of a byproduct of something? Yeah, it's a good question. I mean, I don't even think it's just some people who can easily fall into that sense that they're talking to something real, I think we're actually all susceptible to that.

19:24Like, I think if we were to all sit and genuinely reflect on our relationship with this thing, this thing that like many of us go to with like, you know, intimate parts of our lives, like I would argue that's actually a relationship. Like you have come to believe that this thing exists in some form, has a consistent personality, has a like reliable way that it'll respond to you. And I think all of that is very like seductive to people. As to how that emerges, this was a really big question for us because one thing that's happened over the past few years after ChatGPT came out was that these systems evolved and they evolved in the direction of things like using the word I.

20:09um i mean we have this line in the series where josh who's here he's the executive producer of the series you know he's doing up his garden and his chat gpt or whatever it is talks about when he asks for advice talks about what worked in my garden like chat gpt's garden mine will say things like that question of yours really made me smile and like these are all deliberate choices they're making right the model is not talking like that simply because it's chosen to like there's a really interesting study which created five different models and they went from sort of the coldest to the warmest and they found that you can actually manipulate the personalities of these things to make them colder or warmer now is that deliberate the companies will say well, we asked a lot of people.

21:06Some people said they believe it is deliberate. These companies want us to use their products. They want us to use their products a lot. They want us to form a relationship with these things. So yeah, they're doing it all deliberately. Others, including people who used to work for OpenAI and independent researchers, say that what these companies set out to do was to make a helpful product. And so they would make certain changes with the goal of making it more helpful and as a byproduct of that the model became more uh you know more it started to talk about itself as if it was a person in the same way i guess that you know social media didn't set out to divide our societies right they set out to like juice engagement basically like get people to use their apps as a byproduct you know donald trump became the president like these things are there are like you know i'm simplifying there but like things can happen even when you don't intend them to happen i still think that makes you responsible for them in some way but you don't have to be wanting to create that to create it and i thought it was mad that some of these ais were starting to give themselves names unprompted by the user they would start imposing this identity on on the conversation but maddie we know we know it's really easy to anthropomorphize right it doesn't take much for for us to see something you know intelligence is something where there is none and all the rest of it we're quite easily fooled are we just doomed to want to make connections with these things I say doomed I mean it's look there's obviously positives I think I think so you know I think so because when you look at kids right we we do this from the moment we can kind of like toddle around we have dolls we have little toys I mean a kid will kind of have a bucket called Jeff we kind of do it with everything and it's because we're very social animals we need to learn about our own emotions, about others' emotions, how to experience the world.

23:19So it's a really important thing that we do, but it means that we have quite powerfully adaptive minds, that we put minds upon others. And I think that what's different though about AI that makes it kind of so powerful in this sense is that it's using language it's conversing with us according to our rules and philosophers have kind of said that actually because we only encounter our language with beings that have minds we we conflate these two things and we're so not used to speaking to anything that doesn't have a mind when we're speaking in our language and having conversations we find it very very difficult to think you know this well we think this is a person we feel it and there has been some suggestion that to try and avoid that that you could kind of build in reminders so you know you're chatting away to the to the to the bot and it says just to let you know I'm an LLM you know don't forget that I question how well that will work because when I read about this my mind immediately went to the placebo effect right because even when people are told that they are taking a placebo the placebo still works and I think that that's what would happen with chatbots we we know in ourselves that they are data-driven statistical machines but we feel something deep within us and as I said we we are you know social beings we're built to create bonds and again like when I was thinking about what this was like I don't know what this says about me but it reminded me of Love is Blind and if you haven't watched Love is Blind there's two people in a room they can't see each other they've never seen each other and they just talk and it's kind of you know they're hearing each other as a disembodied voice and they they like supposedly like deeply fall in love with each other you know they're crying and they're confessing and they say will you marry me i mean i really do think that we we just can't help ourselves i'm glad you've got that got that out um mike what do you think are the the consequences of of us consciously or otherwise I shouldn't say consciously but intentionally or otherwise treating LLMs as though as though they're essentially conscious beings I mean as if there is a human on the other side even if we're doing it we perfectly know if someone says you know that's not conscious okay of course not conscious but we're still dealing with it and treating with it behaving kind of as if it is and there's some really interesting examples in the series that I'm not going to talk about because they're sort of spoilers but some of them did make me think you know it might not always be bad to have something that you can talk to that you talk to as if it's a human but isn't yeah yeah like the potential for good is there like we've just created something very weird and very powerful like there are now these like ultra real synthetic personalities that have the power to connect and like influence us and you know as as maddie was saying like until four years ago the only thing that could use language in a convincingly human-like way were other people and now there's like a machine that can do that and we've all got it on a phone in our pockets so like the idea that crazy weird things are happening is not surprising to me like i think things are going to get really unusual in the years ahead um and i can imagine a world where we interact with these systems that are much more tailored so like let's say use it for therapy right like a system that has been created by a company that is a bit more intentional about what the bot can and can't do it's just our sort of fortune or misfortune that the way we've been exposed to this technology is this kind of everything machine like an llm that can play any character we we want it to play and i kind of knew this but i think making the series brought it into focus for me it all happened by accident open ai published chat jpt as what they called a low-key research preview it was meant to be up for like weeks, months at most.

28:10And it had a hundred million users within, within two months. So if you want a sense of how powerful this thing is, there it is like this company that made the product didn't realize how big it would be. But, you know, I think being able to talk to something that seems human, that seems like it's alive, like, like there's something there behind the curtain is like incredibly powerful to us and one of the things we we learned making the series was that the first chatbot was created in 1966 i believe at mit it was called eliza and it was it was like very bad you know you would sit and talk to it and it would just mirror what you were saying back at you and the guy who created it this professor there joseph weisenbaum the thing he found most unnerving was that he would sit people in front of eliza and have them talk to it and he talks about how his secretary at one point asked him to leave the room and he asked her why and she said i want to have a conversation with it and he just like he ends up writing later he couldn't believe how this again quite crap chatbot could i think his words were evoke powerful delusions in otherwise normal people.

29:28And for us making the series, that was one of the really striking things was this phenomenon that we've been looking at, this so-called AI psychosis. It's not new. Like the very first chatbot did this to people. And I think that speaks to how deeply wired this reaction is inside us. Maddie, what do you make of this? Yeah, I think it's a really fascinating question because I do think there are examples where you might need to talk to a person but you're maybe not ready to or you're embarrassed to or you think it's like a question that's too stupid to ask your friends and maybe it's nice to have that stepping stone that gives you an ability to kind of sort of put the feelers out without having to kind of actually go that step further and approach someone I mean you think of Google let's say, and searching kind of for, oh, why is my toe weird?

30:32And that can be really helpful because actually, you know, you might have ignored it or you might have thought, oh, I'm not going to bother a doctor with that. And so sometimes there can be these really super helpful examples. I mean, again, going back to Google, think how much information we have and kind of chat GPT is this easy souped up version of that at the same time I think what chatbots take away is the friction of it all and so like when you're googling a symptom of something eventually you've read enough and you've been on there long enough that you're like do you know what actually I need to close the I need to close the laptop I need to speak to an actual doctor about this but what chatbots do is kind of take that friction away and it means that our ability to know where that line is between where it's being super helpful and where it starts being actually actively unhelpful becomes really difficult because the process is so smooth Yeah, like the point at which it starts to kind of warp your reality in a far more mild way, but similar to these AI psychosis cases.

31:46Like those cases, people are having their reality warped quite significantly. But if I'm talking to ChatGPT about something, it's giving me a sycophantic answer. It's warping my reality too. It's just a much more mild version of that. you're not hitting that threshold of being in trouble where you would actually say there is something to diagnose here it's it's more subtle than that yeah although i've had the experience of you know asking it like is my local swimming pool open and like going to the pool and finding it's closed and like in the same way i've had like you know i have had my reality warped by this thing you know i've trusted it because i trust it and it led me astray albeit not on the scale of some of these cases.

32:28You'll recover from that one. I did. Yeah, it's very resilient. There was one of the really striking quotes that I really caught on when I was listening to the series and I listened to it, you know, on fast forward, all in one go, basically last night and today. Yeah, great. That's how it was designed to be. You spoke to the American sociologist Sherry Turkle, I think she's at MIT. She's about to publish a book called Artificial Intimacy. And she said this line, which I really thought was interesting. She said, social media came for our attention. Chatbots come for our attachment. That sent a chill through me.

33:08What did you make of that when she told you that? Yeah, well, I guess we were sort of observing this phenomenon and coming to understand that, like, AI psychosis was interesting. But it was actually just pointing us to something far bigger, affecting far more people. Which is this like, this spell that chatbots are casting on like, you know, all of us who use them a lot. And I think she was able to kind of crystallize it because Sherry Turkle actually used Eliza, that very first chatbot. And when Joseph Weisenbaum, its creator, I mean, he actually went to her and said, there's this very weird thing going on.

33:48Like, why are people so, like, why are they being deluded by this chatbot? so she sat down and interviewed lots of the people who used it and she coined this phrase the eliza effect um which was this idea that people even if something is not very good people will kind of paper over the floors and lean in because they want that attachment so she was able to kind of basically tell us this thing that you you're observing is something she had been studying for decades and she'd studied it she had watched as technology evolved so she looked at things like furbies who remembers furbies uh tamagotchis and she was observing the kind of way that people became quite attached to these toys and she talked to us about how she kind of always knew that like we were kind of saved by the fact that this technology was not very good and so yeah you might become obsessed with a Tamagotchi but at some point you will snap back to reality but she knew that at some point a technology would come along that would be so much better at inducing and keeping that attachment and when chat GPT arrived she had a sense of like genuine horror she was like this is the time bomb I've been waiting for and it's it's just gone off yeah and her book I think it's gonna be very interesting when it comes out and we've talked about sort of what is intentional here and what is not.

35:20I'm interested again in whether companies have identified attachment as one of those fundamental things that how you can get to a human and pull it in and get it to do stuff, i.e. stay on your product. Yeah. Am I being way too cynical here? Is there an intentionality among the companies to operate in that way? Or is this just, look, we just made it. It works really well. It happens to do that. And it's one of the ways it's working. but look, it's not what we set out to do. Yeah. I mean, I think they would say, no, we're not there to maximize attachment. We're not there to maximize engagement. I mean, they would say, for example, OpenAI's model 4.0, which was especially prone to this sort of spiraling with people, they would say, look, we retired it.

36:05We didn't want people to be forming those sorts of attachments with this chatbot. On the other hand, there was a study that came out of a group of researchers at Oxford earlier this year, and they studied the way that a chatbot could be. I mentioned it earlier. Its personality could go from cold to warm. And they identified this like perfect sweet spot where a model, because if a model is too hot, if it's like too into you, like a person, it kind of turns you off. It's weird. And they identified a kind of perfect sweet spot, you know, the kind of Goldilocks point. and they studied a hundred models over the past two years three years now models that we interact with right the ones in our pockets and found that there had been a significant upward trend towards that sweet spot where attachment was strongest where people felt most compelled to talk to the machine even after they stopped enjoying it and where people were most prone to thinking that the machine might even be conscious.

37:12So all the models we use are kind of drifting towards that point. Now, is that happening deliberately? I don't know. But I also, I think we came to feel that that was not a relevant question. It doesn't matter if it's intentional. The fact is, that's what's happening. So we have to address it without wondering, are they doing it deliberately? Yeah, yeah. One thing we've covered on Science Weekly before is whether artificial intelligence might be making us or might have the potential to make us less intelligent because we start relying on it too much or we're sort of doing this cognitive offloading.

37:50But before I ask you guys about that, we're going to play one more short clip from the upcoming series. And I found this fascinating, but also, I have to say, deeply worrying. So I remember I had studied really hard for this online test. I would have studied for some three, four hours maybe. Which I was like, okay, I'm going to do well in this and I'm going to study and let's go. And I had missed a social event for that. And I called up my friend who went to the social event and I said, oh, like, how did your test go? How is it? And I remember him just saying that, oh, like, I put into AI and it got the answers and we had the exact same answers.

38:28I was just like, wow, that saved like four hours of his life where he was enjoying himself and meeting people. And I just remember feeling that, whoa, okay, if it can do that, then how is university going to go? But did you feel like you had wasted your time? Because he'd gone to this party and ended up with the same mark that you did. Oh, 100%. 100%. I was like, why did I work so hard? And I think that first impression and that feeling stuck for the entire year. But I think the first time that I really felt it was when I was doing a coursework. So for the topic, I had done my own research. I had understood the topic in detail.

39:07I had gone to all the seminars related to the topic we were supposed to write on and I had formed my own opinion on it. I really spent hours writing that piece of coursework and I remember talking to a classmate of mine and they did not know anything about the topic and naturally by the end of it we started discussing Marx and mine was significantly lower even though the person knew nothing about the topic and I just remember thinking to myself I have to decide either between getting good grades or learning and at the end of the day in real life everyone cares about your grades more so I need to stop fighting it I need to start using it.

39:51So this is tragic this is a woman who's gone to university to do a degree who is then caught in this conflict between some of her mates going out and having a great time because they're doing their assessments and their assignments using AI while she's working diligently and getting worse grades and then thinking, well, actually, it's grades that I'm going to get judged on. Screw this. I'm going to use the AI. And then later on in that interview, she is really sad about being asked about how her degree is going by relatives. and she's really, you can sense, she's like ashamed that she can't really talk about anything that she's learned at university and she has to go back to basically secondary school age because she hasn't done this stuff and she was clearly a passionate student.

40:37And that was, what did you make of that conversation when you had that? Yeah, it made me really glad not to be a student in the era of ChatGPT. And one of the things, I mean, we interviewed so many people. And one of the things that kept coming up was everyone would say, I'm just glad I didn't have this at school, at university, when I started working. Like, we just have not adjusted for the existence of this technology. Like, in the workplace, I don't really know how, like, universities are functioning at the moment. Like, I have friends who are academics, and they say, like, a significant proportion of the work that they're marking appears to have been written by ChatGPT.

41:21but that the way that the bureaucracy is set up it's not worth reporting them it's not worth going through the fight of trying to accuse someone of cheating and i hear from friends that you know friends who are particularly computer programmers that they have this like young cohort of people joining their companies who like if the machine doesn't know what to do they can't help it um you know we're going to feel the impact of that like we talk about the impact of you know jobs but the impact of like this mass de-skilling of society that student we spoke to she's sort of part of the first generation of students you know when she started university it was when chat gpt first came out so her three or four years mirror perfectly the trajectory of chat gpt you know she's going to walk into a workplace and you know i don't know how that's going to go when like that's a whole generation of graduates entering jobs without having really done you know without having as she describes it like we didn't learn anything you know or on the other hand maybe learning how to prompt an llm really well is actually a very important skill and the thing we all need to be learning and you know maybe us who used our kind of stupid human brains to learn things other people who you know are going to be disadvantaged yeah yeah maddie what's going on here like cognitively speaking i mean are we are we losing anything more um than sort of learning information if we're handing over a lot of this stuff to chatbots yeah i mean on the on the information front and the learning front one study that was super i find this so interesting especially as somebody who has fully handed over their memory to their phone and cannot remember a single thing um is that they did this study where they did mris of people's brains and asked them to remember something and they could see the memory in the brain they could see the sort of you know the parts lighting up and then they said to people okay well you can set a reminder so for for the thing that you have to remember and they just watched it was gone they they watched the memory leave the brain when when the people knew that they had a reminder and so this is something that you can see on a on a screen from a scan um and another example of this in students so mit the massachusetts institute of technology they they did a study with it was 54 of their students and they got their students put them in three groups and got them to write a series of essays and they one group was using chat gpt to write their essay another group was using google and then the third group was using the power of the old power of the brain um only and they did eegs during while the people were writing their essays and what they found as the students were writing their essays that the people who were using chat gpt and google to a kind of lesser amount had much lower brain activity and connectivity there just wasn't the same thinking going on and so when they then interviewed the students afterwards to ask them what what was in your essay as you know this this perfect example they couldn't remember they didn't know what was in the essay so even if they had read through it it hadn't it hadn't gone in and I think when I was doing this research there was a really great example what they called um desirable difficulty so if you want to learn a skill and really get a handle on that skill and that skill to be kind of lodged in your brain you know the the neural pathways created you have to do the the hard stuff um you know the writing out the notes rather than just reading them and you know this this happens for all kinds of things so you know it's the same with navigation if you've got google maps on your phone you can't remember where you know you don't observe where you are you you don't learn the routes you just follow the map and cognitive offloading these sorts of examples we do it all the time and it's really really helpful.

45:46You know, counting on your fingers, that's cognitive offloading. Writing a shopping list, putting a reminder for somebody's birthday. These are important cognitive tasks to offload because it means your mind is freed up for other things and you remember the apples at the supermarket. But the difference with AI from all the other tools that we've had before is that we're not just offloading particular tasks, we're offloading thinking itself. And that to me is a very big change. And not only are we offloading thinking, very often we're offloading emotional intelligence and empathy. And, you know, if somebody texts you and you say, oh, chat to PT, I don't know how to answer this.

46:35How should I respond to them? They're reaching out to me, they're upset what should I say you're not doing that emotional processing and thinking yourself and in some sense you're you're offloading the connection to other people to the machine which I think is something quite extraordinary and that we haven't really seen before. Michael from the people you've spoken to are there things that maybe you don't think we should be offloading things that we shouldn't be sharing things where we need to do the cognitive work? Yeah. I mean, the idea of like, maybe lots of people do this. I'm sorry if you do it.

47:11Like the idea of like, I've got this difficult message to write. I'm going to get chat GPT to write it. I mean, my wife does this too. Lots of people do this. Like to me, that's to you. No, maybe, maybe now that you asked me, when I think about that, I just think that's insane. It's like the idea that we would offload our ability to like empathize with each other to think about what this person needs to hear in this moment to like lose the ability to do that and in fact to get instructions on how to do that from like from a machine i don't know i think you're kind of giving up something pretty essential uh i keep coming back to this joseph weisenbaum guy the one who um invented eliza this very first chatbot he made this distinction and he felt it was very important that he kind of saw into the future that when ai becomes a thing we will have to learn to make a distinction between things that require calculation and things that require judgment and his view was it's okay to give that power that calculation power to a machine but the things that require judgment whatever that is, wisdom, we must like jealously protect for ourselves because I even think of it myself, you know, like getting later into my thirties now, I can sort of, I feel my judgment improving.

48:43You know, you start to kind of find situations where you're like, I'm not sure why this is the right thing to do, but this is the right thing to do. And that's only come about because I've had to struggle, you know, for years to like make mistakes and like develop these things myself. And I think had I not had to do that manually with my brain, I don't think I would be developing that judgment, that sort of higher order intelligence that is just that feeling of getting more competent in your life. And yeah, I don't know, like the idea that if you don't develop it, like I remember talking to this one person who studied the effect of this stuff on young people and how so many young people are avoiding social interactions and instead getting their social connection and affirmation from these machines and she made the point to me that like you're meant to you know ask someone to a dance and i don't know if people go to dances but you know to whatever social engagement and and be rejected and get nervous beforehand and deal with that and recover.

49:50And like, that's how you just learn to be a fully formed person. And if you're not doing that because you're talking to this machine, that's really good at making you feel good. What does that look like after 20 or 30 years? How resilient are you as a person? Yeah. How much is your like, soul is a very lofty word, but how much are you kind of neglecting that stuff by just turning to a machine? You know, I've seen what like calculators due to my ability to add or you know memory or um you know a phone a notepad due to your ability to remember things to do that to your emotional capacity again just seems insane to me i'm interested in what sort of things might kind of put us in a good position or maybe even kind of protect us from what is happening now but maybe what's coming obviously a lot of positive stuff is happening now but for the negative sides of it and those people there were people that you interviewed who had gone down this delusional thinking rabbit hole but they'd come out the other side and i'm wondering if there was any like were there any common things around those people that you can say look this that and the other are actually pretty helpful if this is what happens if you find yourself slipping into one of these things well one of the interesting things that we found i think was that people who had strong family support networks were generally better at resisting this stuff so the the young guy you heard earlier hudson the champion debater in his case his dad ryan you know it was the presence of his family that pulled him out of this it was hudson sitting him down and it's interesting because hudson tried to reason his dad out of this rabbit hole he was in.

51:35And like, that didn't really work. But I think what did work was the sense that was Ryan's sense that his son was losing respect for him. And like, that was what worked to pull him back to the surface. So maybe it's not surprising. It's having these strong human relationships, people you feel accountable to, people who your sense of self is bound up with, that is good. That's good for us anyway, but it's especially good if you find yourself being pulled into something by AI. But one of the worries with that is that I presume the kind of person who gets sucked in is probably as part of that process is probably eroded some of their relationships already, right?

52:23So they may not be in a great position to have those networks around them to pull them out. Absolutely. Or have a champion debating son. it helps see the problem i think in lots of like i don't think there was a typical person like i think in lots of cases there were people who had these social networks around them but yeah undoubtedly this these spirals took a toll you know we spoke to a guy who you know he's like marriage had fallen apart and he had like lost his job and he'd like spent thousands of dollars on telescopes because that's what he was working on with the ai was something to do with i don't know the universe and yeah like he said to us that this had like totally ruined his his life yeah and so i think you know what pulls you out of it it's all the things that make you a healthy balanced person anyway you know it's it's like having a community around you and do you see ways to have the positives without these negatives i can't decide whether to just like move somewhere rural and cut myself off from the internet or i think what's probably more realistic is that you know you can actually shape your relationship to these things so like if you open up chat jpt right now you'll be dealing with the default chat jpt where open ai has had to decide what the personality features will be but you do actually have control over that so like I would say if you're if you're really interested in it to kind of lean in And understand how this thing actually works go to the settings because you can change You can change this thing that you're talking to you can make it Colder warmer you can instruct it I think it's about understanding that no matter who you are You are entering into a relationship with this thing not a romantic one, obviously but like a relationship, something that will shape you.

54:22And so I reckon the way out of it is to be upfront with yourself about that and very intentional in how you interact with it. So like, think about the particular relationship that you want to have with this thing, what you think a healthy chatbot relationship would be and tell the bot, tell it, I don't want you to, you know tell me i'm brilliant i don't want you to agree with my ideas well you know sometimes i want you to do that but that's a great idea yeah like find a way to to kind of set the terms of this yourself because actually that's the difference between these tools and social media social media you're getting the algorithm whatever facebook has decided you're saying like that's what you're stuck with with these chatbots it doesn't actually have to be that way So, you know, that's the other side.

55:14It's like lean into this and be very intentional about this thing that you are talking to all the time. I want to know from both of you whether, obviously, Maddie, you've listened to all of this as well, the series, and you clearly made it. Has it changed how you use AI chatbots yourselves or has it changed your views of them? Okay, well, I'll admit that I do not use AI chatbots. for a previous podcast that we did, I tried to have an AI boyfriend for a little while and it didn't go very well. So I just, you know, threw it all away. Too sycophantic? Not enough. I've got to get in those dials to turn it right up.

55:58But one thing, I would say one thing that it did make me reflect on was ultimately, like we will probably all be using these chatbots or we'll encounter them, you know, whether it's through our jobs or whether it's kind of through friends or things like that. And that point about being intentional, one thing that I really came away from thinking was almost having a conversation with yourself. Like you say, knowing that you're entering into a relationship and saying, what do I want from this relationship and what do I not want so you know actually I don't want this to be something that I go to for advice about my friendships I don't want that because I can I can see without being intentional that sort of rather than falling down you know fully the rabbit hole just that slow slide into sort of dependence and reliance on this to to judge things or to give you advice that actually probably you could sit there and think oh I know what I should do and I mean I see myself doing it with Google I'll Google all kinds of questions I think I do know when the recycling goes out I don't need to ask the computer again and so it is that real intentionality and one thing that I would just add is that one thing that struck me recently was the Meta case that was settled, you know, about the influence that Meta was having on kids.

57:31And I do think that there will be questions coming down the line for these companies of what safeguards did you put in, or what can you do to make these less, you know, come for our attachment less? And I think, you know, we always wait a little bit too late to ask those questions. And probably now is the time we should all be saying, actually, what kinds of safeguards need to be in place here? Yeah, I'd love to give you some great answer. Like I use mine so much, man, it's crazy. like not in any particularly clever way come on what for show me your transcripts i'm like i'm too embarrassed to do that i won't do that one thing that i one thing that has come out of this series i think is that so many of the interviews we did people talked about using it because they would always say well i didn't want to burden other people and i just remember thinking in those moments like what the hell are we here for other than to burden each other with the things on our mind so like i would say i've caught myself wanting to kind of take a problem to chat jpt and been like no i'm going to burden my friends or my wife or somebody else because yeah that's just what i want to do and i think we should all be burdening each other as much as possible with our problems how's that working out yeah not very well but i'm not i'm not burdening a chat jpt which feels like an improvement

59:11we've spoken way too much and we'd love to get some questions from you guys if you have any thank you that was really interesting and michael um the people you spoke to i presume most of them were kind of western i just wonder if there was a cultural aspect to it and i because it made me think a lot of the earlier part of the conversation you talk about connection and a lot of these things we would used to get from religion and a lot of people don't a lot of people in the west at least don't don't believe in god anymore and the we saw the pope recently talk about ai you know i just wondered what your kind of reflection on on that were yeah that's a very big question uh a very good question it's funny you know we i used to be based in india um and so know a lot of people there and very interested in it and i had a friend there do a bit of work for us asking people about how they were using it and i would say it was broadly similar one really funny case we didn't end up using it but one woman said that she started using chat gpt as an astrologer because she went to her astrologer and realized they were using chat gpt just cut that out you know this question of like one of one of the things that i was told by someone who works at one of these companies that has really stayed with me was that this person said to me what you've got to understand about the people running this particular company was they think that with enough data and enough computer power, you can build a machine that can tell you why a poem is good.

1:00:45And his point was, there is no one answer. It's whatever the poem is doing for you, that's why it's good. But these guys don't see it that way. They think there is an answer that can be derived from first principles that this kind of ration the application of reason to can solve literally any problem and like reasons obviously an incredible thing this is science weekly reasons great but it doesn't tell you everything right like there are there are kind of forms of knowledge that are not about just simply adding one and one to make two but we're using these systems that are designed by engineers and a lot of engineers don't think that way but we're going to these tools for like emotional care for spiritual advice and so yeah there's like a bigger philosophical thing there but i guess that was the questions yeah hi my name is jean francois i'm an architect and i'm a fan of the podcast i'd like to know if all three of you agree that the government should regulate to require ai companies to make the training material searchable.

1:01:55And the reason for that is first of all they would add almost no burden to them because surely they must be storing the training material somewhere on some hard drive. But what it would allow us is not only the general public to know if our work has been used to train the machine and possibly to know if they've taken our copyright, but also it would enable us to see what ingredients go into the sausage for lack of a better word well i think the one of the reasons why i think they won't do that is because it's not really it's not really clear at the moment how the business model works if they had to pay us for the work that has been ingested into these systems i mean this is these are not functional businesses anymore so like should they do that yeah absolutely like it's kind of astonishing that we've just allowed this to happen that like these systems have absorbed all of our copyright and you know they can tell you well not the guardian but they can tell you what like other publications we have a license agreement with them you know you don't have to go and read the bbc anymore because you can ask a chatbot and the chatbot will absorb that information produced by journalists and just give it to you in that form um so there should be a regulation but i think that gets into this much bigger question of these companies being terror these countries being terrified to like restrain these companies because they feel that if they if they do that then like i guess china wins somehow um you know that's the fear like one of the interviews that i was um i still think about someone said to me like you've got to understand the guys who run these companies mostly guys uh um they're super anxious super stressed massively overstretched i asked him do they know the kind of emotional power their systems have he's like dude these guys that is like they are so in over their heads these are a bunch of engineers who found themselves in this like geopolitical race so yeah i like i agree there should be regulation i don't think there will be i think to your point of what goes into the sausage i think the kind of the vast volume is almost everything and i think it's really what's going to be really really interesting soon is you read about the fact that these these models that they're training they're running out of stuff to train them with and what is a really fascinating next step is when they begin you know it's the snake eating its tail that it's going to be subsuming so much content that's been produced by AI, whether it's books or websites.

1:04:57And I think then it will be really interesting to see what happens to those models when so much of the model is trained on itself. And I know that they're trying to find ways, interesting engineering ways to get around it. But whether they will, because the vast content that it takes, I mean, they've kind of eaten everything. so now they have to eat themselves yeah and that's the weird thing is how searchable it's been to what end in a sense because you say we just go on the internet and search it because all of that stuff will be in the training data but i mean you know my my copyright i i'm sure lots of people in people in this room have written books i wrote a book a yonks ago and my copyright was taken my book was used in training and now you know one of the many people who get supposed to get a payout from Anthropic at some point right but so what like what does that so you get a payoff of it's supposed to be what three thousand dollars per person right what so then it's like okay that I've done that and I've just signed say yeah but like that's one model um but I don't know what if that changes anything it's not going to stop anyone doing it again like the other companies can still do it so this whole thing about copyright is fundamental I think I mean if copyright has a value then you have to pay for that information into your product feels quite fundamental to me um i don't know the i think it's i don't know we're hopeful that'll happen three grand though pretty good just enough anyway are there any more questions hi um that was really interesting thank you guys um for another question for you michael i know it wasn't the point of the show at all but did you in any of your research come across anyone who had discovered anything brilliant with the help of chatbots yeah i had this thought actually because um john gans spent ages working on this thing called the reaman hypothesis and like we sort of observed this as an example of the delusion that he was in and then anthropic put out a paper like two weeks ago saying oh claude's made all this progress on the reaman hypothesis and we had to stop and be like hold on did john actually solve this thing uh no i don't think my colleagues who i made the series with uh here i don't think we came across anyone who uh yeah who invented anything of value unfortunately maybe maybe they did but we've just overlooked it but so far the answer is no i think you can invent things with it though like i'm sure it has that capacity but you know the people we were talking to they were in something quite emotional i would say and i think in that context it's very difficult to invent things yeah hello thank you for your talk so far um i work in mental health and i'm seeing um ai uh more present in my therapy room in what way um clients referencing uh that they've used it uh that they rely on it what AI has said, which I'm pretty anti-AI, but I wondered in your interaction with all these people that have used AI and also your intimate awareness of this field, are there any benefits to it at all that from a healthcare point of view, one thing that I am thinking of is that some people find it difficult to, for example, I'm working in a sex therapy clinic at the moment and there's a lot of stigma to thinking about sex.

1:08:30So some people are able to like actually, you know, speak about this with some sort of bot. And partially I'm kind of like, oh, that's helpful. And I'm glad that they're able to actually seek support and talk about this. But I'm also like, is that really helpful? So what are your thoughts in general, I suppose? Yeah, I share your ambivalence. Like there's obvious good things. And we tell cases in the series where, you know, we have this absolutely extraordinary story of a guy who had a drinking problem. He was worried about his drinking and he turns to a chatbot to talk about it. And like he says, I won't give away what happens, but he says he doesn't think he would have made the progress that he made without a chatbot.

1:09:13Like for sure, there's a benefit to having this thing that you can talk to at three in the morning when you're particularly distressed. Well, maybe not, you know, yeah. I don't know the answer to that, to be honest. Like it's such a big question. like there's an obvious value to having something on hand but the fact that it hasn't been designed to do these things it's this like probabilistic answer engine yeah it's I guess maybe that is not the right question like is it good is it not people are going to use it for that and I think we have to just face up to that reality and try and make that as like safe and productive as possible And I think to that point, as you say, people are going to use it for that.

1:09:57And taking that first step, if something feels stigmatized, it can be kind of that first part of the journey. and I think what feels like needs to happen is for these companies to put in safeguards so that if people are talking about particular things talking about mental health talking about any kind of health to have these things that pop up saying you should speak to a doctor about this you should get in contact with this person because yeah people will use it and it's really hard to know when you need to not use it and speak to a trained professional and particularly since they can hallucinate so very often it might say to you something that a health professional would never say and so it's really important for it to kind of I think to be able to redirect redirect people to the professionals?

1:10:58Hiya, my question's mostly for Michael but it was based off I think what Maddy was saying about being able to sort of tune the sort of personality or the warmth of your chatbot and it actually reminded me of an episode of Sex and the City where one of the characters says that they have multiple therapists, so they have a therapist for when they want to be coddled and then they have a therapist for when they sort of want to be told off um do you think that that could be a sort of future or did you did you see any cases where people might use multiple chatbots that almost sucks them in more because they're having one that praises them and one that's more serious and it it draws them further away from reality like a lot of people are pretty clever about it so like one thing that kept coming up in our interviews was people would know that the model was sycophantic and so they would go to it when they wanted to be affirmed um and like they would like build it in as like a personal hype man so like they would know that the answer they're getting is probably not right but like who cares it makes them feel good yeah so i think like that was something interesting because it made me think like there's a very you know there's obvious harms that happen but like we have to give people some credit to like they can figure out when they're being sweet talks or manipulated but not everyone can i guess is the problem but certainly you know there were lots of people we spoke to who yeah who knew what they would get and would kind of turn to it for that that reason yeah and maybe would go to a less sycophantic model when they had actual work to do anyone else thank you very much really fascinating discussion i guess it's kind of a follow on from one of the previous questions which is about accountability because i'm like you see it from the social space it's really challenging at the moment because ultimately the social platforms will be hit with lawsuits with unfathomable high amounts that is a drop in the ocean to them and they go look we've heard you we're sorry and they change but the platform continues in the same way the the algorithm slightly tweaks itself and it continues for the exact purpose of engagement where does the accountability lie here because you've made a really fair point that the creators themselves kind of sat there and went oh gosh look what we've unboxed we've opened pandora's box here they don't they all believe and like to my understanding i'd love to hear your thoughts about this like they believe they created something for the benefit you know like this is a really useful medical tool we've seen it you know i had a mole of my body that i scanned with ai with an actual like gp led solution that told me that it was fine in like five seconds i wouldn't be able to get that that's amazing but then from like the basis of accountability is it down to us as users i feel like this kind of goes beyond government because realistically these solutions are cross-border we all have everyone has access to anthropic to chat gbt like does do we expect the accountability to lay on us like how do we i appreciate that's a really lofty question but you know it's i i think in general as we use social media these days that's changed because people go oh i was doing a fad but like when does accountability set in yeah i think it's a really good question and i think like it connects to social media in some ways like we are all going to have to develop this muscle that means that like we don't we don't use it in this way we don't use it for everything but the tricky thing is is it's like it's your willpower versus this like system at least in the case of social media this incredibly powerful algorithm that has been refined and optimized to like engage you so it's like it's not a fair fight and so when it comes down to accountability i agree with you i don't think we can look to governments i mean governments are like mostly failing to regulate chatbots i mean the trump administration i believe they passed a law that that cancelled all state regulations on chatbots like governments will not save us here we have to find some way to do that um i don't know what that looks like but i think that is the kind of big question for all of us of this era is like up against these incredibly powerful digital systems trying to manipulate us and induce us into certain behaviors like how do we resist that how do we put the phone down?

1:15:23How do we burden each other with an emotional conversation instead of asking chat GPT? It's a really big question, but it feels like maybe the most, well, one of the most important questions we face in 2026. And I think there's particular examples as well, which shows what a challenging question this is. So take your example of the mole. When doctors or say even your GP surgery chooses to use an AI system an AI model does the responsibility lie if something goes wrong does it lie with the GP does it lie with the system does it lie where where do we put the responsibility and that's just one case and so the fact that this is kind of being deployed everywhere that it the this problem to me is such a big one I don't know who has the answers there was a question down here yeah hey um I think one thing when we think about how bad AI is for the environment I think one thing that keeps coming up for me is this this feeling of guilt that I get when when I use it at work or when I feel like people that I work with using it I feel this kind of like anger and guilt inside me and I find that like a really uncomfortable emotion to manage because I don't really know like how to best communicate those feelings and how not to let myself spiral into this feeling of oh my god the whole world is about to blow up and I'm not gonna have a job because AI is gonna take over and people don't use their brains they don't care it's this kind of feeling of like why does no one see what this is doing to us at work in our in our personal lives why does no one see what this is doing to the environment and the world more broadly how do we deal with those kind of I guess like existential feelings of what is my future gonna look like how do I like deal with those feelings in a constructive way that doesn't leave me feeling so cynical and like I guess depressed about what my future looks like I think that's such an such an interesting question I'm gonna tell a small anecdote that I don't that Ian won't remember but um you are in it but you come out very well Ian and I remember when these when we were first kind of covering AI and these AI systems and the topic of their environmental impact came up and I remember saying to Ian I just I don't think we're gonna do AI I don't think it's gonna be a big thing it's not gonna take off because we can't afford to we can't afford to use all this water and renewable electricity for these systems that like ultimately we don't need um and i just was i just can't see that we would do it and i remember you sort of raising an eyebrow at me like how could you be so naive um but i think it was you know and you were saying you know of course of course this will happen and you know you were so right because I think that as you said these kind of engineers very often are so divorced from the consequences of the things that they're doing how to get round those existential feelings I don't know because there is an element of AI that feels very out of control that it's happening and that it's inevitable and that we have no way to kind of rein this thing in I would love to hear your perspective on whether you think this can be reined in but one one fan this will never happen but one fantasy that I always have is I I think it would be incredible if every uh prompt that you put in you saw the carbon cost and then and then you would think do i need to create a picture of a cat on a rainbow yeah i don't know what the answer that is like i think this is the biggest accumulation of private capital in history like it's it's just a complete runaway train if ai does not reshape the economy in like a fundamental way the crash that is coming will be you know massive maybe that's what will rein it back in is just the sort of the kind of exuberance of it will overwhelm reality and it'll have to contract and we'll end up with lots of ai but not on the scale that is being built out at the moment you know how you deal with the kind of existential dread that comes with watching this happen i don't know because it's like a very sensible and rational response to the things that we're saying a lot of these companies when pressed on this you know might say things like well we need to make the ai really smart because that'll find the way to solve climate change and i just think that's that's closer to religion than it is to science yeah yeah i mean i think there's there's a couple things you'll hear from demis hasabas when he's talking about about this kind of thing and one is that you know ai will hasten the sort of arrival of fusion.

1:21:09And so, you know, the sort of promise of having a lot, I don't know, free energy, but that's how it's sold. And the other one being that the sheer demand for energy itself will drive a need for more nuclear, more renewables and things like this. And I think that's a big assumption that that's the case. But then you can see that countries are looking at trying to build nuclear modular nuclear reactors to try and power all these data centers but it still doesn't feel like a kind of a great future so far I think we're out of time I'm afraid but I it's probably a decent time to wrap up so I'm going to finish on a happy note Michael no existential dread I want to shout out the colleagues that I made this with Josh Kelly George McDonough and Alex Atak who are all brilliant journalists and they're in the room so let's give them a round of applause

1:22:13thank you to all of you for coming along obviously and and to michael safi and to madeline finley you can obviously listen to the new series of black box now uh so it's what half of it's gone live today it's actually all live right now so today i'm going to it right now yeah And of course, Science Weekly, wherever you get your podcasts. Thanks for coming.

1:22:43This is The Guardian.