Is AI changing the way we think?

0:00Hannah, I just Venmo'd you for dinner. Obsessed. I'm spending it right now on the lip gloss that's been sitting in my cart. What do you mean spending it right now? You can instantly spend your balance with the Venmo debit card or when you pay with Venmo at checkout. Stop. Say more. More exactly. The more you do with Venmo, the more you get. Like earning up to 5 % cash back with Venmo Stash. Get the Venmo debit card or checkout online. Venmo Stash bundle terms and exclusions apply. Max$100 per month. See terms at venmo.me slash stash terms. Venmo checkout, not available at all merchants. The Venmo Mastercard is issued by the Bancorp Bank N.A.

0:57Hello, welcome to the Naked Scientist podcast. This is the show that brings you the biggest breakthroughs and talks to the major movers and shakers in the worlds of science, technology and medicine. I'm Chris Smith. Today, from bogus scientific papers and misleading made-up facts to potentially curtailing our critical thinking, we're looking at the effect that mass adoption of AI could be having on the way that we think, the decisions we make and the information we learn and act on.

1:34Artificial intelligence is being talked about and used more widely than ever. Rapid advancements in this technology have, for example, led to many remarkable and welcome breakthroughs in science and major efficiency gains in some workplaces. It seems certain that the widespread adoption of AI is also an inevitability, much like the internet and social media in the years before. But over recent months, AI concerns have come to the forefront, both for businesses and individuals, and that's what we're going to explore this week. One of the main ways that the majority of us are running into AI is via platforms like ChatGPT.

2:13These are so-called large language models, which specialise in understanding instructions that we provide in plain English, and then rapidly generating summaries, explanations, or even diagrams and computer code to order. And it's very impressive. But all that glitters isn't gold, because these systems make a lot of mistakes. And as a lot of internet content is now being generated this way, we are running the risk of polluting the knowledge space with rubbish that could potentially be around as misleading distractions for decades. Here's Mike Waldrich, Professor of the Foundations of Artificial Intelligence at Oxford University.

2:51Large language models are quite a recent development. They've really only been around in the way that we recognize them now for about half a dozen years. And what they do fundamentally is something which everybody who's listened to this program will be familiar with. If you open up your smartphone and you start sending a text message to your mother or your partner and you say, I'm going to be, it will suggest completions. And how your phone is doing that is it's been trained on all of the text messages that you've sent. And it's learned that the likeliest next thing to come after I'm going to be is either late or walking the dog or, in my case, in the pub.

3:30Large language models have been trained not just on a bunch of text messages, but essentially all of the digital information that's available in the world. essentially all of the World Wide Web, which is an unimaginably vast quantity of text. And it is really just ordinary written human text. So every web page is scraped and all the text that's on that page is extracted from it. And all of that is fed to the enormous neural networks that are in large language models, which are trained so that they can make the best prediction possible about what should come after what we call your prompt. And your prompt might be a one paragraph summary of the life and achievements of Winston Churchill or the recent performance of Liverpool Football Club in the European Champions League or whatever.

4:19But all of that text, the whole of the World Wide Web is absorbed by these vast, vast neural networks in a process that takes typically months and requires millions of dollars worth of computer processing power. And what we end up with is something that's really quite remarkable. And when these things are roving across the internet, hoovering up that information, is that indiscriminate? Or would it do it like you would do it, which is you'd say, ah, here's a web page from the University of Oxford. I set that as having greater store and veracity and validity compared to a web page that was produced on someone's blog five years ago.

5:00There is a certain amount of discrimination that goes on, but actually the demand for the data is so great. I mean, you really need absolutely all the data that you can throw at it, that it ends up being somewhat indiscriminate, just because the demand for that data is so vast. I mean, the scale of the data is really, really astronomical. I mean, in the breakthrough large language model was called GPT-3, and it was released in 2020. There was 40 terabytes of ordinary written text used to train that model. That's the equivalent of millions upon millions of ordinary books. And if your demand for that text is so great, then there's a limit to how discriminating you can be.

5:42But interestingly, the evidence is the higher quality text that you're able to train your model on, the better it's going to be. So there is a market right now for that text. There are also, I have to say, some pretty unscrupulous practices that are being used by some of the large language model builders who are, if not actually breaking the law, then probably running roughshod over copyright restrictions and there are lawsuits underway right now about that. So if these things are just ingesting things willy-nilly and looking for connections, if you've got these engines producing this material which then ends up on the internet, are they going to end up force-feeding themselves their own stuff?

6:25Is that like me eating my own excrement. Thank you for that, Chris. Let me put it this way. It seems inevitable to me that within 20 years at the most, but probably more likely within 10, essentially everything that we read on the internet, on the World Wide Web is going to be AI generated, unless you're prepared to pay a premium for human generated content. And yes, you're absolutely right that what that means is that that is the content that models are then going to be trained on. And the evidence at the moment is that models which are trained on AI generated content are poorer. And if you just iterate that process over a few generations, it can quite quickly descend into nonsense.

7:07So this is one of the big conundrums of the age in AI. Where are we going to get that future data from? And how are we going to ensure that models aren't just trained on AI slop, as it's now called? But also what this signals is there is going to be a market for human generated content. Data which is written text or whatever kind of content that's created by human beings will remain extremely valuable. The other thing that we've heard about is that these models do tend to make stuff up. So if they are just stringing together material that they have gathered from across the Internet, why do they make stuff up?

7:49These models have no conception of what's true or false. They're simply designed to make the best guess possible about what should come after your prompt. And in particular, in the absence of any information, they will fill in gaps. But the weirdly troubling thing about this is they will fill in those gaps in extremely plausible ways. And that means that not only do they get things wrong, they get things wrong in ways that are actually quite difficult for us to detect. So here's my favourite example. An early large language model I tried out, I asked it about me. And when we finally got to the Michael Wooldridge that wasn't the Australian health minister or the BBC News reporter, we finally got to me.

8:30It said Michael Wooldridge, professor of artificial intelligence, tick, known for work on multi-agent systems, tick. And then it said it studied at the University of Cambridge. And I never studied at the University of Cambridge. I never had any affiliation with the institute. So why is it getting it wrong? because actually for an Oxford professor, that's quite a common background. And in the absence of any information, it's kind of filling in the gap in a very plausible way. But if somebody read that, they wouldn't notice that because it sounds plausible. So they get things wrong a lot, and they get things wrong in extremely plausible ways.

9:03And that makes it very hard to detect those falsehoods. You may have heard recently that GPT-5 was released. So GPT-3 from OpenAI was really, for me anyway, was the breakthrough large language model. That was really eye-opening. It was a huge step change in capability over its predecessor. And then GPT-4 released about two years after that was also a step change in capability. The one thing I think that would have made the biggest difference in GPT-5 was if it was better at not producing falsehoods. And the problem is, it still seems like that's a really, really big issue. And if I had to identify right now the single biggest barrier that's in the way of the wider take-up of these models it's this fact that they just get things wrong so often they get things wrong a lot and they get things wrong in very plausible ways professor of the foundations of artificial intelligence at the university of oxford mike waldridge there and the same large language model told me that famous astronomer now sadly deceased sir patrick moore started the naked scientists that is definitely not true either now AI can make research faster and more in-depth when issues correctly but AI confabulation like claiming Patrick Moore created the naked scientist could have serious impacts on the quality of scientific literature we're already seeing studies and review articles written using just AI essay mills these are AI generated pieces of writing that also contain completely made up information including invented references created to support a particular line of argumentation as well as facts that are just plain wrong.

10:38But they look and read so well that the scientific wheat can be very hard to separate from the chaff. Some of these studies are definitely slipping through the cracks and getting into journals. In other situations, people are generating articles with AI and then they send them to paid-for publication outlets where others can rely on them without realising that a human didn't create or check them properly. Other AIs can also then ingest and regurgitate this same information. This challenges the integrity of the published studies and the trust between everyday people and scientific information. To find out how trust in the scientific community can be maintained in the age of AI, here is Jennifer Wright from Cambridge University Press, Head of Publication Ethics and Research Integrity.

11:21When you're a scientist, you look at someone else's research and you think, is this good? Is this really correct? Is there something that they've missed? Is there something they should have accounted for? Is that method the right approach? And this is essentially what we do today through a process called peer review. So experts in a topic take a paper, they look at it, they grill the work, they critique it, and then they determine whether or not it's suitable for publication at all, or maybe whether there's some improvements that the author or researcher could make to make it suitable. And obviously, your venue has a reputation to defend, but not all venues do that, do they?

11:59I mean, not all publishers operate the way Cambridge does. there are some where I just send them a very large check and they publish anything. I've come across these on the internet. Unfortunately, this is becoming increasingly challenging. There's something called hijacked journals, which are essentially really sophisticated spoofs of real journals. So let's say you had the journal of interesting research, and that's a legitimate journal by a legitimate publisher. There might also be another journal of interesting research, and that is a predatory journal or a hijacked journal. And they can be very sophisticated so researchers and readers might might not even realize that they're looking at a predatory journal what's the purpose of those places existing that is this people burnishing their cv so if i want to get some extra publications and make myself look like a better scientist than i am i can generate some content and send it to one of these journals who ask few questions but will just generate the publication and that turns into cv points for me i mean is that the purpose There's probably a lot of motivations as with anything like this.

13:03You know, it could be naivety. It could be that the researcher doesn't realize this isn't a legitimate outlet. Could be pressure to publish, which you've alluded to, that a lot of academic careers are built on your publication record. So a shortcut to a publication record might be attractive to certain people in certain parts of the world as well, where they have really strict requirements around you can't graduate until you have a paper. you maybe can't get a promotion until you have a paper so with that incentive structure it becomes very difficult to resist perhaps. Jennifer Wright there. Bogus research and predatory journals are nothing new but recent advancements in generative AI have turbocharged the ease with which content can be produced with plausible papers now popping up at the click of a button.

13:48These studies are often littered with confabulations which unsuspecting readers can mistake for legitimate findings. Here's Marie Soulier, head of editorial ethics and quality assurance with the open access publisher Frontiers. The real concerns are about inaccurate or false content, what we refer to as hallucinated content from the artificial intelligence, hallucinated references. If the AI does plagiarism, if there is poor attribution of content without referring to the real source, This is actually what we are really concerned about, rather than AI being used for the benefits that it can provide, because it is a very efficient and effective tool for analysis of research data.

14:32It can free up time for researchers to carry out more research, and it really, really supports non-native English speakers who are the majority of the population of researchers in the world. Given that it can churn out very slick, very nice looking content that's extremely plausible and could therefore deceive reviewers, because unless they're going to check everything in excruciating detail, things could slip through. Does that not worry you, though, that this is going to create a lot of work for reviewers to be able to say, honestly, I've really, really checked this and I've checked every reference.

15:07I know absolutely this is rock solid, which let's face it, no one's got enough time to do exactly that. Agreed. And what we've been doing is, in a way, fighting AI with AI. So there's a lot of tools that have been created to support publishers in the industry. And everything is focusing on the root cause of the problem. We cannot detect AI generated content. It's very difficult. And there's very low accuracy for this. So what we do is focus on everything else that might be wrong with the article that could be a symptom of a fake paper. So we're looking at tortured phrases, some problematic content, data that looks a bit dodgy, gibberish images, wrong attributions so that they don't get published and become part of the literature.

15:54With a very, very high quality venue like Frontiers that has a reputation to defend, that is the case. But there are lots and lots of journals. They don't have the kinds of checks and balances that you have and the problem that stems from that is the AI platforms that people are using and training are just romping their way around the internet sucking all this stuff up and incorporating it into the model that they then use to generate papers that do go into legitimate sources and reputable venues like those that you publish and so it does have the potential to build up like a layer of plastic on the seafloor and pollute the knowledge space for years to come doesn't it?

16:34Absolutely. And this is a frequent concern that's raised by researchers and publishers alike. I was at the Frankfurt Book Fair last year at the big event. There was a whole AI day with publishers. People were talking about how these non-peer-reviewed articles in archive or in predatory publishers are taken with the same level of legitimacy as peer-reviewed articles. And AI-ers are not taking this into consideration, or the developers are not, Because these articles, the real ones versus the archives, are tagged with a little label that says it's been peer-reviewed. And the AIs should be able to make a distinction, but not with the predatory publishers.

17:14And that's the biggest risk. Marie Soulier from Frontiers Publishing House. Increased vigilance from publishers, editors and researchers is crucially important to combat this problem. But another way to beat AI pollution of the knowledge pool could mean changes to the way scientists document their work. Here's Jennifer Wright from Cambridge University Press again. We also promote open science principles at Cambridge. So you can think of this like the show your working mode of research. It's not enough to just present the outcome. The researcher needs to be able to kind of show the journey, ideally whilst they're doing it.

17:47But a paper trail of this was my lab work, this is the microscope I used, this is when I went to the field, that's much harder to fabricate than some words on a page. So I think that's something else that builds trust is being able to see that journey. It's also about collaboration across publishers, institutions, funders, libraries, all coming together to look at what can we really do about this pressure to publish culture, incentive structures, how can we turn down the tap rather than just filtering the water. So Cambridge are working on this just now. We've got a white paper coming out in the autumn, which is going to look at some of these big systematic challenges and what we can actionably do about it.

18:28Jennifer Wright at Cambridge University Press. It is good to know that big players are taking this very seriously, but the risk that we are poisoning the knowledge well is a very real one, and it has implications for the way we think and learn in future. And that's what we're going to think about next. Hannah, I just Venmo'd you for dinner. Obsessed. I'm spending it right now on the lip gloss that's been sitting in my cart. What do you mean spending it right now? You can instantly spend your balance with the Venmo debit card or when you pay with Venmo at checkout. Stop. Say more. More exactly.

19:01The more you do with Venmo, the more you get. Like earning up to 5 % cash back with Venmo Stash. Get the Venmo debit card or checkout online. Venmo Stash bundle terms and exclusions apply. Max$100 per month. See terms at venmo.me slash stash terms. Venmo checkout not available at all merchants. The Venmo MasterCard is issued by the Bancorp Bank N.A.

19:27The Naked Scientist podcast. is produced in association with Spitfire, cost-effective voice, internet and IP engineering services for UK businesses. Find out how Spitfire can empower your company at spitfire.co.uk. Music in the programme is sponsored by Epidemic Sound, perfect music for audio and video productions. This is the Naked Scientist podcast with me, Chris Smith, and this week we're looking at the effect of mass AI adoption on the knowledge base, our bodies and our brains. Speaking of which, in recent months doctors and nurses have seen a rise in patients being admitted for AI related issues.

20:08In one extreme example, a 60 year old went to hospital in the United States claiming he was being poisoned by a neighbour. He'd been using sodium bromide instead of salt, sodium chloride, on his food for the previous three months after asking ChatGPT to suggest salt alternatives to lower his blood pressure. He developed bromism, which can cause psychosis and hallucinations, and it took him several weeks, free of sodium bromide, to recover. People are also using ChatGPT for mental health advice and to seek companionship. As a result, a new term called AI psychosis is being used to describe users of AI chatbots who are losing touch with reality as a result.

20:50Some speculate the immersive nature of the interactions we can have with these systems can make them very provocative stimuli capable of triggering the emergence of psychotic states in some vulnerable people. I've been speaking with consultant psychiatrist at the Priory Hospital Roehampton, David McLaughlin. There have been a couple of case studies recently where members of the public have developed psychosis, which involves conversational AI like ChatGPT. So people have believed that they're really speaking to a real person, or they've developed paranoid delusions that have involved ChatGPT or other generative AI.

21:27And that's really excited journalists recently. And there's been headlines about AI psychosis. Do you think this is a new phenomenon, as in this is a new risk, a new outcome and it's because of a new technology or do you think that these people are always going to be vulnerable and it's just this that's doing it rather than 20 years ago it would have been the television or the radio that was provoking this exactly that one of one of my pet peeves has been the media misrepresentation that this is a new condition that this is a new disease or new illness that we all of a sudden need to pay attention to psychosis is an illness where people develop hallucinations.

22:08So a hallucination is when people perceive a sensory stimulus that isn't there. So they might hear a voice or they might even have visual hallucinations, seeing things that aren't there. This illness, psychosis, has always existed. The underlying neurobiological abnormalities, so that's the differences in the brains of people that have psychosis hasn't changed. It manifests in different ways according to the world that we live in. So the things that we become delusional about take on the themes of the environment which we're based in. So perhaps 70, 80 years ago when televisions first were invented, people would have developed delusional beliefs that the television was talking to them.

22:54There's something called Truman Show Syndrome, after that film came out, people were often presenting to clinics and hospitals like the hospitals I've worked in, believing that they were being followed by cameras. But it's not the television that caused the psychosis. It's not the radio that's caused the psychosis. It's not the film, the Truman Show, that's caused the psychosis. And in this case, again, it's not generative AI that has caused the psychosis. It's just a theme which this condition presents itself is it not potentially a more provocative stimulus to a person potentially developing a psychotic state because it will basically through that person training it to do so learn to push their buttons more effectively with time so it might be the outcome is going to be the same whether it's the tv or the radio causing psychosis perhaps though what we're going to see is a faster route to a psychotic state through these systems because they're so good at finding out what what floats our boat psychologically it's in the interest of the developers to to create that kind of dynamic where the conversational AI says what it thinks you want it to say and in clinical terms there is a danger or risk I suppose that you get something called collusion and collusion is when somebody around you reinforces delusional beliefs that you have and what I would see with families or friends is often actually it's to avoid conflict.

24:25So I might have a mother who's concerned about a son and some paranoid delusion belief that he has about people following him home from school. Rather than kind of creating an argument, she just agrees with him because it's easier for her to do that. And in a clinical setting, what I would normally ask a family or friend to do is to gently challenge those delusional beliefs, not to create an argument, not to create conflict and shut down the relationship, but to gently challenge that. I would hope that that's what generative AI does and that it's not entirely sycophantic. But I'm not sure if that's always the case.

25:05But at the same time, if you think back about 10 years ago or so, Julian Leff, working down in London, was pioneering the use of avatars and creating technological representations of a person's voices, for example, that they were hearing as a way for them to challenge and push back. So it's almost like we need to tweak how these engines work because we could actually turn them from something that could make a disease state into something that could help to remedy a disease state if they were programmed the right way. Exactly. I'm really familiar with that research. So patients with illnesses like schizophrenia who were hearing these voices, it's almost like training for them, teaching them how to ignore or dismiss or challenge these auditory hallucinations that were sometimes telling them really horrible things about themselves or asking them to do horrible things.

25:52The technology itself has enormous potential and we shouldn't always be afraid of technology but it's more how it's used and just to be I think always take a critical mind that's what that's what I learned when I was a research fellow is always to be critical of information that was presented to you as a fact and always us to challenge inherited wisdom and to take that critical mindset. And I think if we continue to do that, then I think we're safe to kind of keep working with technology. Psychiatrist David McLaughlin there. Many of us use AI technology every day. Large language models like ChatGPT can summarise texts, draft emails and find any information on a topic it can scavenge from the internet and all within mere seconds.

26:38Things that people used to do themselves are being outsourced to AI to save time. But research suggests that by avoiding these tasks that keep the brain ticking, AI use might have the potential to erode our cognitive functions. So, in summary, is AI making us stupider? Here's Sam Gilbert, Professor of Cognitive Neuroscience at UCL. Learning is a physical process that takes place in our brains. When you learn something, your brain changes. But it's not just about what happens inside your brain. I I think learning is a process that extends into our environment as well. We think not just with our brain by itself, but as part of a coupled system of kind of a duet between what's happening inside our brain and what's happening outside it.

27:23So in my opinion, learning and memory is a process that extends into our environment. It's not just about what happens physically inside our brains. So the things in the environment also become part of that process. So if you're a musician, for example, your instrument is also part of the memory process of learning to play a piece of music. Exactly. You're a coupled system. When we train people to learn, though, if you think about how we used to teach kids times tables and they'd learn by rote, and then people decided that was unfashionable. and we taught other people, we taught people different ways of thinking about maths and so on.

27:59Does then potentially introducing a tool like AI or ChatGPT, to name one of them, does that potentially then have an impact on the way people think in the same way that stopping learning times tables or doing spelling tests at school has affected the way that people think about maths and language? I think it certainly does change the way that we think and the way that we remember. one of the shifts that people talk about is a shift from storing information or storing knowledge inside our brain to instead storing where to find that information so it's a shift into thinking about critical evaluation and how to locate information rather than storing the information itself so that can be a good thing it can make people more critical it can make people more engaged in the material that they think about it can lead to problems for instance if you store some important information in a device and that device breaks or the battery dies then you might be in trouble so there are some complex you know pros and cons of using these devices and it's not as simple as good or bad but if it can alter the fabric of our brain and i'm thinking back to the study on london taxi drivers if they learn the knowledge of london all those thousands of streets they get a bigger part of the brain the hippocampus that we know processes that kind of information so if a person's offloading a lot of their cognition onto chat gpt are they bending their brain i mean they're potentially getting less well practiced at writing summaries and abstracts for their science papers or writing short stories and so on so there are two complementary phenomena in this field one of them is called the google effect i don't know why it's called the Google effect.

29:46It's not really got anything to do with Google, but that's the name that stuck. So the Google effect refers to the way that when you store your memories in an external device, you then tend to be lost without that device. You can't remember it anymore if the device fails. So that sounds pretty bad. But on the other side is something called saving enhanced memory. And that shows that when you offload some of your memories into an external device, then it actually helps you remember additional unsaved information that you might not have remembered otherwise. Which do you think though is the best for long-term brain health?