The future of AI

0:16Mike Wooldridge:Hello, welcome to the Naked Scientist podcast, the programme that brings you the biggest breakthroughs and talks to the major movers and shakers in the worlds of science, technology and medicine. I'm Chris Smith and today we're going to examine where AI is headed and how researchers are increasingly coupling different AI systems or agents together, each with different strengths and specialisms to make a much more powerful machine.

0:50Mike Wooldridge:Emerging stocks have been doing incredibly well lately, with the artificial intelligence boom pushing equities in Asia to record highs. But lots of people, including Pope Leo and some tech leaders themselves, are less happy, and they're calling for stricter regulation of the AI sector to protect jobs and privacy. So let's first hear what two experts themselves think. Zoe Kleinman is the BBC's technology editor, and Mike Wardridge is a professor of computer science at the University of Oxford, where he's devoted his research career to the subject of AI. AI is about getting machines to do things that currently require human or animal brains or potentially nervous systems or potentially human bodies.

1:33And one of the weird things about AI is that lots of things that we find very, very difficult, AI has been very good at for a long time. Like so playing grandmaster level chess, computers have been able to do that now for 30 years and yet most people find that a very demanding challenge. Whereas some things that we find basically trivial, like riding a bicycle, driving a car, turn out to be phenomenally hard for AI. So AI is about pushing back the frontiers of what machines can do. And the current approach, which has turned out to work extraordinarily well, I think much better than many people expected, is that if you start with a huge amount of data, in particular, for example, textual data, just ordinary written text, just the kind of stuff that you can pull off social media or the World Wide Web, and you train an AI model on huge, huge quantities of that, it turns out to be rather good at being able to, for example, summarize text, answer questions about text and so on.

2:35And so that's what's led to the current AI boom, which is around large language models like ChatGPT. And the thing that turned out to work extremely well after 70 years is throw incredible amounts of data at the problem. And to process all of that data and build the AI, you need incredible quantities of computer power.

2:54Mike Wooldridge:I was going to say incredible quantities of money as well. Would you go along with that, Zoe? What does the tech sphere say AI does very, very well? Traditionally, AI is basically very good at pattern spotting. And that is because, as Mike just said, it's trained on a huge amount of data. And so it can fairly confidently predict what's going to come next in a sequence, whether that sequences numbers or words in a conversation or anything else that you're throwing at it. It's really, really good in some forms of health care, specifically spotting symptoms on x-rays or scans. I did a story for the BBC a couple of years ago.

3:33I went to see a tool that had been trained to spot the early signs of breast cancer in women's routine mammograms. And this tool had identified 11 tiny, tiny changes that the human doctors had missed when they'd looked at the scans. And, you know, potentially saved these women's lives because they wouldn't have known about it. They had no symptoms. They wouldn't have had another scan for a few years, by which point it would have grown into potentially something, you know, more difficult to treat. It was a really humbling and lovely story to do, actually. And I interviewed one of the women who said cancer was massively in her family.

4:08She was so grateful to this tool that had potentially saved her life. But the other side of that was that there were hundreds of false alarms because we're talking about patient data. And obviously, people have very strong feelings about their medical history and their medical data and the privacy that should surround that. So this tool wasn't given anybody's medical history. So in addition to finding these 11 cancers, it flagged hundreds and hundreds of, you know, lumps and bumps, breast tissue, I'm told is notoriously lumpy and bumpy, that had already been discredited, but it didn't know that.

4:39And I think that kind of sums up for me, you know, where we are with AI and healthcare. We kind of really want it because it can really speed up, really help and come up with new formulas for drugs. It can churn through existing formulas of drugs at speeds we've never been able to manage before. But coupled up with that is this sort of, are we ready to let it loose on everybody's data?

5:00Mike Wooldridge:Downsides aside that Zoe was just referencing there, Mike, in terms of things like data concerns and so on. What about the downsides of AI? What does it not do very well that we had higher aspirations for and have been disappointed? Well, Zoe, I think, has exactly characterized what AI is good at, which is picking up on patterns. What it doesn't do is what many people, I think, imagine it does. Many people imagine that when you give a problem to AI, it goes away and it computes very cleverly the solution to the problem that you've given it. And that is exactly not what it's doing. What it's picking up is patterns in the problem that you've given.

5:40and basically it's matching those against all the examples it's seen in its training data so there's a famous example that went viral a couple of years ago somebody tried um the following prompt how many r's are there in the word strawberry chat gpt came back there are two r's in the word strawberry and the user said are you sure about that yes two r's in the word strawberry are you absolutely sure about that yes i'm absolutely sure would you bet a million dollars and chat gpt said yes, I would bet a million dollars. We presume that isn't a binding bet. But the point is there, the computer code to count the number of R's in a word is one line.

6:15It's trivial, absolutely trivial. And yet that's not what the most sophisticated AI in the world is doing. It's not counting the number of R's. It's just pattern matching and probably just picking up on the word berry rather than the word strawberry. So that's at the nub of the problems that we have with contemporary AI. If you've got a problem which matches something very well in the training data that they've seen when they've been trained, then you can expect a pretty good response. But for unusual situations, situations that are outside the training data, for example, it's not going to be very good at.

6:52And in fact, actually, the evidence is it's very bad at. But also, more fundamentally, what it's not doing is actually computing the answer for you. It's just pattern matching exactly as Zoe said.

7:03Mike Wooldridge:It's not putting us in a position with what it knows already Zoe to discover anything new but that doesn't mean it can't tell us where the gaps in our knowledge are and therefore point us in the right direction does it? What both excites me and terrifies me about AI is that it is at its worst right now than it will ever be right because every single moment it's getting better and these tools and these systems are getting more powerful and they are able increasingly to do a variety of different tasks better than they did the day before, the month before, the year before. I mean, you only have to look at how quickly the industry is scaling up, if you like, to see that this tech is not hanging about.

7:45You know, by the time we're having this conversation now, I mean, I don't know, there might be a tool released tomorrow that will completely validate what I've just said. It's just really moving that quickly. And there has been a sense that the more machines you can throw at it and the more data you can throw at it, the better that that will be. And we have seen that it's kind of slowed down now. But to an extent, that's what this kind of mad chase has been to throw as much money and data as possible at this technology to see what it can do next. And I think, you know, we've seen some real advances in things like coding, the whole kind of vibe coding thing, which started off not being brilliant and is now getting better.

8:22And the cybersecurity stuff is amazing. You know, you've got this tool that Anthropic has built called Mythos that it says is so powerful, it's too scared to release it to the public. Now, you know, you could say that is amazing marketing hype, and it is. But everyone that's looked at it has said, actually, it is pretty good. You know, it's finding hidden bugs in systems, some of them are 30 years old, and nobody's ever spotted them. And of course, this is an incredible, valuable tool potentially to a hacker, right? Because it's the key to every hidden backdoor that there is in all of our digital systems.

8:53And if you think about the fact that everything runs digitally, our banks, our power grids, everything, that's quite a scary prospect, isn't it? And it's not the only one. OpenAI has one as well called ChatGPT 5.5 Cipher, I think it is, that it's also very proud of. And people that select banks and companies that they are allowing to try it are saying, yeah, these tools are another level. It would take a human days, months, years to find these weaknesses in our systems. And this thing just finds it straight it away.

9:26Mike Wooldridge:And therein lies another problem, doesn't it, Mike? Because technology classically, when it moves incredibly fast, has a habit of leaving behind the legal system, which moves notoriously slowly and never catches up. So how do we get to grips with this? How should we regulate AI and do it in a way that it means it works everywhere for everyone? Because that's going to be the problem otherwise, isn't it? We'll all slavishly follow the rules and people who don't want to and have nefarious intent won't. The world's wealthiest companies are grappling with this and by and large trying to avoid being regulated because they claim it will stifle innovation and that other countries will get there first if we start regulating and so on.

10:07So the regulatory landscape is very, very unclear. And it doesn't look like AI regulation is going to be high on the US agenda throughout this Trump administration. So it is going to be very, very difficult, I think, to regulate on top of a moving target. I do think the UK's historical approach to this, which is to, rather than trying to legislate around the technology, that is to say, thou shalt not use neural networks for this, instead get different organisations like the medical authorities, the financial services authority, and so on, to think about the issues that arise there and regulate around those use cases.

10:43I think that's probably quite a wise and sensible way forward. One big challenge, by the way, for government at the moment is they lack the talent, not just the UK government, I think governments and regulatory bodies don't have the AI experts who are at the absolute cutting edge. So they just actually literally don't have the expertise in-house. And our government has made a lot of strides to try to build up that expertise, but it is still a very, very, very quickly moving target. Do you use it, Zoe? Yeah, loads. I use it like a second brain. I use it a lot. I don't use it for writing. I don't use it for scripting.

11:26But I do use it, I suppose, I kind of use it on a par with the way I use Google. I fact check, but then I have to fact check the fact checking, you know, but it's a very useful springboard for ideas or when you just want to know something really quickly. You know, what year did the iPhone come out? Was it 2007, 2008? Something like that. I use it to think through ideas, just kind of brainstorm stuff, I guess. I'm thinking about writing a book about something and I've been using it to sort of brainstorm how that might work as a structure. because I don't know, I've never written a book before, I don't really know how that works.

12:03But I'm very, very keen to start exploring agents. But it's quite difficult with the way things are set up for me at work because I'm at the BBC. But I would like to explore how good it is at admin because I'm really not very good at admin. I'm not very interested in it. I would love to be able to detail more of that off than I currently do.

12:26Mike Wooldridge:The BBC's admin lover, Not, and technology correspondent, Zoe Kleinman, and Oxford University's AI guru, Mike Waldridge. Thanks to both of them. This is the Naked Scientist podcast with me, Chris Smith, and today we're examining artificial intelligence and how it is pushing science and medicine forwards. The Naked Scientist podcast is produced in association with Spitfire, cost-effective voice, internet and IP engineering services for UK businesses. Find out how Spitfire can empower your company at spitfire.co.uk.

13:04Music in the programme is sponsored by Epidemic Sound, perfect music for audio and video productions.

13:10Mike Wooldridge:Now Zoe was just mentioning some of the benefits of AI in healthcare and with that in mind there's perhaps nobody better to speak to than Raj Jenner who's the UK's first official clinical professor of AI in radiation oncology Now three years ago he appeared here on The Naked Scientist having developed a system that uses AI to analyse body scans to mark up organs to find the best routes into the body to zap cancers with radiation while minimising the dose to healthy tissue. He's since taken it much further, allowing the system to bolt on other AI elements, these are the agents that Zoe was just mentioning, that can decode and then incorporate details of other tests and even medical records to build that optimal treatment plan.

13:57Mike Wooldridge:I went to meet him at Adam Brooks Hospital in Cambridge. We'd built an AI tool that could process the scans that are taken for patients who are waiting to start radiotherapy. And the tool could mark up all of the healthy tissues on those scans to help the oncologist plan the treatment and target their tumour. What does that mean, mark up, and what's involved practically? It means that instead of having to draw around all of the healthy structures on every slice of every CT scan, the computer does that for you and the oncologist focuses on targeting the tumour. That means patients were able to get started on to radiotherapy treatment much more quickly.

14:39Mike Wooldridge:Why do you need to draw around healthy tissue if the tumour is the focus? Because the dose has to get into the patient somehow and it has to go through some healthy tissues in order to get to the tumour. A computer helps calculate that dose. If it doesn't know where all the healthy tissues are and what they are then it can't do that job properly. And previously you would have painstakingly gone through what tens to hundreds of slices through a human to say there's a liver, there's a kidney, I want to minimise the dose there, oh here's the tumour. That's correct and for some of the most complex treatments that might easily take four to six hours of an oncologist's time and now it's done in seconds by the AI.

15:21How does it do that? Basically the model has learnt to be able to recognise where it is in the body and paint it with a label that says I'm in the right lung or I'm in the left lung or I'm in the heart and so on and it can do that for every location that we need it to.

15:35Mike Wooldridge:Given it's a number of years since you were doing this you've got a lot of data now so do the data bear out the use of this? Yes it's really nice Chris that what we had developed right at the cutting edge has now become absolutely commonplace in radiotherapy and certainly in the UK in the NHS this has now become standard practice that we have this kind of technology available and it brings a lot of power to the oncologist's elbow. It means that we can work much more quickly. In terms of outcomes though, is there evidence that that's translating into better outcomes for the patient? Yes, it's saving a doctor like you considerable time and stress, but you can work hard.

16:15Mike Wooldridge:You're a nice guy. I'm sure you don't mind putting the hours in. Is it actually making the patient's care better? We know that it saves time and we know that it's safe, but we've actually been trying to explore algorithms like this to actually see if it can make patients' treatment better. And the way that we've done that is basically by asking the algorithm to not just mark out the healthy organs, but actually to predict what a high-quality radiotherapy plan would look like directly. correctly so give that prediction to the human experts which is a very interesting application of exactly the same technology.

16:53Mike Wooldridge:Is it doing that just from the image though or can you arm it with more data so it knows more about the cancer that it's got a zab? What we found is is that initial work just showed that it would try and take in the image and it would say okay well I know that you've got a lung cancer I'm going to predict what the treatment dose is going to look like effectively and it would use just image data. We realised though that we weren't getting good performance because it's not doing the same task as a human would. You see if an oncologist sits down to plan that treatment they know that it's a lung cancer, they know how big the cancer is, what type of cancer it is and where it's involved.

17:32So what we started to do is to combine the AI that's very good at looking at images with a kind of AI that's now very very commonplace in our society of being able to read text and we've built what's called a vision language model. This is two AIs joined together, one that's exceptionally good at extracting information from text like chat GPT if you will and the other one that's exceptionally good at extracting information from the images and lo and behold when we combine those two we're starting to see much much better performance than when you just give the AI an image and say okay we'll predict what a good treatment would look like for me.

18:08Mike Wooldridge:How do you make them talk to each other? Because one is talking a visual language, one is talking extraction of information from written English language. What do you do to bring the two together? How do you make them overlap so they do understand each other in that way? We have long thought about ways of encoding information into text that are just as readable by a human as they are by a computer. And there are various forms of language or technically a schema that are actually very easy for a human to read and a computer to read. Now it turns out that things like ChatGPT, they're extremely good at taking information from just plain text and putting it into a schema.

18:49So that's really what we've honest effectively. But it was basically because we'd already decided that it would be useful to have some common framework, something that was just as easy for a human to read as for a computer to read.

19:02Mike Wooldridge:how is this used would this inform a team meeting because the way we do kind of care like this is in the form of a team we get all the different practitioners who all have a stake in the care and we reach a consensus decision as to how we're going to treat that patient where does this fit in does it become a member of the team or does it land on the table and then everyone looks at it sort of says well yeah that looks okay how do you actually use the data and how do you know what to do with it? This is an example of using different types of AI tools or agents as we sometimes describe them to support human decision making particularly in the fast-paced clinical world and in particular to do reduction of information because we have so much information now about patients there's always the worry that the human might miss something effectively and what we have built in a very very small way and because it's small and simple it's easier to test and easier to demonstrate utility as an example of exactly that so can we build effectively an agent that reads the referral letter for somebody that needs radiotherapy and an agent that reads the scan and they can sit there in an mdt basically and interact with the other human experts and say would you like to see what i think a good radiotherapy plan might look like because that might be of interest to the surgeon in terms of thinking well would this patient be better off with an operation or with radiotherapy and it's hard for the surgeon to picture that if they can't picture for example how good the radiotherapy plan would be so this kind of technology really feeds into very much that shared decision making process but it's a very very simple example by keeping it really on this task of radiotherapy planning really interesting to see how far that has come Raj Jenner, who's Clinical Professor of AI in Radiation Oncology.

20:51Mike Wooldridge:We've just heard how different AI systems or agents can be yoked together in a specific situation like radiotherapy to provide complementary analyses of different elements of the data to produce a clearer picture overall. So can the same trick work more broadly across the entire scientific research arena? Discovery is after all driven by generating new hypotheses, doing experiments to test them and then analysing the resulting data. But as scientific disciplines become more complex and overlapping, deeper subject-specific expertise is needed, along with broader knowledge straddling those different disciplines.

21:29Mike Wooldridge:Now human jacks of all trades fitting this bill are hard to come by, but this is where an approach published just this week in the journal Nature by Google DeepMind, called CoScientist, comes in. It's a multi-agent system that uses a set of interlinked AI agents that each take command of different parts of the research process. The system can generate hypotheses, propose experiments, interpret the experimental results and then help the researcher to refine their hypotheses on the basis of the findings. Testing it out on the blood cancer acute myeloid leukaemia, co-scientists proposed a number of new drug candidates and therapies and it also highlighted some interesting avenues for creator Annalisa Polosky in her research on motor neurone disease, also known as ALS.

22:19Originally when I started working on co-scientists I was very interested in studying ALS which is a really complex disease. So you have essentially many different dominoes that are falling all at the right time and it causes a pretty aggressive symptomatic disease and we didn't really have any insights on how to solve it. And so the work with co-scientists allows us to span the literature, to make unexpected connections with other different diseases, maybe that have similar symptomatic patterns and ones that have different, to understand what could be the driving factors behind ALS.

22:53Mike Wooldridge:What does this actually do? In other words, if it were a car, you'd ask the question, what's under the hood? So what do you actually do here? Yes, I mean, I love the car example. So for instance, let's say I want to get from point A to be, right? I have symptoms of a disease and I want to understand the mechanism B, the things driving the disease. Let's say you live in London, right? There's so many different directions you could take to get there, different paths, and also so many different ways you could get there. You could bike, you could walk, you could be in a taxi, you could, you know, take a hovercraft or something, right?

23:23And what co-scientist tries to do is it makes those connections between A and B and gives you the reasoning behind those connections validated with web search and papers and literature that's coming out to try to give evidence behind those reasoning paths. So then you as a scientist can decide which path you should pursue. But the thing with co-scientists is you're not just going from A to B, you're not doing just the next step in your research, you're going from A to Z in the sense that we're trying to compress the time to make scientific breakthroughs. So you're not just solving the next step in the disease, but you're trying to get all the way to, let's say, potentially predicting a therapeutic.

23:59Mike Wooldridge:Science is very dogmatic. We tend to do things because that's the way it's always been done. We tend to rely on other people's findings and we use that as a starting point. Is this a way of pressing the reset button a bit and making us roll back and look at things maybe slightly differently or also spot shortcuts? Instead of going the long way around, you're saying I could actually cut the corner with this and show me a route that will save me a lot of time. Right. When you wake up in the morning, you put your pants on first, right? And And so what you can do with co-scientists is to explore a lot of different avenues and pathways without having to move immediately into the experimental validation, which is usually long and very expensive.