Can we trust AI in health?

0:00Welcome to Tech Life on the BBC World Service with me Shona McCallum. Today the benefits of using AI in healthcare. AI stethoscopes can mitigate for the absence of that type of technology in low and middle income countries. Using them really just needs you to have a smartphone and a degree of connectivity. And what to be wary about? Large language models are being hailed as the future of clinical decision support. But they're remarkably vulnerable to information that has nothing to do with health at all. Plus the ex-YouTuber Stampy, best known for his Minecraft videos on designing games based around online communities.

0:41And I speak to two students from India whose award-winning energy project harnesses the power of the oceans.

1:05Now, if you were listening to last week's Tech Life, you would have heard Paula from the UK, who got in touch with us to say she was monitoring her heart health using a heart rate monitor and AI. She also had a couple of questions for us, so we decided to put them to an expert. More from Paula a little later, but first, here's Dr. Patrick Bachtiger, who's based at the Health Impact Lab at Imperial College London. He works as an NHS doctor, but also researches AI. Our team at the Health Impact Lab are really interested in helping more innovations, health technologies, AI technologies, get from the point of having lots of evidence, getting them translated into the health system so that patients and doctors can actually reap the benefits of these innovations.

1:50We're seeing a lot of hype and discussion of AI technologies for healthcare, but in reality, very few of those have got to the point of large-scale system-level implementation. Is there a feeling amongst, well, this is the way we've done it, and perhaps not being completely sure about how tech can really have an impact on the day to day, especially when you're dealing with people's lives? Yeah, of course. I think one of the challenges in healthcare is that very rarely are we able to justify spending a lot up front. And a lot of technology opportunities, technology innovations do require quite substantial upfront investment.

2:26However, we have seen globally that there are public health priorities that really do need to shift to more preventative focus. And one of the things that you worked on was the AI stethoscope. Can you tell me about that? Yeah, so what we did is we took these AI stethoscopes that can record both the electrical signal of the heart as well as the acoustics. And those signals can be interpreted by AI to give you a really accurate indication of the presence or absence of three heart conditions that are very common. That's atrial fibrillation, heart failure and valve disease. And picking these diseases up early can get patients onto the right pathway, onto the right treatment and can let them live more healthily for longer.

3:10And do you think that solutions such as the AI stethoscope could be implemented evenly across countries throughout the world? Or do you think that some parts of the world are further behind the UK when it comes to embracing and trying these types of tech? I'd suggest it may actually be the opposite, where some of the most innovative healthcare systems are in lower and middle income countries, where AI stethoscopes actually have some of the biggest opportunities for impact. So one of the gold standard diagnostic tests for some of these heart conditions is an echocardiogram. So that's a big ultrasound machine requiring expertise and training and skill.

3:51AI stethoscopes can mitigate for the absence of that type of technology in low and middle income countries. Using them really just needs you to have a smartphone and a degree of connectivity and the world is becoming ever more connected. So the opportunities to pick up heart disease early, I think, is universal, but particularly compelling, actually, in low resource settings. Now, Patrick, last week we heard from one of our listeners, Paula. She has a heart condition and she also uses some health tech to help monitor that. She had a couple of questions that she wanted us to put to an expert. So I wonder if you don't mind listening to what Paula had to say and see if you can give your best take.

4:29Is that OK? Of course. My pleasure. When using different artificial intelligent apps, you get different answers to the exact same questions, even if you're using the same AI but on a different day. So how reliable is the health information you get from AI? Paula asks a really, really important question. So the issue here is that these are not medical devices. You don't have regulatory approval to give you clinical level advice and guidance and counseling on medical problems. Some of the later versions of these AI apps actually link you back to the reference and to the source material from which the information and answer was drawn.

5:08My suggestion to Paula would be to always go to that trusted resource to get your answer. So, for example, in the UK, the NHS has a variety of NHS approved data sources to give you inflammation on heart conditions. OK, great answer for Paula. There's another question coming up. I bought several different wearable heart monitors. Does the cost of wearable monitors have any comparison to their reliability? I mean, a really good question from Paula again. So we need to really split whether or not the heart monitoring technologies people are buying are intended for wellness, which don't have any regulatory approval.

5:43And they just assist you in giving you some information about wellness related heart metrics or if they are specific health technologies, where if they are health technologies, they'll have had to go through regulatory approval, evidence generation and actually have a very high bar. so the cost it doesn't necessarily correlate to the quality but by any measure certainly if they're wellness tools uh they're just information to help you understand how high your heart may or may not be going during a run or during a walk but for specific clinical indications such as measuring your ecg your electrocardiogram as long as there is regulatory approval in place the difference in cost is fairly uninstructive in terms of how good that technology actually is you just may find that it's a slightly nicer device.

6:29And I think manufacturers of these technologies all really are quite cautious about what claims they make around what their technology can do, and generally will advise that indeed you should seek the advice of a medical professional if certain things are flagged within specific bits of their technology. Thanks to both Paula and Dr Patrick Bachtiger from Imperial College London there, it got us thinking about the vast amounts of research that's going on in AI and healthcare. So here's another leading researcher, this time from the US. Hi, I am Marzia Ghassimi. I am a professor at MIT and hello to TechLife.

7:08And where do you stand then on using AI in healthcare? I stand at the strong regulatory body precipice, I think, which is AI is a really fantastic transformational technology. However, as with all technology, with electricity, with aviation, with automobiles, we had to go through, in some cases, decades of strong regulatory action before use of that technology benefited everyone and was safe for a majority of users. We have not gone through that more rigorous, robust process with AI and healthcare. We are on that journey. And I've been looking at your research, some of the mistakes that can come up.

7:54One little typo, for example, can mean that medical AI concludes a completely wrong thing. I think large language models are being hailed as the future of clinical decision support, but they're remarkably vulnerable to information that has nothing to do with health at all. It's concerning that when you look at these patient questions or patient vignettes, small non-clinical changes like typos or shifts from confident to more uncertain phrasing can tip the scale of a model's recommendation. If a patient note is written in this more polished language and that generates the model to say you get more resources, but the same note, same clinical content that has minor errors or has a less formal style gives you the opposite advice.

8:45Stay home, self-manage, treat yourself. This is very concerning. You were part of a team which showed AI could detect people's race by just looking at an X-ray, something that a physician would never be able to do. It's true. It is sort of amazing. There are a range of human demographics that are obviously perceptible. And by obviously, I mean the prediction, the correct prediction rate for AI is so high. But they're impossible for humans to, even human experts who have been studying medical images for decades, they cannot do the same thing. It's also something that we need to be concerned about because if a model can extract something perfectly from images and that model also as a secondary step has some biases against people of specific demographics, whatever they are, right?

9:46They might incorrectly predict outcomes, medical outcomes for just those kinds of people who might be in the minority. So it wouldn't happen that frequently. And it would be very difficult for a doctor or a hospital system to even realize that that was happening. Speaking about race bias and gender bias, obviously yourself being a woman and being a Muslim woman in this space, have you been more interested in that aspect of AI and how it works? Initially, I wasn't. It was really the technology that intrigued me. What happened was at the end of my PhD, a member of my thesis committee who is a practicing doctor had encouraged me to do an audit of model performance along subgroup lines.

10:36And when I did, I found that there were differences between how models worked in different subgroups. That really, I think, changed my perspective because I think prior to that, I had had many negative interactions with many systems of power, right? The health care system, educational system. And if you happen to hold identities that are not in the majority in any context, you need to be very aware of what your rights are, of what potential human biases are. You need to be able to advocate for yourself. But I think until that point, I really hadn't thought about how deeply ingrained such information might be in casual data.

11:19The shift in machine learning from smaller targeted data sets that have intent to scraping everything that's out there really has also shifted what we need to be aware of and concerned about. Well, your research has been very extensive, but I just wonder what's next for you? Now we're looking into what might happen if a malicious actor, somebody with bad intentions, maybe on a development team, tried to create their own biases in a medical model and insert them in something we call a backdoor. Both how easy is that to do and how we can protect against it and identify such things in a medical setting.

12:00That's Professor Marzia Gassimi from MIT.

12:08You're listening to Tech Life on the BBC World Service with me, Shona McCallum. We've already heard from one listener, Paula, on the programme, but I want to say hello to John, another listener from the UK. John WhatsAppped us to tell us about a little invention he's come up with. He came across a discarded e-cigarette or vape with a battery inside. After experimenting with a circuit board, some electrical components and a soldering iron, he's turned it into a small portable mobile phone charger. And he thinks it might be of use to people who live in parts of the world where there's no mains electricity.

12:44Brilliant. Keep up the inventing, John. Remember, we always want to hear your mini-adventures in tech. Or maybe like John, you've come up with a techie invention. whatever it is we want to hear about it email us techlifeatbbc.co.uk is our email address or send us a voice note on WhatsApp our number is plus 44 330 1230 320 Still to come a little U-shaped pipe which could help generate renewable energy but first what a game designer learnt from over a decade on YouTube

13:27In the overworld Wow Anything you can dream about You can create So they built all this? Yeah, most of it Some of you might recognise that from one of the biggest films of the year so far a Minecraft movie It's of course based on the hugely popular game played by roughly 200 million players around the world. This place makes no sense. Don't I know it. Minecraft is a sandbox game where users can create and build their own tools and items to play with, as well as interact with others online. It's got a huge and very dedicated community of players and also a whole ecosystem of content creators, lots of who live stream on platforms like Twitch and YouTube.

14:17Joseph Garrett, better known as Stampy on YouTube, used to post Minecraft videos to his over 10 million subscribers. But now he's gone from creating videos about gaming to creating games himself as part of the developer 4G Studios. But I started by asking him about his Minecraft days and whether he'd seen the film. I'm going to be ashamed. I haven't actually seen it yet. So what are the promises? I wouldn't be able to go to the cinema because I often get spotted there anyway. So if I went to a Minecraft film, I think I'd have to be in some sort of disguise. So I will get round to watching it on streaming.

14:54Absolutely. I'm looking forward to it. You've obviously stopped posting on YouTube now. And it is sad to be saying goodbye to my lovely world, but I think it's the right moment to do it. Our time here has helped make us who we are today. And who knows what's going to happen next in our lives. For our audience that don't know, why did you make that decision? and any regrets? No so I'm not posting at the moment but I still have the channel so I could always go back to it so I don't have any regrets because it's always still there waiting for me if I want to return and the reason I stopped is just because I did it for a very very long time and I loved doing it but I felt like I had explored everything I wanted to explore there so yeah I was looking for something new.

15:37What did you learn about balance and burnout? Obviously daily uploads are quite intense. Yeah, definitely. Especially in the early years, kind of 2013, 14, 2015, where I was doing at least one video every day. It kind of is very all encompassing. You have to make a lot of other sacrifices in terms of lifestyle and your social calendar. And it was too much. I managed to get a better work life balance later on. But especially in those early days when I was enthusiastic and I wanted to be doing it all the time. Yeah, I kind of maybe neglected other part of my health, I suppose. But I have a much better balance now.

16:16And what was it like for you personally going from playing games? Obviously, in Minecraft, you are creating the worlds as well, but you're now designing them at 4G Studios. I mean, did you see that as a prospect? It's very different when you're making something from scratch and it has been a steep learning curve. But yeah, I jumped on the opportunity when it arrived. Now you gave a talk to a big games industry conference. What did you want them to know? So the talk is about involving the community with game development. So the game 4J Studios is making at the moment, Reforge, we made the conscious decision to involve the community very directly from a very early stage, from literally the day after it was announced, we had people in the studio playing it.

17:03And so the talk is really about what we've learned from doing that and how it's going so far a year and a half into development. And how is it going? It's going great. We've just released the biggest update we've done yet, title update four, which is the first time the game transitioned from a proof of concept creative building game to now there's a proper survival loop and a progression and it's starting to really shape into the game it's ultimately going to end up being. So we've had a really positive reception. How has the community aspect of gaming changed? The Discord is really the hub for the community for Reforge.

17:39We're asking questions, we're getting feedback every single week. I'm sitting down with the leads of the company and the director and we're going through all of the feedback that we've received on Discord and it's every week making immediate changes to the game. There's even mobs in the game, creatures that have been suggested from the community that have made their way in. Really, the goal of the project was to bring the community in as early as possible to ensure that we're going in a direction that the community enjoys. And so far, it really seems like that's been working. And you think that's the way forward then for the future?

18:13The type of game that Reforge is, which is a survival sandbox game that we're going to be continuing to develop for years and years and years. It's very suited for this development style, you know, a shorter, more narrative based game, you know, it might be a little bit more difficult, but certainly for this style and genre of game, I think the sooner you can bring the community in to start hearing what people think rather than hoping they're going to like it, you can find out for sure. You can just ask them. And so I really think it's working. And is it funny having colleagues? Yeah, it's great.

18:47That's actually one of the biggest thing is on YouTube, I basically did everything myself. You know, I'd sometimes speak to friends, but in terms of actually doing it, it was kind of, I guess, a little bit lonely or a little bit of a solitary experience, which is rewarding in its own way. But it's really nice now having colleagues that I can bounce ideas off. And it's crazy to think you had over 10 million subscribers, but yet you were lonely. I wouldn't say I was lonely, but I think that that creative process was, you know, by definition of me doing it by myself, it wasn't collaborative. So the preparation, the designing, the planning, the script writing, and then the editing, that was all kind of done by myself.

19:30Thanks to Joseph Garrett.

19:35As many of you will know, gaming can be an expensive hobby. The latest Mario Kart World game on the Nintendo Switch 2 costs 80 US dollars. And it's widely expected that the next Grand Theft auto instalment will top$100. So are games becoming unaffordable? Or has the rise of free-to-play games led us to expect more for our money? Will Chalk reports. He's a tiny cartoon Italian plumber known for saying things like this. It's me, Mario. But some fans haven't been thrilled with the price tag on his latest outing, Mario Kart World. I met Chris, a gamer, outside a gaming shop in central London. That's insane.

20:20When I was about, what, 14, 15, games used to be£40. Now you have to, like, really think about buying games. Chris isn't alone. Look online and you'll find legions of gamers complaining they're now having to fork out a day's wages to buy the latest titles. So do they have a point? Well, maybe not. Sam Nagy is a video games analyst who's been working in the industry for more than 15 years. When you account for inflation, new games were valued between$90 to$100, always. In relative terms, they're actually on par to what they were before. So why might games feel more expensive, even if they're not?

20:58Well, one argument is we've been spoiled. Fortnite fans from around the world, Huge games like Fortnite are completely free to download and play, making their money instead from microtransactions in the game. and this impacts public perception of the whole industry. Here's Sam Nargi again. Full-priced games are now competing against those games which are essentially free. There is an argument, though, that even if game prices have remained relatively stable in real terms, they should, by rights, have gotten cheaper. Patrick Moran is the curator of an international touring exhibition about the history of video gaming called Game On.

21:34So cartridges, because they contained microchips and particularly memory chips, were really expensive. And then that really changed in the mid-90s when you moved to CDs, when you suddenly had very, very cheap media. But the reason games didn't get cheaper then is apparently the same reason they don't get cheaper now, even though they can be downloaded directly, cutting packaging and shipping costs completely.

22:00Jane Campbell is from the team behind puzzle game Monument Valley. So every time we're making a new game, we want to make something that is bigger and newer, and that increase in scope looks at bigger team sizes. And Jane says the cost of making a game has increased exponentially. Historically, game companies tend to undervalue their games and actually sell them at lower price points than perhaps they should. So it's very, very hard and it's very complicated to make money as a game company. To put this in perspective, if you buy a cinema ticket for$10 to$15, you'll be entertained for a couple of hours.

22:37A good game can keep you hooked for 20 or even 200 hours. And that's got to be worth something. That was Will Chalk reporting.

22:53Ah, the sound of ocean waves hitting the beach. One of my favourite noises instantly transporting me to being on holiday. But there's a more serious side The potential for power Rarely interrupted and almost always in motion Waves offer a continuous source of energy With some estimates even suggesting oceans have the capacity to power the entire world renewably Plus there's the issue of communities really suffering from coastal erosion When the sea wears away at the land Two seemingly different issues but could there be a tech solution for both? I'm going to be talking to two young engineers from India who have just won the UK Dyson Award for exactly that.

23:44I'm Pratikan and I'm currently working in an institution in South India. So hello, this is Janathan Vengdachalam here and you can call me Jana. I'm actually an undergraduate in mechanical engineering. Energy is very close to my heart to support this world transition from fossil to renewable. Their invention is called Kino wave. So coastal erosion is happening because of the ocean's power. What if we could harvest that ocean power? But at the same time, by harvesting can reduce the ocean erosion. The device is a bright yellow pipe in a U shape filled with fluids. When it rises and moves down in the waves, the fluid in the cylinder tries to maintain its level and surges over to the opposite side of the U.

24:29These relative motions can be further processed to generate electricity. Since we are mimicking the ocean within our device, there is no need of mooring or permanent structure in the ocean. So we can extract electricity directly from the balancing force of the water. The Kino wave generates energy from the relative motion of waves and protects coastlines by absorbing the force of the waves. The team have been testing their solution in wave tanks, but they plan to scale up the device and deploy it in oceans to prove its reliability. We are trying to move to simpler islands like Satchel or something.

25:08We are planning to create a pilot product there with the support of government of Satchel. By validating in Satchel, we can move to any country and scale down, scale up, and any number of converters can be inserted. We are initially looking for a smaller region with smaller landmass and have highly prone to climate change. Whenever the sea level rises drastically, there are high possibilities that coastal erosion can be harsh. So we are initially looking for smaller regions, smaller government who actually need this at the first place. So Seychelles, you think, might be one? Yeah, Seychelles and islands of Indian countries.