AI and the Future of Health with Joelle Barral

0:03I'm hoping AI contributes to bringing back the joy of practice in medicine. And finally, with all of the data that physicians and all their healthcare professionals have taken so much time to enter, we are going to be able to derive insights and help them.

0:23Welcome back to Google Deep Minds, the podcast. I'm Professor Hannah Fry. In this episode, I'm talking to Jarelle Baral, Senior Director of Research at Google Deep Mind about AI for health. Now, for years, we have joked about how the internet is the hypochondriac's best friend, capable of turning a small headache into a terminal illness and vice versa, with a quick search and a click of a button. But quietly, behind the scenes, the role of algorithms in medical care has been shifting. We've already talked a lot on this podcast about the impact of AI on drug discovery and research into proteins.

0:58But now, AI promises to change diagnosis and treatment too. Welcome to the podcast, Charles. I mean, there have been some pretty big changes already with AI, but should we expect that healthcare will look very different in 10 or 15 years' time to how it looks now? Absolutely. I believe so. I think healthcare is really posed to be drastically changed with AI, but it may not look too different. I picture healthcare as a fairly sticky ecosystem, you know. So 10, 15 years from now, we will likely still go to our primary care doctor and then follow through with specialist visits, etc. But each of them is going to be augmented, if you wish, by an AI agent.

1:43So underneath, the system will be very different, but it may not appear as different from what we know today. How long have you been working in AI and healthcare? Oh, wow. My whole career, I would say, I was still a student when I think it was CS229, right? The class on machine learning was ticking off. And I remember, you know, working with, I was further along in my PhD, but working with students on how to segment the larynx, right? And AI was, you know, this new tool that could really help us do that better. And I haven't stopped ever since. But why this area for you? I mean, what's the appeal of it for you?

2:18In healthcare, you're always challenged, right? There are lots of things that we don't do that well or you don't have infinite data. And so you're never perfect. So you're always trying to leverage the best tools to do what you're trying to do. And then I joined Google a decade ago to work on surgical robotics. And I remember vividly it was the beginning of us really realizing that with images, anything that a human could do in terms of interpreting those images, machines were going to be able to do. And so I sat down with my surgeon colleague and we took 100 cholecystectomies, which are, you know, when you remove the gallbladder, we manually segmented livers and gallbladders.

2:57And I fed, you know, a pretty simple neural net to see whether, you know, the algorithm would be able to decipher between those two things. And I remember being incredibly surprised, actually, when it did that perfectly, because most of the other tools we had at our disposal were never perfect, right? Like it did the job, but not always. Now, you know, livers and gallbladders really don't look the same. It's a very easy task. Like any student will be able to tell you which one is which. But still, right, like that really for me meant, okay, now we have something that will be infinitely more powerful than, you know, any of the algorithms I had been working on previously.

3:35And people forget, right? But when I was still a student, it was hard with computer vision to decipher a QR code, right? Like that was where we were back then. And so, again, AI has really opened up a world of capabilities far beyond anything we could envision before. Okay, well, let's start off with some of the examples of places where AI has already made a lot of headway. I'm thinking about diagnosis here and maybe medical imaging in particular. Talk to me a bit about what's been going on. Yeah, we've seen a lot of that over the last 10 years. That's what we typically call narrow AI. Narrow because it's solving one task, right?

4:11You mentioned medical imaging. So you can take a test x-ray or an MRI or we've done also like diabetic retinopathy work. Those are already tools that have been FDA cleared, for example, and deployed in the clinic. And they are augmenting radiologists in the sense that for those specific tasks, the machine does a pretty damn good job at annotating the image for the physician. Let me make sure I understand this then. Okay, so if we take diabetic retinopathy as an example. So this is people have diabetes and it's a cause of blindness. Is that right? Absolutely. It's the leading cause of preventable blindness worldwide.

4:47And it's pretty simple. So you take an image of the back of the eye with a fundus camera, and that gives you an image of the retina that then ophthalmologists will grade on a scale from one to five to indicate whether patients have no disease, moderate disease or severe disease. And what was striking when we started working on that particular problem was the fact that it was actually really hard to get to what we call ground truth, meaning, you know, as we're training our machine learning models to learn a task, we first need to provide them with a set of ground truth images, that means the pairs of the image and the label.

5:27And for that particular task, you know, we asked maybe one, two, three ophthalmologists, and often we got different answers. And I think one thing we did particularly well was to be very rigorous in trying to establish ground truth. And so we ended up working with 50, 5-0 ophthalmologists. And we realized that actually on some images, they really didn't agree with each other. Some would have given one, two, three, like each one of the scores could have potentially been the diagnosis a patient would have received because patients only see one, right? They don't go and have their images graded by 50 people.

6:02Yeah. Boy, so some ophthalmologists would say this is a mild or no disease and others would say all the way up to severe on the same image. Precisely, which sometimes is not too surprising, meaning that most ophthalmologists, and that's true for most physicians, see what's in their community. And the rare cases, by definition, they don't see them very often. So it can get very hard to properly annotate an image if you don't see that type very often. Well, how do you do that then? If even the experts don't agree, how do you decide what the correct answer is? That's exactly why we had to go to 50, right?

6:36Then you have this consensus among all of the largest and sometimes you go to even deeper experts, right, to really get to what is the correct label for that image. And that way you properly train your algorithms. And then, you know, we also did a lot of rigorous testing to make sure that we had algorithms in which we could trust. But then I suppose on the flip side of that, even though that makes the labeling quite difficult, if you know that experts are disagreeing, then actually you can create something that is a new gold standard, really, like better than experts. Absolutely. And I think that's where the AI has really its role to play, right?

7:10It's where for tasks that it is really good at that are really not very easy to be done by a single human. I seem to remember reading this paper where some researchers had all of these images at the back of the eye and were like, you know what, let's just see for fun if you can tell the sex of the patient based on the blood vessels at the back of the eye. And no ophthalmologist in the world, I think, could do this reliably. But the AI can. Yeah, absolutely. It was very interesting to then interrogate those images for other things than just diabetic retinopathy and see that the model performed better than flipping a coin, right?

7:50And for some things, I'm not sure you need a model to decipher the sex of your patient, but I do think for indications of potential cardiac disease, for example, that can be a useful tool because if you're going to do that screening for diabetic retinopathy anyway, but now you have a test that can also provide you with additional information, you could potentially screen for more diseases. So indeed, and it was, you know, kind of both interestingly, scientifically to realize that those images contained a lot more information than we initially thought, and with potentially also applications that could derive from that.

8:27What does it imply maybe that even though you've created an algorithm to do this narrow task, actually, there is potential for the AI to help improve the overall understanding of the human body? I would answer maybe in two ways. The first is, you know, in radiology, there's this thing called incidental findings, because it's very often that if you screen me head to toe with MRI right now, you'll find things, you know, you'll find nodules, things that are abnormal, they will get me worried. But actually, we never did clinical trials to see whether having that thing somewhere is good or bad, right?

9:01And many people leave with those things forever without any issues. So you don't really want to do that type of screening that will get everyone worried. The other thing I would say is when you do a blood test, you do a blood test for a very particular thing. The lab who did the blood test for finding a particular virus is not responsible for checking everything, right? And if they haven't noticed that you had something else that wasn't what, you know, you got blood drawn, they're not responsible for that. And so for imaging, it's always a little ambiguous what to do with those incidental findings, right?

9:34And for those aspects of narrow AI, I think the jury is still out whether or not we want the AI to always look for everything else when it's looking at one particular image, even if it could provide pretty helpful alarms. Those are images that we've spoken about, but are there other inputs that you can use for this very narrow AI type of diagnosis? Absolutely. I would say all types of modalities, right? We've done a lot of very interesting work with sound, for example, leveraging sound to be able to detect tuberculosis from cough. And people have also explored that quite a bit during COVID, obviously.

10:10And so we see good biomarkers actually in sound that can help also provide some relatively cheap ways to diagnose those types of diseases. So wait, it actually works then? Yes, it does. You record a cough and it can tell you if you've got TB? In shorts, yes, that's exactly what it is. Okay. But there are limitations to this stuff though, right? I mean, are they getting it right every single time? No, I think that's, you know, a very good point. Like with any algorithms, right, there is sensitivity and specificity, so it can have either false positives, false negatives, so thinking that something exists when it doesn't or missing something that it should have picked.

10:48In general, though, what we do and what we did with the diabetic retinopathy paper is really checking how it does with respect to the best physicians or the best panels of physicians. And then letting the human who is always working with the algorithm decide where on that specificity sensitivity tradeoff they want to operate. And so they can decide that really they cannot afford false alarms or the other way around, that they cannot afford missing anything. But in both cases, I think for an algorithm to be cleared by regulatory authorities, it needs to have shown very strong performance. OK, let me break that down then.

11:26So it could be that an algorithm could say that it's TB when it isn't, but it also could be that an algorithm could miss a genuine case of TB and say that it was fine. And obviously, you don't want either of those things to happen. But what percentage accuracy almost are you willing to accept before you say, OK, this model is adding value? So it really depends, right? If you're going to deploy your, let's say it's a screening test, in places that had nothing before, it's a very different story than if you're replacing an existing alternate way of screening that same population. If you're replacing something, you'd better be at least better.

12:03If you're coming in where there was before a void, then public health authorities are going to decide what they deem is acceptable. It also depends what will happen to the patients that are provided with that diagnosis, right? If they go home and you never see them again, it's a different story than if your tool is a pre-screening tool and then you're going to reroute your patients to an additional screening tool for confirmation. When you launch these things, for example, the TB model or diabetic retinopathy, does that mean that you aim initially to go to places where they don't have these existing screening tests in place?

12:40It really depends, right? For diabetic retinopathy, for example, it is something that we've indeed deployed in Thailand. And then we often work with partners to really bring it where it makes a difference. We've already screened 700 ,000 people in Thailand, and we're 10x-ing that over the next few years. Why Thailand? Thailand is one of those countries in which the patient population that each ophthalmologist has to care for is fairly large, right? There are many places in which there aren't really enough doctors. And so it's a good example of a place in which AI screening can really help improve outcomes.

13:15Let's say that you have a situation in which some sort of medical imaging is being used in conjunction with a doctor. What happens if the two disagree? I think at the end of the day, the doctor is really the one making the call, right? So it's very important that they retain control and they're the ones making the decision and they're the ones signing their name on the report. And so they would have to explain if they're saying the machine is wrong. It could be for a variety of reasons. In other cases, you could think of the algorithm making the physician rethink. So it's checking your work like a spell checker, if you wish.

13:55Sometimes I disagree with my spell checker with accents on words in French or things. like it's not perfect, but I will double check. Okay, well, let's talk about some of the more advanced stuff here, because up until now, I mean, the examples that we've been using here is like an MRI or a photo of the back of an eye or, you know, the sound of a cough. What about the more holistic view? So, I mean, really good medicine doesn't see a human as a collection of interesting or otherwise medical problems, right? Sort of sees a human as a human. Can you use AI to think more holistically? AI is indeed very good at looking together at things that up until now we were mostly capable of looking at, you know, individually.

14:40And maybe bring insights that up until now we didn't have. Like what? I mean, if you do manage to connect up everything we understand about the cell to everything we understand about an organ to eventually everything we understand about a human, what do you find in those connections? Yeah, back in the days, we did quite a bit of work in virtual staining, right? So when you have an HNE slide, so that's what happens when any piece of tissue removed from your body in surgery, typically, will be sliced and looked at under the microscope. And the way we do that is we make hypothesis as to, you know, what might be happening, and we stain accordingly so that it makes those things visible.

15:18And you can think of some of those techniques as making that visible, but without having to stain, right? So really revealing some of that information if it's already in the tissue, but without having to bring kind of additional things that are consuming that piece of tissue. Will destroy the tissue for future tests. Exactly, exactly. So to me, you know, there's also something quite profound in the way we're now interrogating tissue with more and more instruments with things like transcriptomics, genomics, single cell, etc. And so that's really a wonderful application for AI because the AI can look at each modality and, if you wish, provide us with eyes for that particular scientific instrument.

16:00And then it goes beyond because it's also capable of bridging between those different types of scientific instruments. So then you presumably can start to combine some of these quite complex AI tools, you know, on imaging, on genomics. Does that actually allow you to advance in your understanding of diseases? I mean, I'm thinking of cancer care here, for instance. Yeah, it's a great example. And that's what we are actually precisely working on with l 'Institut Curie, the Curie Institute in Paris, that is very advanced in exploiting all of those most recent modalities to better understand cancer, in that case, women's cancer, so uterine cancer or breast cancer, which, you know, despite our best efforts for many, many decades, we're still short of answers for many women.

16:45And we're really hoping that by combining those modalities, by getting to a deeper understanding at the cellular level of what's going on, we'll finally be able to crack, you know, for example, triple negative breast cancers or some of the, again, more advanced cancers that today we don't have good solutions for. As well as cells and genomics and imaging, there are other data sources that the AI can bring into the equation here as well, right? Absolutely. And I think that's also maybe one of the secrets AI will be able to decipher, which is health is in everything we do, right? It's how many steps you walked today, what you ate for breakfast, lunch and dinner, whether you've interacted with friends or been lonely all day long.

17:31And so it's typically really, really hard to translate that into either healthy habits or understand how much of that is factoring into something physical that is going on. And we're doing a lot of work with sensors. I'm wearing this Fitbit today. You know, it's just one of the data sources that can really be leveraged in combination with everything else to try to both better understand health, sometimes just as an individual, and also when we're trying to change behavior, right, accompany us on that journey. I mean, you're talking about quite different quality of data here, right? You've got sort of slides from biopsies and genomic data and then step count.

18:12It sort of feels like a quite fuzzy thing around the edges. Would it actually make a difference? It does. I mean, it's been shown, you know, over and over again, for example, how much sleep matters, right, for both cardiovascular health and in oncology, right, in terms of preventing cancer. So it's one of my hopes, right, that AI will bring those two together in a way that is harder to do in our traditional healthcare systems and also very hard to do for us as individuals because you see that type of impact at a population level. as an individual it's really hard to convince yourself that you should go to bed one hour earlier because that's actually the best thing you can do for your health.

18:50There's quite a lot of buzz at the moment about digital twins. Does this also add to the whole holistic view of health? Is there an idea of making a kind of digital twin of yourself for healthcare purposes? Yeah we see that a lot and it can mean different things for different people right. There is the idea that The digital twin is kind of a simulation of someone with a similar persona, for example, and then that can be a good proxy to interrogate the potential impact of different types of interventions. And then digital twin can mean something quite different for the pharma industry, for example, where there's a lot of work trying to see how far we can go with clinical trials.

19:29if we manage to assemble virtual cohorts that are allowing us to gain as much knowledge about the safety and efficacy of a drug, but without needing as many people for that particular clinical trial. Oh, that's so interesting. So is this about, I don't know, making a kind of cohort of like simulated humans almost, maybe not the whole thing, maybe just an organ or whatever it is you're particularly focusing on, but so that you don't necessarily have to use as many people in your clinical trial. Exactly. For clinical trials, that would be exactly that. But then to be able to do that effectively, you have to really, really understand what real humans look like, which means having a wealth of data from individual patients somewhere along the way.

20:14How do people feel about contributing their own data for this kind of end, for the research of medical purposes? I think, you know, when we talk about privacy, we see different concerns in different parts of the world. In Europe, there are a lot of sovereignty concerns. People want to know that the data is, I mean, not in all countries, but is staying on their soil. And we have solutions for that, right? All of the trusted public cloud solutions, for example, are complying with the highest levels of regulation on that topic. In other parts of the world, we see a lot of appetite for people to not think that they are sick for nothing.

20:50By that I mean it's actually quite compelling to know that if you're sick but you're contributing data to research, you're helping making the last person that is going to have the same fate a little less sick, if you wish. So I think really enabling that virtuous cycle is really where we have to be. Because it does feel like healthcare data is a particularly sensitive case, because on the one hand, exactly as you described, there is huge potential to advance our understanding and improve conditions for future generations. But on the other hand, if healthcare data gets into the wrong hands or is misused or used irresponsibly, I think people have real concerns about the potential ramifications of that.

21:34So, I mean, how do you strike that balance? Once you have the technology to actually protect the data, there's also accountability, right? That meaning that if, you know, we're saying that this research will be helpful to the people, that indeed, at the end of the day, it is research that is helpful to the people where the data originated, right? And I think that's a very important principle that, you know, in the same way that you shouldn't do clinical trials in places of the world, in some places of the world, and then the drugs benefit in other places of the world. So you're not just extracting from one to give to another.

22:04Precisely. Okay, I also want to ask you a bit more about the patient's experience in all of this. Are large language models changing the game here already? I mean, are you concerned that people are trying to diagnose themselves with generative AI? Yeah, I mean, I would say even before, you really shouldn't do that, right? Like, you shouldn't play your own doctor. People do, though, don't they? I think people want to know, and especially in places where waiting time to see a physician is becoming a real issue, then, of course, you know, patients are trying to learn as much as possible and to, you know, decipher what's happening.

22:39And sometimes quite effectively when they are there because they are their best advocate, right, or parent trying to figure something out for their child. And sometimes, you know, we have a couple of examples where for rare diseases, actually, we've seen patients do a remarkable job. Now, I would say, you know, at Google, we have always tried to provide the most helpful answers to our users. And so with symptoms, for example, we have the knowledge cards that you've probably noticed that are telling you about a disease and, you know, with authoritative content, right? Like trying to be clear in the explanation of common symptoms and potential options for treatment, but never crossing that line and telling you what your own diagnosis might be.

23:24Now, with large-longed models, our model Gemini will also not tell you what your diagnosis will be, right? It will tell you, sorry, I'm not a medical doctor. You should go and see a doctor. But you can get more helpful answers if you're asking, you know, what potential conditions could explain this, right? And it will give you more of the textbook answer. I like to think of it as, you know, those family guide for healthcare, those, you know, I don't know, the big, thick books that you might have where you have small children and you're trying to know what to do in those particular cases. Well, it's that same level.

24:02And if we believe that large language models are going to be able to provide diagnosis, we need to continue the research. And that's what we're doing with our work with Amy, the Articulate Medical Intelligence Explorer. However, it's a research project in which we are trying to see which conversational abilities a large language model can have to establish a diagnosis. So can it ask the right question, like the way a physician would do, right? Like dialoguing with a patient. How close are we to seeing a system like Amy being used actually in medical settings? So we've been working on this project for quite a while already.

24:41And, you know, our first research papers were all based on research done with patient actors and simulated scenarios. We're now doing a clinical study under RB approval with Harvard at the Beth Israel Medical Center to see what happens, you know, again, in a very controlled environment with physician supervision with the system. So that's our next step. Hard for me to predict, you know, how long it is before such a system would really help physicians in the clinic. But we are already seeing, you know, in parts of the world, similar systems that are typically being used to answer low-risk questions, if you wish, and that are always supervised.