#411 Dr Ross Harper from Limbic: Is It Unethical Not to Use AI in Mental Health Care? (Part 1)

0:00James Somauroo:Welcome to the Health Tech Podcast. Here we talk about everything healthcare and technology and I'm your host James Somauroo. Hey everyone, delighted to be joined by Ross Harper, CEO founder of Limbic. Second time you've been on the podcast Ross. the first time uh we were just looking weren't we 2019 2019 how many years i don't want to say out loud how many years ago that was uh the but i tell you what still here is a health tech company what was it six years later that's pretty good going man how have the last six years been they've been good they've been good it's a different time 2019 pre-pandemic obviously yeah we went through a pandemic um since we last spoke so hope you're right you know hope that was good yeah but uh yeah it's uh when when i spoke to you last we would have been gearing up to launch and now we have been in market for five years awesome and obviously i will have got your story from last time but people didn't want to go back to episode i think 72 75 something along those lines if they didn't want to go back and listen to it what's the uh what's the short version we build ai for mental health care we are a clinical ai rather than a wellness ai and we deploy within clinical settings to amplify the supply of trained mental health professionals so that we can actually meet the astronomical number of patients who actually require mental health care So launched here in the UK in 2020, a good year to launch a tech solution in the NHS.

1:53And now we are used by 45 % of the UK's NHS talking therapies. Wow. Yeah, really pleased with that. We are approaching 500 ,000 unique patients who have used the AI as part of routine. and it does everything from intake, triage, diagnostic support and then actually supporting on cognitive behavioral therapy.

2:20James Somauroo:Amazing so you said some some killer words there diagnostic and treatment those are well what would have been incredibly bold things to say until I guess quite recently when we've started to now get AI regulated and what a time to be in healthcare. Now for people that have been listening to this podcast for a little while they will know that I've been on a bit of a journey with my I guess my own understanding of AI and I've tried to be as sort of intellectually honest and emotionally honest as possible of sort of how I felt as we've kind of gone through the fact that AI exists in its machine learning capacity then as large language models and then large language models now being applied to healthcare in lots of different ways the frontline nature of large language I'm gonna say large language models for now and feel free to correct me of exactly what your technology is doing but large language models being applied for diagnosis and particularly mental health treatment has been a scary thing for me to battle with and I think that's because of what was initially quite a binary nature to the conversation of you have a chatbot that's delivering mental health therapy and now that's what's going to happen and in my mind I'm like oh god there's now this like slippery slope to like we're no longer ever going to have a human it's just going to be AI because it's just going to be so economically favorable like all that kind of stuff and I've sort of come around to the idea of it being a lot more nuanced and and one of the most important arguments to to use your words the astronomically high number of people that require mental health therapy it cannot be ignored and I think the I did a keynote in Australia about this and it was and I was sort of explaining that the conversation in my own mind went from a place of can you live with yourself if ai is delivering therapy to can you live with yourself if it's not and i think that it's it's i just want to get your take on this because you're you're doing this it's doing intake it's doing it's doing the you know splitting this into clinical and non-clinical you're doing some non-clinical stuff with it and you're doing the clinical stuff with it and half a million patients it's incredible work man so congratulations first of all um but talk me through kind of yeah that constellation of stuff and where you sit on all of that stuff because clearly you're pioneering this man yeah appreciate that well look there's a lot to unpack there so maybe i'll just start talking loose dream of consciousness and you can nudge let's do it into the direction and i appreciate your jet lagged as well i can see you in your co's la ventures uh quarter zip i know you've been out in the u.s I don't have any clean clothes.

5:21Do some laundry. I got back.

5:23James Somauroo:It is a sad state when you get to the merch part of your laundry. Like, oh, God. I was on the West Coast of the US for six weeks. I just got back a couple of days ago. So I was saying to you before this began, apologies if I'm a little bit foggy, but I will do my best. So clinical AI. I mean, first, let's just get some definitions in place. If it is used to support patients in treatment in a clinical healthcare context, and it is primarily focused on a clinical function and driving patient outcomes, it is a clinical tool. It is clinical AI, and it therefore must meet a very high bar for quality evidence and patient safety.

6:16That bar typically means a long list of peer reviewed papers that demonstrate efficacy, just like any other clinical solution. You know, if you think about drug development, you can't just throw something into the world and say, good luck. you have to have an evidence base you also likely need to have uh approvals and accreditations on information governance data privacy um for us limbic we are a medical device you know we are regulated as a medical device we are the only mental health chat bot anywhere in the world to have secured class two medical device approval which means we are really able to take a degree of responsibility for patient outcomes as part of care.

7:11So that is a world apart from just a wellness chatbot, right? So different from we made an app, we called it a coach or a therapist, and there's not really very much policing on what you can and can't say. It's a bit the Wild West in digital health. And so you can release something in the app store. And there are 1000s of them, which are these wellness solutions designed to make people feel a bit better in a consumer context. And it's not clinical, there is no evidence base, there is no regulatory defensibility, there is no integration into the clinical workflow. that's what makes it clinical ai um and you can't use limbic if you're just a general consumer

7:58James Somauroo:you use it if you are a patient have you struggled to separate yourself from the wellness chat bots to people that matter i mean mainly i'm thinking that people like investors and things like that like what's been the what what's been the overall kind of education of people yeah we had to do it more and more like most things it's getting easier and easier with time but um most investors didn't get it a few years ago they were unable to see past a consumer wellness solution and a clinical grade ai um and that's just the nature of things very few investors are actually um courageous and effective at predicting the future.

8:45The ones that are, are hyper-valuable. And they really are, I think, contributing to building the future. But the vast majority of investors are risk-averse and they want to follow. They want it to be as proven as possible. And that means their thinking lags behind the frontier. But you're right. both for investors and potential clients and partners we really had to do an education piece around why this is very different and really it comes down to acknowledging that there is something very special about the way mental health care is delivered in a clinical setting it is so powerful and it helps so many people but the biggest problem with it is there are not enough trained mental health professionals alive on the planet.

9:43And then we get back to your point, you know, about whether we should or shouldn't do this. With hundreds of millions of people with a diagnosable mental health condition currently going without care, I think it is an ethical and moral imperative that we figure out a way to simply and compliantly integrate AI into the care pathway because it is only through AI and automation that you will scale limited clinicians to meet the needs of the many. And if you don't do this and we aren't brave and we don't challenge ourselves to change the status quo, nothing will change and hundreds of millions of people will continue to go without support.

10:31And that's not a world I want to live in.

10:33James Somauroo:In terms of how your technology works then specifically I guess where does it fit into the clinical pathway you've mentioned it can span quite a lot of it but I guess specifically where is each part fitting in and what is the technology behind it so let's get technical on that element and start to split some hairs of exactly what's going on here the sort of patient experience would be without us without limbic yeah you want to access a therapist here in the uk you would typically call a phone number or go onto a website and try and get access to one of the therapy clinics within the nhs they're called talking therapy more often than not if you phone the call will ring out or you'll go to voicemail.

11:29If you're lucky, you'll get hold of somebody straight away and then they'll take some basic information and they'll get back to you with an assessment session with a clinician. You can typically wait many months for that session. And then when you have it, a mental health professional will talk you through, kind of do an interview. It's like a therapeutic interview and they will try to figure out what your primary presenting issue is so that he can get you onto the right treatment pathway and then treatment can begin and then you have a series of therapy sessions maybe once a week maybe once every two weeks and uh hopefully you get to a point where you recover that's kind of the flow there is a lot of waiting and there is a lot of um just due to supply constraints uh there is a lot of people who will either be on a waiting list indefinitely or um don't actually have anybody pick up the call when they first call in when you say indefinitely are we really talking indefinitely it can span anywhere from two months to over a year in the uk and if you're struggling with uh depression which typically is uh comes with low motivation your help seeking behavior is transient and very often the waiting list and then come back and so they just kind of get stuck in this cycle also uh sometimes very commonly uh you will see symptoms deteriorate while you're waiting and people get into a crisis, they may self-harm and you'll find them in the emergency department rather than where it should have been.

13:06So at Limbit, we said that's not good enough. There are ways to deploy AI safely to solve for this. So the first experience that a patient might have is when they reach out for support, they're greeted by RAI either over the phone it can speak now or on a text-based chat bot on a website right and then that does intake and triage all in one go so it's always a statistical reasoning to identify the likely issue and then book that patient in directly onto their treatment pathway completely um you know saving saving a bunch of time both administrative and clinical in those first steps and just a quick question on that ross um

14:01James Somauroo:do you does that ai tell the person it's an ai oh yeah it does yeah okay interesting yeah you have to if you're misleading people that's unethical and so that yeah you have to tell them what's more interesting isn't you know should we tell them of course it would be unethical not to what's more interesting is if you use the latest in language modeling the fact that you tell them you're an ai does not change their experience very much so we ran some studies let me tell you this this is interesting yeah we ran the largest study of its kind 130 000 patients in the sample size and these are patients not users these are people who were in treatment okay so this was a clinical study 130 000 patients across multiple sites in the uk and we found that patients are very aware they are talking to an ai but they still rate the AI as incredibly empathetic, engaging and this was the best finding, for almost all minority demographics you know, ethnic sexual, gender all minority demographics saw a statistically significant uplift in self-referrals into care when there was an AI from Limbic at the front door rather than it was just human-led acquisition channels does that make sense it does the fact that there was an ai at the front door lowered the friction to seeking support and mysteriously changed things across demographics but particularly for minority groups and james i said look we've got to find out what's going on here it's not enough to just empirically have the finding do you know i was literally about to say what does that tell us about the humans more than what does it tell us about the ai like that's that's a frightening statistic to come across, right, without acting on it.

16:08I think it's a marriage of human and AI. I am not an advocate for substituting humans with AI because I think there are strengths to each. But in this particular case, it really shone a light on the power of AI to drive healthcare equity. And what we found when we spoke to these different demographic groups and we conducted interviews was that limbic was seen as empathetic and engaging even though they knew it was an ai but because it's an ai it doesn't have its own demographic identity and therefore it could not clash with the identity of the individual seeking support so typically what you will find is um we have a heavy skew in in the clinician panel to white white um ethnic um identity and uh if you are someone from a different demographic background it is already known that there is a friction to opening up and engaging and feeling represented by the individual you are speaking to.

17:26And so that friction point translates into challenges around accessing support. By removing the human at the very first step and making it a non-judgmental empathetic AI, you actually saw a statistically significant increase in referrals because you removed the friction point. I just thought that was such a powerful finding because so much of media attention goes towards the risks of AI in furthering personal inequality. And that is absolutely a risk that must have attention paid to it. But what people don't talk about is that done correctly, done responsibly, AI can actually level the playing field and really solve for problems that currently cannot be solved for with just human resource.

18:23I love that I'm learning so

18:24James Somauroo:much here, mate. That's absolutely incredible. So one word I just want to highlight here is the word friction because what i don't want people to go go away with is is the i guess the notion that you're suggesting any kind of organizational or structural racism or anything like that although the connotation is perhaps there it's i guess with it's difficult to explain but it's a statistical friction, I guess. It isn't. Is it? Oh, that's a question. Let me clarify. So obviously, as you pointed out, I'm not commenting on any sort of institutional racism. That is not the point. The point is that it is already known in the data that individuals from minority backgrounds are not seeking support for mental health.

19:25um as much as they should be as much as the prevalence of the disorders in the population

19:31James Somauroo:so we need which i assume is due to a a lot of different things language barrier and fear and exactly potentially different degrees of social stigma but in different communities you know there's a picture there but the empirical data is saying that we are not seeing as many individuals from minority backgrounds seeking support. They are underrepresented. And one of the known issues is that there is a demographic mismatch between the individuals seeking support and the individuals providing care, just due to the representation of different backgrounds in clinical service. And it is already known, outside of limbic, it is just a known thing in the literature that the mismatch between and you know patients not feeling represented by the clinicians available to them problem it creates a friction point around getting started on an already hard first step to actually you know seeking support yeah and so you can remove that challenge by getting rid of wow the the demographic identity see most people have tried to solve this problem by getting clinicians to match the identity yes individuals seeking support yes which is very challenging that becomes a recruitment challenge um and you know the um the funnel of trained clinicians becomes a limiting factor there yes actually a solution that maybe no one predicted but is absolutely true in the study we found um is that you can actually solve for this by removing any demographic identity because it's an AI and it doesn't have an identity wow and a stupid question the voice of the AI how do you I assume you go for like a neutrality there like how is is there any thought behind that I assume there is yeah it's something we're trying to figure out as we go um I think there are loads of directions and creative ways you can improve but initially just a neutral voice gets you pretty far and the sky's the limit on where we can go from here yeah yeah sorry yeah I mean apologies for how you know detailed and nuanced I'm going here but I what I'm the reason for it is because I don't get to ask these questions to someone who's built mental health chatbot AI that's actually delivering it to half a million people and so I know that there are so many answers in here for for the blueprint of how we do this at scale because the devil is in the detail the devil is in the nuance of of quite rightly not going binary what I used to say AI should not be placed at the point in suffering alone to now going I can't live myself actually saying that out loud anymore like the devil's in the detail of like what's the actual model of doing this and things like what voice is it what what you know how how does it sound like does it declare its ai these are the points that actually matter if we're going to be able to use this at scale so being able to actually talk to someone that's figured this out academically with research is amazing so we're at step one of your of your whole patient journey here we've taken a while so i'll try not to interrupt you for the rest of it But yeah, talk me through the rest of that patient experience before we then move on to the tech.

23:05Yeah, so I mean, let's talk about the tech in tandem because the product is tech. So many AI companies are not AI companies. I'd say 99%. They are companies that are either using AI as a marketing buzzword to gain interest. or they are applying AI that was developed by others and just wrapping it up into a product. And that just makes you at best and apply, like, you know, it's part of your tech stack. To do this right, beyond clinical evidence and regulatory approval, which we've done, you also need to have AI scientists who are trying to use AI for clinical purposes. So we've got, I think, 10 PhDs in AI on our team.

24:00Really, like we look like a research institute rather than we're saying we use AI. And they are developing the systems in the background that make this technology safe and effective. So to put that in concrete terms, at that front door, yeah, you're speaking to this language model, which is very empathetic and engaging and very clever. But behind the scenes, other AI, which has been trained on clinical data, is predicting the diagnosis, which is predictive AI working behind the scenes. So you've got different types, right? You've got language modelling, but then you've got clinical predictive AI, kind of the brain of this system.

24:45Behind the scenes, trying to figure out what you, James, might be presenting with based on everything that you can do. and this is really important because previously or maybe some other solutions i've seen in the space they're all very rules based they're very like if this yeah but the benefit that ai gives you is it can merge multiple different types of information and come up with a very accurate and very precise guess that has been proven as you know uh very very powerful in clinical papers that we've published, like way better than a rules-based system, way better, because it's able to integrate all these different information streams to come up with a precise and quantifiable estimate.

25:34And so that is working in tandem with a language model to try and identify what your issue is and get you booked into the right pathway as soon as possible. And we published another paper with 65 000 patients in the sample which showed that um there is a 45 percent reduction in care pathway changes for patients that access care through limbic versus traditional channels and if i if i try and if i try and sort of uh give you the conclusion what that means is typically individuals will access care they will they will receive a clinical assessment and then they will go on to a treatment pathway but that process isn't perfect and often you will find that patients will be on a depression pathway but actually after three sessions with a therapist it becomes obvious that they should really be receiving treatment for social phobia and what originally looked like depression because they were staying indoors and not going out actually turns out that social phobia is the correct treatment pathway because really they are struggling primarily with being around other people so this is a metric of efficiency of resource it's a metric of of whether or not that initial assessment was correct or not yeah when it's done by done by a human clinician and i'm very i'm i'm very bullish on human clinicians i i really admire and believe in this limited resource that we have.

27:12But how can we expect them to be right 100 % of the time when they are so overburdened and overstretched and just deal with the volumes? So what Limbic showed was by having our AI use the diagnostic model behind the scenes to make the prediction and be very accurate there, pathway changes downstream decrease by 45 percent it also doesn't stop a hybrid model at that point focusing on diagnosis now as well it certainly doesn't prohibit you know thinking of a utopian future it does not

27:47James Somauroo:prohibit the human being the front the face the the one in front of the patient with the ai working in the background it's just at this point in time we also need the ai there it's a marriage I think it is so simplistic and naive to think that AI will replace human clinicians. But I think what it will do is it will scale them to 100 or 1000x. I think the way you think about that playing out in practice is, you know, right now you already have a staffing pyramid. You've got maybe a psychiatrist at the top of a care team. maybe you've got some psychologists, maybe you've got a few more CBT therapists or licensed clinical social workers.

28:33You've got a different number of all these different roles. And then you should just think of a clinical AI like Limbit as the final base layer of the staffing pyramid, but it is infinitely large. You'll never need another. And the job of these AI is to go out and do a lot of the heavy lift but they are overseen and supervised by clinicians and i think it's that marriage which means the original clinical supply is going to 1000x yes 45 reduction in pathway

29:10James Somauroo:changes is an amazing statistic um because as you quite rightly say that is a statistic of you know getting it right first time but i do also just want to make the point about that's basically half the time that you're not having to start again and repeat the resource so even if you said and it wasn't an ai that was going to actually do the diet but even if you just went into an organization and said by the way for those half of times you get it wrong that that you're then struggling to actually diagnose them the treatment's not working you're trying something different you try and then eventually you hit it we can remove all of that that's an incredibly compelling argument for anyone and anywhere in healthcare like that's that's an incredible argument to a save resource and b get it right for the patient first time because let's not forget there's a patient at the end of all of that as well which i admire was the first thing that you went to i i think the resource part of the conversation though also does have to be brought in here that that that my goodness let alone everything downstream but but just just that part being able to change that for the health care industry is an incredible financial argument right and in a resource constrained place where we're struggling for humans you're actually just creating an extra what is it 20 30 40 like the capacity that you're even releasing there is incredibly important.

30:41Yes, exactly. And this is, again, what makes us different from just a consumer app in the app store. We are integrated into the care pathway so that we can generate service level efficiencies, which you can't do with a standalone app. So Limbic is integrated into the electronic health record, and we are part of the care pathway so we can make these improvements. But you're quite right, James. All of this, we can have the best intentions in the world but at the end of the day healthcare you need to think of it as a as a business or at least think about the economic component of delivering care and you have to find ways to improve margin so that services can do more with less and that's where most of our go-to-market thinking revolt it was yes like step one can we build a clinical solution that actually makes patients healthy and then it's step two can we find a way to integrate this into the healthcare industry such that it creates economic value and we have a business absolutely it's one of the areas that

31:52James Somauroo:i think we we struggle with in healthcare a lot isn't it talking about money but it's it's interesting because no one's well in the uk anyway public healthcare system let's simplify it no one's here to make profit we're just trying to save money so it can be spent everywhere else it's just that money's an analog for resource but we shouldn't actually worry about talking about money we should we are really you could call it tokens you could call it energy coins you could call it literally resource coins it's literally it's just that it happens to be money which is what we also take home to spend on things it's just that it could be completely different it it's a resource conversation you can't get away from talking about money and uh we need to just be comfortable with the uncomfortable conversation but you have to talk about money because we have a fixed budget for health care and we need to deploy that sensibly so that everybody gets the highest quality care right now everybody does not get the highest quality care right we have problems around supply and demand so you have to think about how to spend the the pounds the gbp sensibly so that you can make it stretch and ai is a phenomenal way to do this one of the things that i find so um troubling is that we need to be faster the healthcare industry and i consider myself part of it just a very very specific pocket but we need to be faster at thinking creatively on how to stretch budgets so they can go further and a maddening conversation i had uh recently and i won't i won't i won't name names um but i was speaking to somebody extremely senior deploying NHS budgets and I asked what is the biggest problem that you have and they said we don't have enough staff and we can't recruit more because there isn't a sufficient pool of clinicians to recruit from and I said that sounds like a really hard problem so and they said we've now been given a very large budget to solve this problem and i said what ways are you thinking about solving it and they said we're going to spend that money on recruitment but i said you don't have enough you just you just told me you don't have people to recruit they don't exist yeah i said what about what about generating the equivalent of full-time staff using sensible technology and solving your problem that way and they said completely convinced but the budget has been allocated for recruitment yeah so the budget gets allocated for a very specific method of solving the problem, not towards solving the problem more generally.

34:49And they know ahead of time that the method of solving the problem won't work. And it's this, nobody is at fault, but just system-wide, there is this recursive fallacy around how we actually solve the biggest problems in healthcare. And I just feel that, you know, responsible, clinically evidenced, regulated AI should be nationally available. There should be national tenders out for this right now. And I'm not saying we should do it without care or thought or recklessly, but we could easily define these are the criteria, you know. it needs to be a regulated medical device and it needs to have had the mhra come in and all everything they do that's one requirement it needs to have demonstrable peer-reviewed evidence that it creates outcomes great that's another thing that we should absolutely be asking for it's to have many years demonstrating efficacy within the intended environment i mean you can you can be as prescriptive as you want on what the criteria are but there should be nationally available um sort of tenders for ai to solve the staffing crisis yeah i think the difference

36:14James Somauroo:there ross is that you have a vision for it and i think what i think what lacks nationally is a vision for that for those technologies to slot into we should we don't know what we're aiming for i think what limbic have done what you have done leading limbic is that you've set out a clinical pathway of excellence that is safe that is effective that has morality built into it that has all sorts of good stuff built into it right in in terms of it works and it works in the way that you want it to work which is a reflection of your morality and ability frankly in order to understand what's happening on that side this side the other side and what's possible whilst maintaining a view of what's going on technology to keep it updated don't overlook the fact that you're not the majority there in that and there's like a floodgates potential thing of like ai being deployed absolutely everywhere doing absolutely everything under no fixed vision and it all gets a bit chaotic now i think that's probably still slightly better than what we have going on now it's just that what i'd really be advocating for is to for people like you if not you personally to actually have a role in defining this albeit conflicts of interest aside because the way that you've explained what you want mental health care provision to look like in a world of AI I tell you it's not the norm man it's not the norm because like as you you know the amount of apps that are just skinning chat gbt and going out there doing all this stuff you know the bad actors because they appear every now and again as your competitors you've seen this you know this and so So I just don't know how we do that.

38:04James Somauroo:I don't know where the national vision comes from that then we go, yeah, okay, here is the nuance. You must make sure you declare it's AI at the beginning of the phone call. You must make sure that it's a neutral voice. You must make sure that da-da-da, but now let's go bullish. like we we don't want it here we don't want it there stay away from that we're ring fencing those that's human only you're not we're not there but let's go bullish everywhere else where does that vision come from because i think we can all be bullish underneath that vision i agree and there are different levels of um bullishness for going with that phrase but i think james you you can devolve responsibility for figuring out what is appropriate by going with the trusted process for how we do that with everything else but like with with pharma okay yeah fair i don't i don't say oh well just because we're saying that we need to make available SSRIs for depression and anxiety that they're giving to everyone we don't say oh but that means that everybody will be bringing to market products that are untested and unsafe because absolutely if you want to come to market and you want to be in this space you're going to need a deep clinical evidence base you're going to need to have conducted trials and you're going to need to have had the authority the regulatory body find you to be the claims you make have been validated and verified but then isn't that enough like why must we always have another layer of uh evaluation don't we just say let let these institutions um go through the process of validating claims and making sure it's all and then if the claim suits the problem or the solution suits the problem why are we then still so nervous about rolling it out.

40:04I think that's one of the challenges I have found is at every, every regulatory hurdle we have jumped through, I always thought, and now there is no reason why it isn't used 100%, you know, but you still like, there's, there's still some sort of like skepticism. And you're like, what do I need to show to reduce that? And, and I think it's that people still feel that they they all need to do their own evaluations yeah to get together as a community and say this this um evaluation is sufficient and this could be a national tender that just says if you want to be a serious um uh contender then you need to have been found worthy by these trusted institutions that we now put responsibility on to evaluate do you not think

40:54James Somauroo:yeah the solution or i do i know i do i do and that that that sort of answers my question of where does that national vision come from because i think that then becomes a framework yeah of like okay if this if this if this organization this group of people i used to i used to call it a common sense committee that they need they need a common sense committee that is people that can make a decision in the room and they get together and they just make a decision in the room and and and it just removes a lot of a lot of stuff but if they're you know if they're able to set that okay now this information governance has been done for guys and sir thomas is therefore it is finding kings and barts and blah blah blah can we all just relax like it's fine the people that set that then also probably set well here's where we are comfortable with ai and here's where we are comfortable with it being human only and that can change but this is where we are now let's go you know pedal down on this model and then review it in three months like that seems like an appropriate model i just don't know where that comes from but before i go too much into rabbit hole i know we got you for a few more minutes um i think we're gonna have to do a part two on this by the way because i'm going to finish off on diagnosis and then i'm going to get you back on to talk about treatment because i think that's a whole other thing that i want to talk about but with diagnosis so the language model and the predictive model are working together in this diagnostic component of what limbic does this sort of two-way dance between the two in order to arrive at the diagnosis is this being done that this is this is being delivered still by the ai and voice right so is that's a question so is that are the questions being formulated in response to the answers is this a moving model as this diagnosis is emerging and then my my last question on diagnosis really is then how again super practically how is that diagnosis then delivered to the patient to do ai right in healthcare, you definitely don't want a language model implicitly making decisions.

42:56It's not ready. And even if it was performant, it's very hard to quantify the error rate and sort of hold it accountable. There are too many degrees of freedom with the language models. they're, you know, 175 billion parameter models. So you just can't do it. What they are very good at is holding a naturalistic and engaging conversation. They are very good at that. So you need to disentangle any clinical reasoning from just the ability to talk. And that's why we have behind the scenes, this specialist cognitive architecture that we call the limbic layer, which is doing diagnostics, but it's doing a bunch of other things as well.

43:45And is designed to do clinical reasoning. It is not a language model. It is a statistical system. It is predictive AI. It is more like traditional machine learning, where you've seen image classifiers in radiography. And, you know, it's not a language model. It's statistical predictive AI. And then you let the language model, whether it's voice or whether it's text, it doesn't matter. it's just a different medium you let that hold the conversation and collect information and then you pass that through to the clinical reasoning system that has been specialist trained for this job and this job alone and has a known error rate and how you know you can quantify profile and you can publish papers and you can get it approved and then you let that make decisions that then just get relayed through a language model but you never let the language model make the decisions You must dissociate those two functions because they are different roles.

44:42James Somauroo:Have you guys become okay then with the AI actually delivering that diagnosis to the patient? So the AI itself is telling the patient what it thinks they have and that side of things. No, that's a product decision, not a technical decision. And you basically just need to figure out what's appropriate. And what we settled with our NHS partners is you would never relay, you never have the ai say james you have depression yeah right because it's too much uh and you definitely want to have clinician oversight so what happens is i i believe um in the current product incarnation is that we would the ai would say james it sounds like you might benefit from some help with these symptoms of low mood you know that that would be a compliant way to talk about it.

45:33Do you mind if I book you in with a clinician who's going to talk more about that and get you to? And so then we would relay you on to a clinician, but we've massively leapfrogged a lot of steps. We've freed up staff time in the back end. We've got you the help you need at 2am on a Wednesday morning when you were at the peak of your help seeking behaviour. And now you are in the process. Now you are going to get the support that you need. And the clinician will have huge decision support from everything the ai collected so that they can just dot the i's and cross the t's and probe for complexity and then make that final decision i love this man this is i'm learning