DAIM: Inside The Algorithm | Hybrid Simulation, Model Reuse and AI in NHS Planning

0:00Dr Jeremy Bradley:What if demand for kidney replacement therapy rose by X amount in the next couple of years and then have the simulation run and produce those results for people? I think that would be really powerful because I don't think we can underestimate the amount of other things people in the NHS have to do. So anything that we can do to make it easier for them to interact with tools that can help them is a win. And it's worth doing. It will take development time and then.

0:32Dr Lucy Morgan:Welcome to Inside the Algorithm, the show that goes beneath the surface of artificial intelligence into the scientific research, technical breakthroughs, and academic thinking shaping where this technology is actually heading. I'm Jeremy Bradley, Chief AI Officer at Canberra Spark, a leader in transformational data and AI upskilling, career development, and progression. Each episode, I sit down with the researchers, data scientists, and technical experts working at the true frontier of this field to explore not just what's happening in AI, but also the real science and thinking driving it forward.

1:04Dr Lucy Morgan:Whether you're building sophisticated AI systems, leading teams through AI transformation, or driven to understand what's really happening beneath the headlines, this is the show for you. In today's episode, we're getting into one of the most consequential applications of AI and operational research in the UK right now, The challenge of planning healthcare at scale inside the NHS. My guest today is Dr Lucy Morgan, Head of Simulation at the NHS Strategy Unit, visiting researcher at Lancaster University Management School and 2025 Katie Toker Award winner for her work in simulation modelling. Lucy has led the development of a renal replacement therapy planning model that started with the Midlands Kidney Network and is now being rolled out nationally with ambitions to become a UK-wide planning tool by 2029.

1:51Dr Lucy Morgan:Her current research focus, as we'll hear, is model reuse and reproducibility, and specifically how AI could help build the trust and standardization needed to deploy analytical models across new organizations at scale. This is a conversation about what it really takes to make modeling work in the real world. I hope you enjoy it. Hello and welcome, Lucy. Thank you so much for joining us.

2:22Dr Lucy Morgan:I'm really excited to be talking with you today about your OR work and simulation work in the NHS Strategy Unit. You want to tell us a little bit about the Strategy Unit and what it does and how it operates?

2:36Dr Jeremy Bradley:Yes, so we're internal to the NHS, so we're all NHS staff. However, we're kind of a unit of our own and we work across the NHS in a kind of consultancy way. because we're internal we don't make a profit it gets put back into the NHS so it's it's kind of a bit of a strange consultancy because we are there to solve people's problems as a normal consultancy would but it's almost in a slightly reversed way where we try and create solutions that we can then use in multiple places rather than making a solution and selling it to multiple people So the ideas of model reuse and reproducibility and transparency and open science, open coding are really part of the strategy unit mantra.

3:24Dr Jeremy Bradley:So I'm in the analytics and modeling team there and heading up simulation, which is an exciting place to be.

3:32Dr Lucy Morgan:Nice. And they operate a model of backwards consultancy, I understand. So what does that entail in practice?

3:42Dr Jeremy Bradley:That's how I always describe it to people. So when I think about making a model, I'm not just thinking about making a model for the trust or the part of the NHS that I'm currently working for. I'm trying to think, well, can we do this in an open and transparent way such that if other areas want to use this model in the future, they could then come in, take that code, personalise the model to themselves. because the NHS is made up of many parts that look very similar. Like your A &E department in one trust will look very similar to an A &E department in another trust. They may function slightly differently, but not so differently that you couldn't adapt a model potentially to work across them.

4:28Dr Jeremy Bradley:So that's the idea, to try and make something that we don't have to charge again for, or we don't have to keep reinventing the wheel and starting from scratch. and to to try and make more and more basically making analytics as open as possible and as available as possible to um other analysts working within the nhs because i find sometimes that they're um analysts that are within trusts are working in kind of like a very siloed that they're often working on their own so if we can make things that they can use it actually just powers them up to do more.

5:06Dr Lucy Morgan:And then specifically, more recently, you've been working on a very contemporary topic of healthcare waiting lists in the NHS. And there isn't almost a month goes by without some discussion of NHS waiting lists for some operation or procedure or others. This is a really important topic but also quite a quite quite a complex one as well do you want to give us a little bit more on on what your work's been in that area I'd say that at the strategy unit I kind of got into the waiting list area because a colleague of mine professor Mohamed Mohamed

5:44Dr Jeremy Bradley:um kind of when I first joined the unit realized oh I've got a bit of background in operational research and simulation and queuing so he brought me into um to basically learn a little bit more about what he was already working on, which was to kind of orchestrate different efforts that were going on across the country in waiting lists. I got into it through that and then started working with academics at Durham University and other NHS staff down in Bristol. And we were looking at the problem of how many people do you need to remove from the waiting list, say every month or every year to actually achieve the waiting list target that's been set which is um 18 um weeks basically so 92 percent of the people waiting on the waiting list will have waited 18 weeks by the end of this parliament so i mean we're nowhere near that at the moment but we are moving in the right direction so um yeah it was a a great starter problem but it's not the only problem in waiting lists and And it ended up being, well, you could look at it from a national viewpoint, like as a national waiting list or down to specialty level.

7:01Dr Jeremy Bradley:So there's the complexity there from there being many, many different problems to solve across many specialties, across many trusts, etc. So, again, it's a problem of scale.

7:13Dr Lucy Morgan:So doing theory, which you're using to model these essentially very large virtual queues, they're not actually people standing in line there are people just waiting at home of course and sometimes waiting many many months even years to to get to the top of the the top of the queue and get their their operation a a really powerful technique for capturing these these challenges in any complex organization do you find there's a do you find there's a bit of a gap between you know using a theory like that to inform uh inform the process that or inform a hospital or um you know it's just around how many people should be removed from from a list and then

8:03Dr Jeremy Bradley:how they actually then go about executing it yeah so i think um what we did what was done was a really important first step and really important question to answer i mean people want to know how to achieve the target so the first thing you need to do is tell them how to get there so how many how much relief capacity they need to give the system in order to move towards the target so that is an important first question secondary to that is the almost operational question of then if I need to take 20 people off my waiting list this month who on the waiting list do I need to take off and that kind of brings in questions of prioritization and deterioration whilst you're on the waiting list because some people have been waiting a very long time and it's not as simple as just taking those that have waited the longest off the list there's some dynamics in there that mean that you know there's definitely better ways to approach it than that and so that's where I feel like AI could really make some differences or AI techniques let's say could be really powerful to not just look at how many people to take off but how do we actually approach that and on a daily basis or weekly basis scheduling who should come in and when.

9:29Dr Lucy Morgan:I'd like to give a sense of what your sort of levers are here when you're making these sort of suggestions or adjustments to the queue because as you say it's not just a situation where you're saying we just want to take the first 10 people the second 10 people it's it's going to be it's going to be sort of aligned to well this surgeon only carries out this type of procedure so now we're limited to you know the first three or four people who can maybe be seen in a day for whom that's applicable on the waiting list and so so there's an element of going down the waiting list and find finding the people who actually match that that that procedure for instance but then as you say then there's an interesting sort of extra complexity around the how difficult the procedure may be how how how much time it will take presumably things like how much time in recovery there'll be afterwards and whether there are beds available for them there as well does that come into the equation absolutely

10:29Dr Jeremy Bradley:like the operational barriers are almost more important than the the technical ones i think you can come up with some really nice uh like methods and i've seen this i was looking at it into it um there are trusts in the uk where ai is being used to do things like you know prioritization scores for patients on the list to try and understand who's you know the highest priority to be brought in and for elective surgery next etc and um it's even being used for validation i.e. looking at people that have been waiting a long time and seeing who actually still needs the surgery so there are really nice techniques out there that are looking at that type of thing but then yeah as you say your your theatre capacity or who's working when those types of rotors etc it's the complexity is is incredibly high and often you find that the people making the decisions i.e.

11:29Dr Jeremy Bradley:the people that are like waiting list managers or actually doing they've got a lot of um contextual knowledge that is hard to capture in a model that like they know things because they've been doing it for such a long time that kind of learning is something that um

11:46Dr Lucy Morgan:is is hard to replicate how do you find the data in this space because to get usable reinforcement learning approach in that in that area classically you need even for a tabular model you need you need quite a lot of data just to be able to reasonably populate the the decision and the reward table um and if you're using something that's neural network based then then even more data so is it what's your experience with with getting getting good consistent data out of out of NHS trusts is that has that been an interesting

12:22Dr Jeremy Bradley:conversation well I'm in a lucky position because because we're within the NHS we actually have got access to um very granular data often um within the strategy unit however when you're an external that's not the case so I think that has been a barrier for sure so some of these um more interesting research gaps that are closing between academia and practice generally because of NHS data being harder to access it it's harder to do that within the NHS and I think things are generally moving in the right direction on that front but for what we're discussing here it would need to be very local level data and that would be quite hard to get a handle of but And that's kind of why the approaches of, you know, the queuing theory methods actually use open source or openly available data sets like the referral to treatment data set, which is updated on a monthly basis by the NHS and published.

13:28Dr Jeremy Bradley:So they've used what data they have available to use an appropriate technique to actually do something useful, which is good. but we need to go under the hood and much lower, more granular level to look at more, I guess, more granular techniques.

13:44Dr Lucy Morgan:Speaking of really challenging problems, one of the other areas that you've been working on is area modelling kidney transplant processes. Do you want to tell us a little bit about the models and the problem area there?

14:02Dr Jeremy Bradley:this was actually the problem i was brought in to tackle when i when i joined the strategy unit so um they had a relationship with midlands kidney network and our the aim of the project was to look at um trying to provide um a holistic view of renal care um and that came in kind of two parts so i'm now a bit of a kidney expert myself but just to highlight it overall you basically end up on dialysis if you develop kidney disease and that progresses to the point at which you get kidney failure so we're going to model from chronic kidney disease through to end stage kidney failure or end stage kidney disease which is when you have dialysis um so we did a um a two-stage model a sequential hybrid simulation model which if you've not heard of hybrid simulation models they're just kind of a combination of different techniques so we try to think about the process we're actually modeling and match the simulation technique to the physical process that we were so kidney disease is a continuous process you are you are deteriorating over time but you don't wake up one day and have a switch from stage three to stage four five for example it's a it's a slow deterioration and and therefore it suited us to use um system dynamics or it suited the problem to use system dynamics which is a much more high level uh population-based flow-based methodology and the out um kind of output of that model is the incidence of patients into kidney replacement therapy and so it projects how many people it thinks that over time are going to be coming through to actually needing dialysis and transplant.

15:49Dr Jeremy Bradley:And then the second half of the model is actually, oh, actual pathways of people living with kidney failure that need dialysis and need to be going into hospital, you know, need a transplant and actually receive one, etc. So there's the two sides of the model that then fit together to give this view of capacity for kidney replacement therapy, which is something that is a little bit of worry in the NHS because the need for people to be going on dialysis is growing and growing and there need to be intervention to actually control that capacity for specifically for in-centre haemodialysis which is where we're running out of space I guess for people in the future to be coming in and actually being treated so what could they do could they prevent it earlier down the pathway which is a pro of having this holistic view or can they actually change people onto different types of dialysis which is within the second part of the system and the actual kidney replacement therapy

16:49Dr Lucy Morgan:system yeah gosh constructing a model around that i mean i think it's difficult to underestimate how challenging and how difficult it is to get get that kind of hybrid simulation of of a complex system right i'd like to explore that a little bit so when you're talking about that population that fluid element which is describing the pathology is it the dynamic pathology of the of the kidney disease that's more of a continuous sort of simulation a continuous sort of representation of the progression of the disease is that do you have distributions of people with different dynamics how does that side of it work so it's a stock and flow system so

17:35Dr Jeremy Bradley:there are a few stocks within the model like undiagnosed stage three chronic kidney disease diagnosed stage three chronic kidney disease and the model effectively looks at the flow rate between the different stocks in the system so you can progress having not been diagnosed or you can progress having been diagnosed and then there's the flow rate for actually having been diagnosed in the model and there's certain I like how my colleague Sally describes system dynamics because she thinks of it almost like as bathtubs where you're turning a tap on and off and she describes the use of medication that can slow the disease as almost like turning the tap so it's tighter so you're not having as a larger flow coming from one stage of the disease into the next and so there were different ways we could intervene within that system to kind of slow the flow through and therefore the flow through into kidney replacement therapy.

18:34Dr Lucy Morgan:And do you have data which then gives you some insight into how often people get diagnosed so they sort of transition between the diagnosed and undiagnosed pathways across depending on where they are in the progression of the disease?

18:49Dr Jeremy Bradley:Yeah, so there's a big study out there that's ongoing called CBD Prevent, which is like about cardiovascular diseases effectively. But within that, there's data on kidney replacement therapy and chronic kidney disease also. So from there, we're basically able to understand at regional level or down to regional level, how many people are approximately undiagnosed and how many are diagnosed. And from that fitter a model that we were then able to validate against observed numbers that are in chronic kidney disease. So there's a little bit of calibration needed in that model to kind of train it to produce reasonable results.

19:35Dr Jeremy Bradley:And also, we were very lucky because we had experts working with us throughout that from Midlands Kidney Network. So, yeah, clinicians are really deeply ingrained in the model.

19:47Dr Lucy Morgan:I was going to ask then, naturally, as a scientist, how did you go about validating against the data? What did the process look like? It sounds like you had fantastic support. So that's a really great start from subject matter experts. But how did you use the data to then give you that, this parameter, this flow rate, this is a good representation?

20:13Dr Jeremy Bradley:So for the kidney replacement therapy side of things, I mean, there's a really large literature on simulation validation. So we use standard techniques effectively, but they're not too different to how you train any algorithm. So when I was looking at the kidney replacement therapy system, I used data from 2010 all the way up to 2022 and looked at fitting all my input distributions with that, how long people were living with transplants and how quickly they were transitioning between different dialysis modalities, etc. fit my distributions using those data and then from 2022 onwards looked at how well the inputs sorry the people flowing through the model represented what was going on in the actual system to get a good fit and then you know calibrate from there if necessary but generally we get a reasonably good fit to start with unless there's a systematic change which can happen um in in a system so you've got a low level fitting and then a higher level

21:18Dr Lucy Morgan:sort of system almost system level validation based on the sort of population dynamics that you see is that right yeah effectively yeah nice and then on the other side you've got this much more granular probably a simulation model i'd be more familiar with what sort of discrete event um process which follows the the the stages of pathway treatment would that be and then you've got a way of having the two simulations talk to each other so how does how does that how does that

21:48Dr Jeremy Bradley:sort of play out at the moment they are so one's a web app chronic kidney disease and it's a toy you can actually go on online anyone can access it and just look at the projections of what we think out to about 2035 is going to be the inflow of people coming into kidney replacement therapy and then you can at the moment it's a little bit clunky but you can basically just download that instance profile and put it into the interface of the other app and then run that forward through time so we always have the baseline projection ready to be used within the kidney replacement therapies part of the model but if you did an experiment in the earlier part of the model i.e.

22:28Dr Jeremy Bradley:prevention or more more medication for people earlier in the pathway you could take that in instance profile and plug it in across so at the moment it would be lovely if that was seamless but we are running out of budget for um actually creating this so that's something i'd like to do over time uh to improve things but at the moment they are kind of two separate models that can talk to each other and with a human involved

23:02Dr Lucy Morgan:one of the things you talked about at the start was being able to take a model uh whether it's a a cue a cue model from from the waiting list whether it's the for the kidney model that you've i think you've worked on uh you've worked on the kidney model with one with um one particular trust and then try to get the benefit of rolling it out and that sounds like quite a challenging process. I know from personal experience that that doesn't always work first time, going to a different operation with different constraints potentially and different people involved. So how have you found that and what's the pathway that you're observing there?

23:40Dr Jeremy Bradley:I'd say I'm still in the process of learning how to do this well. This is the first time I've ever a model and try to take it to other areas and reuse it. So the renal model that we've built is actually, it's very, very similar in different regions of the country. The input models would need to be retrained on the data, but we have that data. So it feels achievable to actually take it to them. And we started doing that now. We started in the Midlands who were the people that commissioned the model but we've rolled out to London and to the east of England and there are other networks now that are coming on board so we're hoping that we will reach the whole of England in the next year or so but the problem is when we're going to other other regions they haven't been involved in the model building process so where the Midlands were there throughout the workshops that we did we did a participative modelling approach where that we We had workshops where they were with us when we mapped the whole system and they understood that we understood the system.

24:49Dr Jeremy Bradley:So that was the trust between the modeler and the client was effectively built. And then they also came with us through the validation process because we had workshops on that to show them, this is why it works. This is why you should trust it. But the other regions haven't actually had that experience. And it's almost that hurdle that I think is harder to bridge than the technical aspects of rebuilding the model for their specific contexts.

25:18Dr Lucy Morgan:Do you see a role for AI machine learning in supporting that bespoken process or indeed that trust process in terms of helping the regional managers believe that this is a really complementary service that you're developing?

25:36Dr Jeremy Bradley:I think that's where I'm really interested in research at the moment, like solutions that can help people understand models, almost question models. So I've had some conversations recently with academics at Exeter. so professor tom monks there he's been looking at um potentially using agents to allow people to question documentation and that type of um approach so that they could say or or even question a model like um what if demand for kidney replacement therapy rose by x amount in the next couple of years and then have the simulation run and produce those results for people i i think that would be really powerful because um i don't think we can underestimate the amount of other things people in the nhs have to do so anything that we can do to make it easier for them to interact with tools that can help them is a win and it's worth doing but it does it will it will take development time and i'm i've not got a clear picture of how that's going to look or how it's going to work best at the moment but that's why yeah actively trying to do research here so do you have a call to

26:51Dr Lucy Morgan:arms here it sounds like it sounds like you're keen to keen to use open tools and you know listen to the community and bring in expertise is there is there something you'd you'd like from from the