AI in Healthcare: Accelerating Clinical Trials with Patrick Leung

0:00We've seen this before, like in the dark ages of AI, there was another revolution around imaging. Everyone thought all the radiologists are going to lose their job because this is all becoming automated. AI is going to do all the interpretation of radiology results, MRIs, x-rays, and so on. There won't be any more need for radiologists. And in fact, what we said is that there's more radiologists than ever. As a result of all this automation, the demand has gone up a lot. Welcome to the Talking AI Podcast, where we talk AI with both experts in the field and early adopters. I'm your host, Matt Page, and we're here to demystify AI for you so you can get some value from it.

0:32Let's talk some AI.

0:36Every year we spend billions trying to bring new drugs to market, yet most still take over a decade to actually reach patients. The process is slow, it's complex, it's filled with red tape, but what if AI could help change that? And today on the Talking AI Podcast, we're diving into one of the most critical and sometimes overlooked areas in healthcare clinical trials. My guest, Patrick Larn, CTO and co-founder of Faro Health, a company using AI to make trials faster, smarter, and more effective is on with us today. Welcome to Talking AI, Patrick. Glad to be here, man. Yeah, this is going to be an interesting one.

1:10And I got to say, like, I am not a healthcare expert, but some of the stuff I'm seeing from a lot of the experts on AI, like this is going to be a major impacted area, but I'm curious, like what specifically with clinical trials, what makes the trials so slow and expensive? I'm reading that stat, like 10 years, billions of dollars. It's insane. What's your, like the current state of things? Why does it take so long? I think that it's taken decades to get to this point. Oftentimes when I go to conferences or I'm on these podcasts, I talk about Eroom's law, which is a bit of dark humor on the part of the life sciences industry.

1:52And they've taken Moore's law, which is the classic doubling of chip density and computing power every 18 months. They have their own version of that called Eroom's Law, whereby the cost and time required to get new drugs to market doubles every nine years. And that's held over the last 50 years, at least, probably 60 or 70 by now. So anyway, that resulted over time in this huge increase, this exponential increase in cost and time. And there's a number of different reasons for that. So the original author of the paper that introduced this concept or this effect talked about things like over time the number of new drugs keeps on going up but the number of conditions doesn't and so you get end up with drugs trying to treat an existing condition that's already being treated by an existing drug and so the bar for actually creating a new drug goes higher and higher as the existing treatments get better similarly once in a while you get a crisis like the opioid crisis for instance yeah yeah things like are definitely going to cause regulators to pay more attention, to be more conservative, and to try to prevent crises like that from occurring after a drug's been released.

2:56And so there's a couple of other factors like that, but over time, they conspire to really result in this dramatic increase in cost that we've seen. The stat kind of blew my mind because I'm not a healthcare expert by any means, but billions of dollars, 10 years to get drugs to market, and it doesn't seem to be getting any faster. What is driving that? What's driving these bottlenecks in the process? us it's actually been a cumulative effect over the past 50 or 60 years in fact we have this phenomenon called e-room's law that's a bit of a sort of dark take on moore's law which is the doubling of transistor density every 18 months so with e-room's law we have the doubling of costs that it takes to get a front to take a pharmaceutical from inception to market that doubles every nine years and so that's held over the last inflation adjusted yeah wow that's the opposite of what we want is going the other way.

3:48Exactly. And there's a number of different reasons for that, right? I mean, the original author of the paper that introduced this effect said that one example is over time, the standard of care, like the standard treatment for a given condition will get better and better over time as new drugs are introduced. And therefore, given the fact that there aren't all these new conditions to be treated necessarily, you have drugs competing to actually treat a condition that's already pretty well treated. I mean, there's a higher bar for that drug to reach the market in order to actually be approved. So that's one thing over time that leads to just more cost.

4:18There's another effect where if you look at the opioid crisis, it's a recent example, right? Where a drug that already made it to market has some real problems and risks that weren't originally evident. That causes the regulators to become more conservative. They don't want that to happen anymore. They're just going to look back and say, what mistakes do we make? How can we do our part to make sure that these things don't happen again? And as a result, they get more and more conservative and careful over time. with good reason, but over time, this sort of conspired to really result in that doubling every nine years that I mentioned before.

4:47Quick break in the pod. If you're listening to this podcast, chances are you've been thinking about how to actually use AI inside your business. And that's exactly why we built the AI Opportunity Finder. It's a free tool that helps you uncover high impact, tailored AI use cases based on your business, your goals, your pain points, and your industry. No fluff, no generic use cases, just real ideas that fit your business and the ranked by ROI potential. It takes about three minutes to run and it's like having your own personal AI strategist for free. If you want to try it for free, check out the link in the show notes or go to hatchworks.com backslash AI dash opportunity dash finder.

5:26Yeah, so there's the positive side, but also that additional red tape in essence that's accelerating things in a sense. Yes. And I guess, so AI obviously has wide ranging impacts, but where is it being applied within clinical trials? Where are the areas that it's actually helping accelerate things or where are you starting to see these patterns emerge? I suppose. There's a number of different areas. We've been at this for over two years now, just looking at how to apply the latest large language model technology to help make these clinical trials better. And so one obvious area that there's a lot of attention and action around is clinical writing.

6:07And so you'll see out there, a lot of people, ourselves included, are really applying these large language models to try to automate the generation of complex clinical documentation that's required for the trial. And in many cases, that can really soak up a lot of time and resources to create multi-hundred page documents that need to be at a really high level of quality that gets submitted to the regulators and that essentially outline the whole clinical trial. And so what we've done at FIRO is we've taken, we had this very advanced design tool that clinical scientists can use to design the clinical trial in a really easy to use interface.

6:40And it gets stored in this very structured repository. And we use that data combined with the large language model's ability to generate text into this architecture that allows the automatic generation of these documents. And that's one area where we're definitely spending a lot of time applying AI to make this better. Oh, that's interesting. So I'm assuming, what do you think is the range of what that used to take in the past where it was humans actually doing that versus today, the Delta in terms of time savings there? It's we're talking about days and months, or let's just say weeks and months being compressed into less than a day.

7:19In some cases, less than an hour. And so this is for a first draft. And of course there's probably some wordsmithing and some additional edits that are required invariably, but it'll get you 80 % plus of the way there in a matter of an hour or less, which is pretty phenomenal. Yeah. And so I guess on the flip side, if you're using AI to generate this, let me ask you this first. My mind always goes to this with healthcare is hallucinations and things related to that. Because when you have applications of AI, the riskier areas, a lot of people are very hesitant to do things, especially in the healthcare space.

7:56How does that get factored? And how do you account for hallucinations where the risk level is much higher versus other, I'm writing a blog, some social copy. Okay, something a little wrong, not a big deal. Anything healthcare related, I feel like the radar is just much higher in terms of the risk. And not only is the cost of failure really high, like you get this wrong, it might delay the trial or even worse, it might even impact patients' lives in a negative way. Exactly. So the content itself is complex. If you look at a clinical trial, like the document that serves as the root of all the 10 years plus that you might need to bring to this process, there's tons of really nitty gritty technical details there around pharmacokinetics and around inclusion, occlusion criteria, also scheduling information side effect.

8:42There's tons of data in there. And so one way to ensure that the trial is factually correct is to have this really structured way of representing the trial. And so you can query that and put the data very selectively into each section where it's needed of the trial. And then the other thing, of course, is that we ended up developing this architecture, this kind of agentic architecture. I know it's a buzzword right now, agents and all this kind of thing. But we have one of our agents, one of our intelligent agents, essentially can evaluate the output of the generated content. and essentially ask questions and say, did you include information about how to contact the patient once they leave the trial and follow up?

9:19And did you include various other things and factually checking and all this kind of basically evaluation that ensures that the quality of the content is high. And so this helps to take the place of human review. It doesn't entirely eliminate it, of course, but it really helps to ensure that what we're generating is of decent quality and doesn't do obvious emissions or hallucinations and things like that. That's such an important piece, especially like folks listening. This is how you need to be thinking about AI systems in a way. Like you need to build in these feedback loops. And that's a perfect example of that.

9:53You look at like reasoning models today, they're doing this chain of thought reasoning. They're breaking down problems into smaller steps. That's a great approach to solve difficult problems, but you're architecting it in a way to where the agent is. It's almost like a counterbalance in a sense. So human in the loop's great, but I love that approach of almost having agent in the loop, in a sense, like throughout your process. Yeah, that's right. That's right. It's always going to be the iterative process whereby the final published version of this document has to go through multiple revisions, as it does during a manual writing process as well.

10:26And I think there's because of, as you mentioned rightly so before, the cost of failure is so high, the stakes are so high with this clinical documentation. is always going to be humans needed to put their stamp of approval. It's not like generating an email or an image where you can potentially send it off without even really looking very closely at it, because we've seen this before, like in the dark ages of AI, like 10 years ago, there was another revolution around imaging, right? And so for a while, everyone thought, oh no, all the radiologists are going to lose their job because basically this is all becoming automated.

10:55AI is going to do all the interpretation of radiology results, MRIs, x-rays, and so on. And no more need for humans. There won't be any more need for radiologists. And in fact, what we found 10 years later is that there's more radiologists than ever. And it's simply because as a result of all this automation, the demand has gone up a lot. The cost has come down. The demand has gone up. Nowadays, you get imaging. If you take your rotator cost, you get an MRI right away. And so that has increased the demand for human review. And so radiologists are more numerous than ever, despite the automation of, in fact, in some ways, because of the automation introduced by AI.

11:30So that's pretty interesting. It's such a powerful thing too, because that is a good thing. If AI can diagnose things and is better at that task, that's great. A, it allows doctors to do other things, maybe focusing on the patient care. But the piece that you mentioned there, which I wasn't directly thinking about, but we see this in business too, with Jevon's Paradox, with LLMs, when it gets cheaper, easier, better, all these things, you don't use less of it or the same amount. You use a lot more of it. And I think to your point, there's so many underserved folks that need care, that need these things, and they just don't.

12:05Right. For whatever reason, cost, access, all these things. And that can bring that down in a massive way, I would imagine. There's obviously all kinds of other dynamics in the healthcare space, but that's powerful. I was not even thinking about that. And then obviously improving diagnosis and things like that. I want to jump to that in a second, but I'm curious on the document, the clinical trials, the documentation and all of that. How on average, how long would you say the documentation is? Is this like 50 pages, 100, 1000 pages? What's give me your like average in a sense. A clinical protocol document ranges typically from 100 to 200 pages.

12:44Some are longer, some are shorter, depending on the nature of the trial. In general, oncology trials, meaning cancer trials are more complex for a number of different reasons. They're longer. They require a whole bunch more assessments, like all this kind of thing. And so it varies, but it gives you an idea. This is for the clinical trial protocol, which is a blueprint for the whole trial. And there's a bunch of other documents that come further down the line. The informed consent form, the lab reports, the CSR, like all these documents that use essentially oftentimes use the protocol as input.

13:16So that's why we started with the protocol is number one, it's really hard to get right. And so we figured we would tackle something really challenging that other people might have trouble with because we have this really unique tool that we've built, the FireEye Study Designer that allows you to actually model the trials properly and receive insights and analytics and all that kind of thing. And so then after that, a lot of these other documents become easier because you have the source document, the protocol, and in many cases, it's just adapting what we've already used to generate the protocol in order to generate the other documents.

13:44But it gives you an idea. Overall, during the course of a clinical trial, many thousands of pages, many. It's a 10-year project that requires tons of documentation. So our dream is in the future, maybe we won't need so many documents because we can represent the trial in a more structured, electronic way. This is basically protocol digitization, something we believe very strongly in. And so maybe we can just submit electronically, the same way you submit your taxes electronically these days, right, to the IRS. You can submit your protocol electronically, the FDA receives it and is able to review using a similar system and things are much more efficient.

14:16That's exactly where my mind went. Okay, you're using AI, but you're still generating these hundreds and pages of documentation. At what point on the reviewer side are they then leveraging AI to gain the same efficiencies in a sense? Do you see that on the side of the FDA or other pieces? Yeah, funny you should ask. So recently, this is cut off the press last month. the FDA announced that they had this new AI tool called ELSA that they're using to analyze these protocol documents. And so we're using AI to generate them. They're using AI to summarize them and decode them and make inferences. And at some point, you won't need the documents anymore.

14:54You'll submit using some kind of structured protocol, all the electronic and so on. And again, not to say that takes humans out of the loop. It just means that humans have more powerful tools to summarize and to analyze and to really slice and dice the protocol to make sure that it's going to be okay. That's so great. I'm like literally Googling it now as we speak. Funny I use Google instead of chat TV. Yeah, for recent news, sometimes Google is better because GT is not trained on recent data. It's, yeah, anyway, I was at a conference last week where this was in DC, the DIA conference, and made a nice presentation there.

15:27But the FDA was all over the place. They were speaking in a lot of these sessions, talking about ELSA and other things they're doing. So they're really into this too, which is great to see. what so i have two young daughters so when you mentioned the name is elsa i immediately go to the the disney movies was there any meaning behind the elsa name i wonder i saw that it's in all caps i'm sure it's an acronym of some kind but i didn't take the trouble to learn what it actually means it's catchy though i like it yeah yeah so another thing that's interesting to me with your company faro hell you came to be before generative ai and the transformer architecture and chad gpt and all these things emerged, right?

16:06You were doing this in more of a kind of classical AI way. So two things, what does it look like? I'm always interested to talk to companies that were pre-ChatGPT, post-ChatGPT, if we make that like the seminal moment. And then what did that shift or pivot look like? And I got to imagine there was potentially some resistance from doing it in the classical way to leveraging large language models. And then And take me down that path of when that happened. Was there a moment you remember where you first encountered generative AI and you started to think what this could potentially unlock? Yeah, for sure.

16:47In 2023, I was in this mode of figuring out what am I going to do next? And all signs pointed to life sciences. Like everyone I talked to, I did a bit of consulting that was in the life sciences space, applying large language models. And it was just really a case of, it's quite obvious that this is the place where this is going to really take off. And then when I first started, when I was first introduced to Scott Chetam, our CEO and co-founder, and I started talking to him, it really came to life. And he just said to me, look, we need to bring AI into this product. So that's what I'm looking for you to do.

17:18And that was a really great challenge because it's looking at a more or less more traditional SaaS product and really figuring out all the different ways that AI can really augment its capabilities. And in addition to the document generation stuff, this is by way of example, right? So one of the key features that this product, that our FIRO study designer tool has, is the ability to provide analytic. So we can tell you very quickly, very early on in your trial's design process, hey, here's your patient burden. Here's how tough this is going to be on the patient, like in terms of blood draw, in terms of time spent.

17:48We'll tell you, hey, here's your cost and complexity. Here's what it's going to be like for sites. So you can really get some idea, like how feasible is this project going to be? Is this trial going to fly? Am I going to be able to recruit patients? What did similar trials like this actually end up doing? Did they end up having amendments or not? Those kind of things. And so when I was thinking about how to use AI, the obvious one, in addition to generating all these kind of hefty documents, was how do we provide better insights to the end user? How do we build on our existing advantage of providing analytics and say, okay, now we have AI, we can double down on the analytics and provide some really interesting insights into how your trial compares to all the tens of thousands, hundreds of thousands of trials that are out there in the world.

18:27We can use AI to analyze them and essentially give you some comparison, benchmark, and all this kind of interesting stuff that's super useful in terms of designing the trial. Now, if you check out our website, you'll see we published a paper with Merck about this. We published a huge pharmaceutical company, right? And we just studied, like, what is the impact of making design adjustments and optimizations early on in the trial's design process, right? So instead of incurring the cost, you foresee them through these analytics. And the answer was for a decent sized trial, you could save a hundred million dollars over the lifetime of the trial.

18:59Crazy amount of money just by making the right design choices, just by only including the assessments you actually need for this trial. So that was pretty remarkable. It was a really good validation of this strategy of focusing on the upfront design of the trial. And so what was that moment like? So you were brought in when the seminal moment was happening. Was there any hesitation in using the frontier models out there? Was there ever any consideration? Oh, we have to build our own proprietary thing. Because there's always this debate of, are you just an AI rapper? But I feel like in this context, there's the deep domain expertise, which is really where the value and the differentiation comes into play.

19:42And you're simply leveraging what is a tool, in a sense, in terms of the LLM. yeah i think it's one thing for sure over time the foundation models becoming more and more capable and they're ingesting more and more domain specific information and so on and so forth but when you really start looking at the actual process of creating clinical trials with a real sponsor with a drug company or a biotech company or what have you then there's just a lot of nuance there that's really detailed all those checklists i was talking to you about before to evaluate the generated content. We're fortunate in that we have a clinical sciences team.

20:20And in fact, our CEO, co-founder, God, is a clinical scientist. He worked at Novartis and Verily in the past, really top, top companies in this space. And so that helps a lot to have that domain expertise embedded in very deeply in the company and everything we build. And I think a lot of this is just direct experience. It's just like we have a bunch of companies that are really using this to optimize their clinical trials and so on and just getting that real world feedback and embedding the feedback into our product and that agentic architecture i told you about before definitely there's a lot of expertise built into that so i'm not super worried about being eclipsed by a foundation model or any of this kind of thing it's more a case of finding the best tools for the job and really keeping abreast of all the advances out there i know and propic and google and others are really super into this life sciences space and last week at the conference i had a pretty strong presence it was good to see them there so it's exciting there's a lot of change there's a lot of new cool things coming out every month and so just keeping abreast of it all is quite quite a challenge i think too this is a case where as the models get better your product gets better by as a byproduct of that which is like a great example of riding that wave in a sense instead of being eclipsed by it in a sense when we started looking at this it was i think 3.5 had just come out or something or that was the kind of standard cutting as gpt model and then when 4.0 came out a lot of the things that didn't work so well before we just retried and many of them actually worked a lot better and so it is a really cool thing to just not be on the static base to actually every time there's a new model that comes out we can look at it and say okay how can this help us be even better and that's exciting and also a little chaotic like the whole deep stick thing that happened late last year was super interesting like breaking everyone's assumptions around price and many other things.

22:09And so there's this arms race going on right between these big vendors that have the resources and wherewithal to actually produce these foundation models. And I think it benefits everybody really, just as long as we have the capability to easily adapt and try out the latest things and adopt the latest models if it turns out that they actually are better. And do you all use reasoning models in any particular applications? Or do you leverage kind of 4.0 as the base one? Any testing around some different reasoning models and leveraging some of those capabilities? Yeah, I mean, we're definitely looking at Anthropic and Gemini and many other models as they become available.

22:44We looked at DeepSeap as well, so the good news is for us, we have these evaluation criteria, so we know pretty quickly when a model works better for us by just running it against a standard test suite and looking at the results like how many tests passed. And so that's a great way of doing this because oftentimes it is a little bit whack-a-mole, like maybe this new model is better at some things that are generating some kinds of content. Maybe it's more natural in tone but maybe it doesn't quite have the same depth of research capabilities. And so there's just like a lot that we can look at there.

23:12I don't know why this thought's never popped in my head, but what you just said just triggered this new idea. It's almost like this new mechanism you should have as part of your process product, whatever it is, that you are able to validate and test new models as they come to be in a structured way, right? So it's not just willy-nilly and it becomes more efficient in terms of saying, here's the latest and greatest model. Let's put it through its paces in a sense to see how it performs. And you could actually put some tests around that to where it is a structured process. I've never thought about it in that sense.

23:50That's an interesting way to think about it. Yeah, quite like in the sense that we already have a battery of different general purpose tests. In fact, some of them are specific around, for instance, answering grad level science questions. This was a new one. If you look at the Stanford AI report, they tell this every year, and they really do look at how are these various models doing? How is AI in general doing as far as its performance on a range of different typically human tasks, like answering grad-level science questions? and in the most recent report was the 2025 report they have a couple new tests in there that that look at some new capabilities and they look at the over the past year like how how's ai progressed and the scary thing about this graph is that the gradient of each new chart is like steeper and steeper so in some in one case actually this grad level test i mentioned before like it went from being new essentially ai didn't do well at all to surpassing human performance in a year so that's really, wow, the chart really is getting steeper and steeper in terms of the acceleration of AI models to surface human performance.

24:55So that's really interesting. And so my point was that basically there are battery of these tests that we can run against that are being run against every single new model. But that said, it still, I think, is helpful to actually create your own test. So you can evaluate these models yourself and see for your particular use case, is a given model that seems to be doing really well in these general tests actually better for what you're doing. Exactly. Exactly. And that makes a lot of sense. So I'm curious, we've talked a lot about kind of clinical trials, that process. Go wider for me, stay in the healthcare domain.