In short
How Dr Josh Au Yeung moved from medicine into healthtech/AI, and what it takes to learn and apply AI in clinical settings. He connects evidence-generation in clinical pharmacology to digital health, then describes building ambient clinical documentation use cases and joining startup Tortoise AI.
Guest backgrounds
Josh Au Yeung is a neurology registrar (Kings) and clinical lead at Tortoise AI, a startup building ambient clinical documentation. He also hosts Dev and Doc with Prof Jelko, focused on AI for healthcare. Earlier: state-school background in North Wales, studied medicine at Newcastle, intercalated in chaos theory, and did neuroscience research including electrophysiology on brain tissue.
Key claims
It’s OK not to love all of medicine; breadth and research time matter. To truly understand AI, you must “get hands dirty” (coding/immersion). Clinical pharmacology’s evidence-trial mindset maps well to digital health. Networking and visibility can open industry doors.
Notable examples
Using AI named-entity recognition to structure EHR text (e.g., “atrial fibrillation” variants); tracking sodium valproate patients for safety follow-up; automating coding/audits for governance and billing; ambient scribing to draft post-consult letters for faster admin.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOMeet Dr. Josh Au Yeung
0:39 to 2:41
Dr. Josh Au Yeung shares his background and current roles in health tech.
“Hello and welcome back to How to Get into Health Tech with me, your host, Dr.”
Journey to Medicine
2:41 to 5:48
Josh discusses his childhood ambitions and the pressures of pursuing medicine.
“It was so bad they had to bring in a special headmaster to try and bring the grades up.”
Interests Beyond Medicine
5:48 to 7:57
Exploring Josh's interests in sports, technology, and mathematics during his youth.
University Experience in Newcastle
7:57 to 10:34
Josh recounts his university experience at Newcastle and its social environment.
Feeling Like an Outsider
10:34 to 14:00
Josh reflects on the challenges of being from a state school background at a prestigious university.
“So the first day in university, we sat in the lecture theatre and they had these voting devices.”
Challenges in Medical Learning
14:00 to 15:26
Explore the struggles of memorization and the challenges of medical education.
Choosing Neurology: A Personal Journey
15:26 to 18:08
Understanding the motivations behind choosing neurology as a specialty.
“And then obviously Oliver Sacks as well.”
The Intersection of AI and Neuroscience
18:08 to 19:16
Discuss how concepts from neuroscience relate to AI development.
“I did an intercalated degree in chaos theory.”
Navigating Medical School and Foundation Years
19:16 to 22:28
Insights into the emotional and academic challenges faced during medical training.
“But when stacked on top of each other, look what comes out of it.”
Exploring Extracurricular Opportunities in Medicine
22:28 to 24:12
The importance of seeking diverse experiences beyond traditional medical training.
Show all 21 chapters
Foundation Training and Specialty Decisions
24:12 to 28:00
An overview of the challenges in choosing specialty paths during foundation training.
Navigating Career Challenges in Medicine
28:00 to 31:30
Hear about the personal struggles and decisions faced during medical training.
Clinical Pharmacology and Research
31:30 to 36:10
Learn about the intersection of clinical pharmacology and research opportunities.
“Thomas's is a fantastic hospital best one I've worked in and And if it's good enough for Boris, it's good enough for anyone.”
The Role of AI in Health Tech
36:10 to 40:10
Discover how AI can transform healthcare through hands-on experience and learning.
“And I basically said, hey, look, I have academic time.”
Creating the Dev and Doc Podcast
40:10 to 42:01
Understand the motivation behind starting a podcast to bridge medical and tech knowledge.
“It was one of the, a lot of the real joys that I've had are working with people outside the core medical discipline.”
Navigating Health Tech as a Clinician
42:01 to 46:44
Learn the importance of communication and teamwork in health tech.
“Yeah, so I applied to that a year-long post and I got it and basically that led me then to fully focus.”
Transitioning from Academia to Industry
46:45 to 49:59
Explore the journey from academic AI fellowships to industry roles.
The Role of a Clinical Lead at Tortoise
50:00 to 54:29
Discover the multifaceted role of a clinical lead in a health tech startup.
“He's got his fingers in like a billion pies.”
Leadership Beyond the NHS
54:30 to 56:04
Understand the evolution of leadership styles from NHS to innovative environments.
“So how do we make this product into something that's sticky that doctors will use and understand?”
Leadership in Healthtech: A Personal Journey
56:04 to 56:50
Learn about Dr. Josh Au Yeung's insights on leadership and mentoring in healthtech.
Reflections and Closing Thoughts
56:51 to 57:21
Hear the hosts reflect on the conversation and share their appreciation.
“And who knew we would be discussing the great and little Orm from Laduno alongside large language models.”
Transcript
Automatic transcript. May contain errors.0:00Welcome to the Health Tech Podcast. Here we talk about everything healthcare and technology and I'm your host James Summeru. Hey everyone, you're listening to one of our special episodes, a mini-series hosted by Dr Keith Grimes called How to Get into Health Tech, where, you guessed it, Keith speaks to an incredible mix of guests who have each taken a different path into the space so you can learn how they did it and what they do now. This is all part of our mission to make careers in health tech far more accessible. So whatever your background, I hope you enjoy it. We'll be back with our usual episodes soon.
0:38And here is another superb conversation in that series. Hello and welcome back to How to Get into Health Tech with me, your host, Dr. Keith Grimes, digital health doctor and founder of Curistica. Thanks for joining again as we explore the different ways in which people get into this most wonderful of specialisms within medicine and technology, health tech. I started this up, as I've said in the previous podcast, if you listen, because I get asked all the time how you get into health tech. And I've spoken about it a great deal. But instead of boring you all with the same story, I thought I'd maybe try and speak to some people that I knew, friends and other colleagues who've made it into health tech, because there's lots of different ways in.
1:20And today I am delighted to have Dr. Josh Aoyoung, who is joining me to tell us how he got into health tech. So how are you doing this evening, sir? Hi, Keith. Yeah, doing great. It's good to be on the other side of the table. I interviewed you not too long ago, actually. Yeah, this is my revenge. Well, it's not revenge. This is nice and gentle. No, it's good. We're all doing podcasts these days, but Dev and Doc is a fan. We'll probably cover a little bit of that as we talk about this but we'll start I'll try and sort of keep to my sort of very loose structure on this and just start and say so Josh do you want to just introduce yourself to the audience and say a bit about yourself yeah so hi my name is Josh Joshua L Young is my full name I am a neurology registrar in Kings I'm also the clinical lead of Tortoise AI a startup doing ambient clinical documentation I also run a podcast called Dev and Doc, where I'm the doctor sitting with my developer friend slash colleague, Professor Jelko, and we talk all things about AI for healthcare.
2:23A fantastic podcast. And also thanks to Jelko, I was spinning up my own ambient scribe late last night as well. He put a great substack about that. Caution, a lot more to creating an ambient scribe than just putting together some stuff in collab. But we're going to come back to maybe some of that later on. Okay, great. Well, look, in this podcast series, I like to go back in time and ask you to start with, what did you want to be when you grow up or grew up? Oh, that's a tough one, actually. So going back to my childhood, you know, my first job that I actually wanted, it sounds crazy, but I was a kid and, you know, for special occasions or birthdays, sometimes we'd get some fast food like from mcdonald's and my brother's friend at that time was doing some part-time work at mcdonald's as the quote-unquote chef and he would tell us how amazing the job is because he could basically he cooked extra burgers by the end of the shift and then when they closed he got to take home those extra burgers and in my head i thought well this is the best job ever like i agree i i want to work at mcdonald's so that was that was me yeah um 10 years old but obviously as i got older um i started getting yeah like better grades i still wasn't too certain of which way to go but you know being from an asian family and I don't think many people know this, but I went to quite a poor state school, like one of the worst schools in North Wales at the time.
4:08It was so bad they had to bring in a special headmaster to try and bring the grades up. Crikey. So they loved it because, you know, I was one of the few students who was actually getting good grades and they wanted me to be on the prospectus. They wanted to say someone got into medicine and likewise my parents being Asian they thought oh Josh just book smart like let's just push him into medicine that makes sense there's prestige there's pay there's security um and yeah I kind of got pushed into it I I applied I got in uh the rest is history I guess yeah well you had that sort of double pressure I think there's a lot of people that go into medicine that feel the pressure of their parents or their family expectations to go in and they've got good exam results but to be put forward as a poster child for your own school and a school that's struggling is a bit of a bit of extra power behind that but I suppose before we get into the sort of university side of things you know you're book smart you're doing your your exams and everything like that but what were your kind of interests at the same time as well other than McDonald's of course yeah interests were two things i loved playing racket sports so i was playing a lot of tennis and badminton uh eventually i had to choose between one or the other and i chose badminton purely because it was indoors and it wasn't freezing so i could play in the winter north wales whilst it was cold outside exactly had some success of that so like won multiple like regional championships uh went up to basically like the national junior national level um competing a lot traveling a lot um yeah i really loved it actually uh for listeners i would highly recommend to try playing some badminton because it's super fun it's tactical it's psychological it's physical it's like but yeah one of my favorite things to do mental notes don't challenge josh to a game of badminton right enough though i'm sure you'd go easy so you're playing badminton but were there any other sort of interests as well you know trying to sort of find where you know was technology in there at all games were there um you know being i was the youngest of three uh children uh or boys so um we bonded through like playing video games we loved uh playing video games watching movies uh you know like i i got a taste of technology through that because i would you know look at different computer performances like graphics cards i kind of learned a bit about it peripherally um and you know i loved things like sci-fi games i i love things set in the future whether that's like star wars or star trek or you know whatever even or like books set in the future um yeah i i think i always wanted to to be i don't know i could always imagine myself in space i i guess after mcdonald's my second choice would have been an astronaut like i think that would have been amazing like it's still time still time yeah yeah there is um so that yeah i did love technology i also loved mathematics in school that was by far my best subject um so i really yeah i i don't know it's something about mathematical reasoning just made sense to me and i was doing yeah I was doing maths questions like a few years ahead so even in GCSE I was doing like A-level textbooks like there wasn't much resource I remember because of the school they didn't really have an extra teacher but they could give me a textbook to just kind of crack on with my own questions and stuff I did further maths for A-level loved it so I do think I mean we will get to that but yeah this is my base i've always seen myself as quite technical quite logical of a person like i love mathematical reasoning logic deduction um i wasn't so good at the softer subjects like chemistry or biology like things that required a lot of rote learning i always struggled with i could always reason my way through things but committing things to memory wasn't my strong suit yeah what's that was less of it well there you go and then here we are a few years later a neurologist and so so we've got some of the seeds in there as well so and i know what you mean about the mathematics side of things i loved and still love mathematics and there's something deeply deeply satisfying about the understanding and this and the certainty of the answers as well there's something within that when it happens um yeah so i think i think that it's not uncommon for those working in tech to feel the same but uh here you are in north wales you're the poster child you're being put forward and you're off to medical school where did you go i applied to a couple but um i ended up in newcastle i had a backup offer in cardiff but yeah i thought newcastle sounds great at that time i was looking reading different forums and i've never been to newcastle but i read that it was one of the best for medicine that it was cheap and it was a very fun kind of party capital yeah here we go so you're like i go busting out going to gin exactly what are the best decisions i've made actually i think you know i i i still i'm very closely involved with university king's college london like i see the experience students have in london compared to the northern experience and i have to say the northern experience is way better it's so much more fun so much more lively there's so much cheaper you live better you you can be more social uh more fresh air more nature yeah it's a it's a lovely it's a lovely city and um you know i know what you mean i mean like i i went to university in aberdeen as well and um uh you know university cities university towns where everything kind of gets slightly more concentrated really elevates the experience i think because you can you know you you throw in completely i i of course i didn't study in london but i suspect when there's so much else competing for your time and then the sort of pressures particularly now um with the cost of things i think it might make that wee bit tougher yeah it's it's really crazy and um so other than enjoying the big market and all the fun uh up in a newcastle when you were up there what was your time at university like i'll share my experience of the first day um because I do feel like there's not enough representation of people from working class or like state school backgrounds in university and for those who don't know you know Newcastle is actually very posh city despite being up north and you know the medical school is twinned with Durham University so we we used to joke we had a lot of Oxbridge rejects there so like from amazing backgrounds.
11:23So the first day in university, we sat in the lecture theatre and they had these voting devices. So we could, everyone was given one, we could vote on different questions. And there were some basic questions like, are you male or female? Blah, blah, blah. One of the biggest questions I remember was did you come from a state school, a grammar school, or a private school? And I remember it was overwhelmingly private in grammar school there's less than 10 state school and you know when i looked at that graph in front of me i don't know it made me feel i don't know that a message came into my head like okay you're not one of like you're not one of them or like it's gonna be you're an outlier here um and you could see that because somehow even though it was everyone's first day there were people that were like buddies and mates and i thought to myself wait how do they know each other and you know transpired that obviously they all went to the same grammar school like leeds grammar uh eton harrow whatever right um and yeah i i think at that point it kind of made me feel a slightly like an outsider um and maybe that set the tone for the rest of the years it was it was really interesting and you know looking back you know do you know how much the tuition is for private schools like Eton or Harrow I don't know what they are I know that my brothers one of my brothers has some children at a private school two of them and I know it's quite why eye watering for them but that's up in edinburgh wise so i don't know i don't i don't know what it is down in um eton but i'm sure it's uh not cheap yeah i might be wrong but i think it's between like 15 and 30k per semester yikes yeah probably higher nowadays to be honest you know i thought to myself wait so these people have paid you know probably upwards of hundreds of thousands and why are they in the same lecture theater as me right I've paid nothing it was free so I don't know immediately I felt yeah there's just some strange disconnect there and then I could see as the lectures progressed like I enjoyed it but I also saw that everyone had very similar studying styles and thinking styles uh so you know everyone had all these ways to remember things which were taught to them by the tutors no doubt like yeah you know uh mnemonics or like using different techniques to remember things uh like they're very good at learning things that that's what i remembered and it didn't come naturally to me whatsoever like i felt like i was doing a subject where i was playing from the back foot in some ways like you know my strength was like reasoning deduction mathematics but in medicine i always felt that medicine is almost you're committing a string of evolutionary coincidences to memory and you know i still remember loop of henley like why why is it displaced yeah why and what for and the answer is there's no reason right it's just evolution and this is this worked um and you know evolution doesn't necessarily choose what's most efficient or effective it chooses what works and what survives so like i i couldn't anchor these memories or these ideas to anything it's just i just felt like i was learning it for the sake of it yeah that's not terribly satisfying for a person that you know you've already described your interest in you know what you like about maths that kind of understanding and then being able to build upon those principles into greater and greater complexity which if you're rote learning is it's not really quite the same is it yeah but even so that's that's a tricksy way to start your your time at uh university and um but made it through university you did and you know when people go into i don't think anyone who's come into this podcast has said i wanted to be a doctor and i wanted to be this kind of doctor but it's always interesting to know which way look why is it you've ended up with neurology i've got some thoughts hearing about what you've just said but but you know talk me through that i guess after mcdonald's after astronaut my third choice was something to do with the brain um so as i was thinking it has to be neuroscience because at least this is slightly more logical than the other ones um and you know from the lectures i thought okay i can kind of make sense of this you learn the system you learn the central nervous system you learn the anatomy and if this thing goes wrong then that happens if this combination of things goes wrong that could happen like i loved how you could actually just find the diagnosis just by watching someone walk right like i just thought that was the coolest thing ever um and then there was the other side of the brain which i loved which was the psychological psychiatric side uh i don't know just out of interest of reading about the brain uh i came across different books like uh from carl jung and sigmund freud uh and i found it really interesting i i love carl jung's biography i still remember reading it um like I felt like they all lived such fascinating lives.
17:09And then obviously Oliver Sacks as well. Oh, of course. He was also a gym buff. I don't know if you know this, but he won squatting competitions. Yeah, he was interesting all round. I know he was into his motorbikes and all sorts of strange things. But yeah, the man who mistook his wife for a hat, an anthropologist on Mars, Island of the Colorblind, all these great books. Yeah, I think these cases, gave me a sense of bewilderment and and it's like wonder right it's it's it's i think it's the most beautiful thing from medicine is like the beauty of these crazy cases that really make you think and marvel at the human race and like what how things can go wrong in such crazy ways right so i think that's what drew me to there so like partly the mathematics side but also kind of the wonderment and also the i think maybe i was young i kind of love the consciousness side as well like i was trying to be a bit of a philosopher like unsuccessfully but but i love just thinking about that no thinking about thinking there is nothing better actually funnily enough at the time of recording i was reading there was an interesting paper google pumps out lots of papers about ai and the like and there was a fascinating one about cellular automata um which is something that i was very interested in back when i was at medical school in the early 90s and was very heavily into things like chaos theory and the like.
18:34I did an intercalated degree in chaos theory. And so for a while, I spent a lot of time studying cellular automata, these sort of very, very simple parallel rules that lead to very, very complex outcomes. In truth, I'm still interested. But yes, Google has started to put out a paper or some studies about how they've managed to have these kind of synthetic neural nets or these sort of synthetic sort of like logic gates inside the cellular automata that can actually cause them to start behaving in slightly more complex ways. And yeah, there's something I find very satisfying about, I can understand one small bit and then I can understand other pieces coming together.
19:18But when stacked on top of each other, look what comes out of it. Maybe that's why I like, I particularly love things like large language models and so on. the idea that you know at its base you can wrap your head around like the unit but when you scale it beyond comprehension look what happens i mean look at evolution right there's a simple mechanism for that like survival and through that we can grow the most complex of organisms and you're completely right like now there's emerging papers and thoughts where uh complex ai models and we'll get to ai i'm sure but complex ai models like large language models etc uh there's a school of thought which says like we shouldn't think of these as you're building a model you should think of it as you're almost growing a model from an evolutionary standpoint it's very similar right you take an optimizer function so some so some reason for you to train the model and for large language models that's predicting the next word accurately and through something so simple you can grow something so intelligent like it's very similar to evolution that's why i always say like AI and neuroscience are intertwined you know this artificial and biological neurons these are very very similar and definitely AI was inspired by neuroscience for for much of its inception so it seems like a natural fit that you end up working where you are right now but but you're going towards you're going towards well I mean so you're at medical school and you're interested in this but you know when you did your foundation years and everything like that at that point where you're like this is it neurology's for me or were you still unsure so throughout med school there were at least two or three times where i wanted to quit um i remember having quite long conversations with my parents over the phone i don't think i was like i loved the neuroscience part but a lot of the other stuff i didn't love and i don't know i was looking around everyone seems so passionate about learning medicine and that i just felt like an outsider because i didn't have that passion people were going for study groups they were they love i mean i'm glad i finished med school because i remember in every like peri exam period you could just feel the nervousness of medical students and they would almost i hated that feeling they would almost test each other to try and get a one-up on each other i don't know maybe i was in bad circles but i felt like i went to a lot of different circles but most of them shared this attribute like everyone was competitive right and very competitive good marks but it's not competitive in a nice way like people would deliberately try and not share their notes with you or do things to not help you or even bring you down um yeah it wasn't the most conducive learning environment but but anyway like i think my whole point in all this is that i want the listeners to know that it's okay that you don't love medicine or you love parts of it like that's completely okay and undisceptible and probably normal um and yeah just to embrace that like i what i ended up doing instead of studying so much i wanted to explore different things right so i took a lot of extracurricular time to just do more research i sat in a lot of different labs uh just to see what they're doing to learn from them uh i sat in a mitochondrial lab uh there's a professor called doug turnbull in newcastle which i think he's now retired but he did a lot of cool mitochondrial stuff i did a lot of kind of sat in his clinics did family tree mappings um i also intercalated a master's degree in neuroscience where i looked at uh kind of lab-based studies electrophysiological studies where i would get brain tissue from the neurosurgeons from from from tumor resections and then it's quite fun you put it on ice and then you have to run it to the lab and then you slice it super thin like microns thin and then you poke it with electrodes and try and get it to have seizures and then you try and abort those seizures with different drugs all right so i was doing that what else was i doing yeah i was doing a lot of random small projects here and there just trying i think i want i knew that medicine wasn't the entire thing in my life so i think i started seeking out different opportunities and you know we'll come to this later but i think this is a super important trait to have and something which was overlooked by a lot of my medical school colleagues at the time because i think these experiences really widen your view and your vision and your thinking i agree and we'll come to that but yeah like they were so optimized on grades whereas I just wanted a full experience yeah optimized on grades or optimized on path as well you know like the decisions of which where to expend your energies which lectures to go to which lectures to maybe focus elsewhere on which consultants to spend time with you know where to put those efforts and you know I used to look because I I think the same as you I wasn't entirely sure at all I just wanted to use technology in some space and I was just digging around for where that could happen and um and unsurprisingly ended up as a generalist because i'm still unsure but yeah you're right it's like as you start looking around and casting around for me my the the chance encounter in some regards was working in clinical governance and you know and and safety and things which on the outside seem very very dry but there's something again really deeply satisfying about saying something has gone wrong why has it gone wrong you know that unpicking and then the work to try and stop that happening it has the it has that same sort of satisfying nature which i bumped into almost by accident because i was i said well you can use the technology stuff if you also do clinical governance i said all right but yeah yeah so keeping your mind open like serendipity is really important yeah i think embrace your oddities um and then you know i started f1 foundation training in manchester i chose manchester just because it was closer to llandidno my hometown um lovely town sorry i didn't know from llandidno it's lovely with the great orm and the little orm yes yes yes oh my gosh yeah so yeah so for listeners this is i rarely meet people that know what llandidno is or where it is it's very nice place down south it's a rural town in north wales seaside town uh pretty nice good fish and chips some nature a nice promenade and a big at a big rock yeah two yeah wedged between two big rocks called the little orm and great orm um yeah the distribution of ages like either young children or kind of elderly retire retirees but but being in manchester was a bit closer to home and you're right it wouldn't have taken so long to get back and then f1 f2 i was pretty much focused on um just getting clinical experience uh building up oh i i tried to get specialties that i wanted i i couldn't get neurology but i got stroke which is a close um i got psychiatry um so i that kind of covered the two sides of the brain which which i wanted to do so i was quite happy and you know i would always say look yeah it's hard it's hard to balance you know whether location is more important or whether specialty is more important i think i'll leave that up to the listeners i suppose you can you can it's difficult it's difficult enough to know exactly where you want to go but even if you do it's then very very difficult to choose exactly the right path and i think people worry a great deal about it particularly when i speak to people earlier in their career they they have this great fear that they're going to misstep and therefore fall behind or something we've already talked about the importance of that breadth of experience too but sometimes all you can do is choose things that are roughly pointed in the right the vectors are more closely aligned as opposed to being spread out and now you have no choice right like um after you finish final year now foundation placements are randomized it's no longer points based so yeah i know people that got places like ireland or west wales and then just declined their number and then had took a gap year in the attempts to to get bumped up the list uh so that's quite depressing that is quite sad as well particularly when people are maybe i mean you're still reasonably young at that point maybe for many folk and but but people may have family commitments or relationship commitments or anything and the and particularly if there's maybe more than one in a partnership and you get sent to the other sides of the country or whatever you know they're not making it terribly easy are they i can rant all day about this but but i think that yeah so the point was i got some specialty experience i did psychiatry and i realized that you know it wasn't for me just um just because i found it really mentally draining like seeing all these depressed patients bipolar schizophrenia um yeah i i just didn't feel like it was the specialty for me so i turned my sights on stroke and neurology and then this is when i applied for core medical training and at this point i kind of knew like i wanted to do more research because i enjoyed research in university and i also it's going back to the idea that medicine isn't everything i didn't want to just do clinical medicine i wanted to do research and other things as well so i applied to loads of academic clinical fellowships and for the listeners that don't know you know these are the posts that where you have an academic component and you have a clinical component and your academic time is protected normally it's between 25 to 33 percent and this is huge right because if when i applied the extended core medical training to imt so it's three years now and if you get nine months to one year doing research that's pretty big yeah like that changes your training completely um and not many people know this but i applied to a hell of a lot of jobs at that time acfs were independent applications they still might be now but i think i remember i applied to about 15 jobs like 15 i think that's very rare for a doctor usually just go for the main you know the core one the whatever like gp or some backups but 15 academic clinical foundation posts wow i chose obviously the neurology ones i chose stroke ones and then i chose ones which were peripheral to that so there was clinical pharmacology in cardiovascular and stroke disease yeah because i also thought oh clinical trials sound really fun as well um i ended up getting second like short shortlisted for about half of them um and then came second or third for most of them or actually all of them so at that point i was super disappointed because i remember i had an amazing interview on one of them and the professor even said to me it's like oh i wish you came to me earlier uh you know as i was walking i was like wait why did he say that but yeah that's a bit of a telling thing isn't it if only you did before the last person you know i don't want to point fingers there is there may be some nepotism in academia maybe there's well there's maybe more there's maybe more at play than just what's on the piece of paper uh yeah you or you you certainly you certainly were on the rough end of something right now 15 that's quite a few but you know again we're we're a happy place now josh uh so this is your wilderness years so so when things change but luck happened so someone dropped out i don't know why i don't know what who i don't know when they dropped out so i ended up getting um our second place i ended up getting the academic clinical fellowship in St.
31:55Thomas's in London in clinical pharmacology doing um doing basically clinical trials drugs all that stuff um and it was yeah I think that was probably the first part of my journey where I felt okay you've really got lucky here but you've got started getting your ducks in a row because this is you know St. Thomas's is a fantastic hospital best one I've worked in and And if it's good enough for Boris, it's good enough for anyone. For listeners unaware of this, this is Boris Johnson, our former prime minister, who I think had his care in there. Is that correct? Yeah. I don't know if I should say this, but there is a funny story where the night Boris Johnson came to St.
32:45Thomas's, I was working in the ITU in COVID. so he was supposed to come to me you're there getting ready to receive this man yeah me and an imt1 he was uh basically seconded to intensive care you there so i was getting so excited and i remember the nurses were so excited as well they're like oh my gosh like uh boris johnson is coming um obviously then they i think they realized okay josh is just like at shro let's maybe let's wait till the morning yeah let's get the consultant the consultant actually answers the phone call at that point and comes in yeah well yeah i mean yeah yeah that's exactly what happened but of course but uh but happen but a fortunate result all around maybe a fortunate result for you as well but uh but yeah you're atst thomas is doing all this good stuff now this is uh you're getting in there and um and actually the funny thing you're saying about clinical pharmacology there's there's quite a lot of parallels between pharmaceutical medicine and digital health as well particularly in the evidence-based trial structure and everything do you think would you say that's right for sure i mean that's i mean what are you doing right when a drug or any intervention whether it's digital whether it's some kind of physiotherapy or talking therapy what are you trying to do you're trying to generate evidence right you're practicing evidence-based medicine and you're trying to generate the evidence to accept or reject the hypothesis that this intervention is useful or efficacious um and yeah there are many parallels and that maybe that's partly what drew me to clinical pharmacology as well because i thought you can make a big impact here and there's maybe some stats involved some maths my research project was initially meant to be a lab-based project but because of covid all the lab-based stuff basically was cancelled i started helping out with the covid vaccine trials so i helped with the astrazeneca trial the first um yeah the first vaccine trial in london the phase three phase four so i was working on consenting patients going through administering the jobs etc etc but i also took this time i saw this as an opportunity right like okay the lab-based research is gone but this is an opportunity because you can see this this disease that's coming to the world that no one knows about it's a mystery disease and this is fascinating and no one will ever get this i don't think it not in my lifetime anyway right you can see in real time what happens in the world when an unknown disease comes in into play and you can see the power of data and the power of stats that that was the lesson for me what was the most useful things right it was the data banks during covid it was the platform trials all of that is just data data data collecting data processing data and running data crunching numbers and maybe a bit of artificial intelligence or machine learning if you want so i thought this this is the future right So I took this time, I reached out to a research group in King's called Cogstack, led by Professor James Teo and Professor Richard Dobson of UCL.
36:11And I basically said, hey, look, I have academic time. My lab stuff's cancelled. Can I just sit with you, learn from you, sit with your team and see what you do? and basically I taught myself over the course of two or three months with the help of the developers. There was a guy called Anthony Shek who's one of the research leads there and data scientists and he taught me a lot actually. I used to sit with him. I used to just do courses, AI courses, data camp, go on YouTube, watch courses and then I'd sit with Anthony and basically work on different use cases try and code something try and spin something up and i think that was actually the most important part of this educational journey like for those who are trying to get into health tech you'll understand that the barrier to learning ai and machine learning and health tech is very high like i don't think it's easy to do if you're working full-time like i i believe i could only do it because I had time research time to just sit there and actually learn it's I think doing it you have to be super diligent to do it outside of hours well I think uh what was it it was uh was it William Osler who said you know to study medicine from books is you know or just to start to practice medicine without studying books is to set to sail without maps but to study medicine from books alone is to never set to see at all and I suppose it's a little bit like that with um And with AI as well, I spend so much time trying to get people to use it because it can be hard to wrap your head around it.
37:51You can get kind of close, but there is nothing, nothing that beats just getting in there, using it and feeling when things work, when it doesn't work. Because it's hard to, sometimes it is just hard to get to understand exactly why it's doing something. you can understand the principles you can get close to it but there's an element of utilizing all those other soft skills which i suppose you uniquely come from the fact that we're humans that are used to dealing with other intelligent creatures as well and uh and there's an element of recognizing what is intelligence-like behavior that comes in too but yeah you've got to get in there yeah completely and you know you we can talk about that i'm sure you've interviewed different people in different paths but i firmly believe that if you're working in ai and technology you have to get some experience like i like i hope i don't offend anyone but i don't believe you can truly know ai and get immersed in it without actually at least getting your hands dirty doing a bit of coding understanding a bit of coding because i know a lot of people talk peripherally like there are people who in policy there are people i don't know who who are digital champions or i don't know like it's very hard that those people know what to say because they've heard it enough times but to fully grasp it i do believe you need to actually do it yourself i agree i when i was working at babylon and there was a lot of work we did we weren't using you know the work that we're doing was on probabilistic models and some machine learning and um i remember a lot of the work the doctors did was very very repetitive you know like coding data sifting through spreadsheets, analyzing outputs, and so on.
39:32And it was irritating and painful. But there is something that comes from that. And I think it's the same with coding as well, is that commitment of time that then subsequently doesn't work. And there's something about the discomfort of that, that forces you to optimize your own function towards getting it right, that cannot happen just by learning the words and putting them in the right order, like a large language model. You have to have that kind of reinforcement, about it and some of that is uncomfortable and it does take time um but within that you then you then know where to apply your efforts exactly so i think listeners if if you're listening yeah i would urge you to maybe get some experience join a research group and that's why i started my podcast i mean we can get to that maybe now let's talk about it now yeah it's good time i thought the content that i was receiving as a doctor was either super technical there are loads of technical tutorials at that time it was jargon i didn't understand or there was like really high level courses i think like i did one and there's like a stanford six-week course but it's super high level and by the time you finish it you still don't really know what's happening it's so high level and so you know the buzzwords but i didn't fully understand anything by the end of it uh so I wanted to make a resource that was the middle ground and that's what inspired Dev and Doc, so developer and doctor Zhaoko is my colleague working in Kings he's now a professor in UCL he builds large language models, multi-modal models does AI research for a living I obviously offer the clinical perspective and also have AI expertise as well especially in implementation and bias understanding the data and yeah we found that the conversations that we had to unite both sides were usually very fruitful we wanted to share this on a wider platform yeah yeah and it is it's a fantastic podcast and yeah there's something about experts from two sides coming together to sort of find that third or intermediate space as well is really good.
41:45It was one of the, a lot of the real joys that I've had are working with people outside the core medical discipline. I bring, you know, a certain background and a certain kind of understanding and starting to learn their language and understand when my input was important, when their input was important and finding that balance. I think that's a really important thing that I've said on this podcast before, is that if you want to, if you're a clinician and you want to work in health tech it's principally about learning how to be more than any other time a real team player and understanding what the limits of your competence are and you bring some important things and you have to be very good at communicating them but you also need to be very very good at knowing when someone else needs to be running the show exactly and i think during this academic time i i i realized as i learned more i realized how little i knew right typical thing um and that's what drove me to then apply to a one-year AI fellowship with the team, with Professor Theo in King's College Hospital and St.
42:46Thomas'. Yeah, so I applied to that a year-long post and I got it and basically that led me then to fully focus. So my role there was AI clinical fellow, whatever that means but what i actually did was build use cases right so uh gelco would build ai models for whatever um named entity recognition that basically just means like finding terms in free text which you can then structure so like a simple example would be you know a doctor can write atrial fibrillation in a million in a million ways af fast af paf AFib, right? So you train the model to learn the nuances of language and then you can use the electronic health records in Kings and St.
43:38Thomas's which span over 20 years, over tens of millions of patients. Imagine the power you can have if you can structure all those records which is not routinely coded and you can say oh yeah, I can see this many people have had atrial fibrillation and then you can do all kinds of fun data stuff, right? okay af and hypertension in covid do these patients do better or worse you can make match controls you can do all kinds of use cases you know another use case was uh sodium valprate as you may know uh came with a kind of drug warning recently that's right yeah it's fought to now um affect the fertility of uh of young men and women who take it although now the literature is going back and forth a bit but regardless you can use this ai to track down patients that have taken sodium valproate because the prescription is not always tied to them in the electronic health record right the gp may be prescribing it they may get it from someone somewhere else they'll be visiting so with ai you can basically track down these patients so we use this model to to basically find these patients and reach out to them and say hey are you're still taking this we need we need to discuss the pros and cons of this um i spent a year there building about 40 of these use cases you know we can spend all day talking about it but these these cases ranged from simple audits to uh governance issues to big research projects uh to a lot of cost saving stuff so like simple stuff right like some departments they may not code certain procedures.
45:22For example, ENT, they might do the scoping clinic, but they don't code it. If they do, they get an extra 50 quid. So with an AI to do that in an automated fashion, you can actually rack up quite a bit of income for interest. Income for the department, yeah. As a GP, I spent a lot of time doing that manually and trying to code patients to make sure all the different coif lists and everything were up to date. And of course, now it's a very popular use case for generative AI to attach codes for the purposes of billing and items of service and so on. Things that can be very, very hard for a person to do, but can be done at scale and with a degree of accuracy by AI.
46:03So it all helps. So you're doing all this academic work, I suppose, as we sort of come towards the end. You're working at Tortoise now. Of course, with Dom, who has a background in pharma medicine as well does he not he's a you know he brings a lot of that kind of pharmaceutical trial rigor to the work that you're doing there so you're doing the cog stack stuff and some so so this is maybe I was like this is great this is great how did he get into tortoise how did he get this so so what was what was the bit where you made that change from that kind of academic research position or that the academic fellowship into the world of industry i for better or worse applied to neurology training and i got my number so i basically had to leave cog stack um well i continued there part-time um so i basically took a less than full-time role okay uh in clinical training so four days a week with full-on core rota and then one day a week i'd do the cog stack stuff uh but then i think that became more and more difficult code so cogstack basically said look probably better you just focus on something else or you know one day a week you can't really contribute that much no um so that's when i actually thought hey i've got these skills and i've started posting a few of the papers and the things i've been doing on social like writing blogs and posting on linkedin and companies are reaching out to me they said hey your expertise is really interesting um can you help us like can you hop onto a call and have a chat i started chatting to lots of different companies started consulting on the side um which was completely new thing to me i didn't know you could earn money just by giving your knowledge to people yeah i started a company doing that yeah i mean yeah exactly a great company as well so um so i stayed less than full time and then as consulting consulting one of the most standout companies was called maggie health i don't know if you know them but it's a predominantly a croatian company but they're now also expanding in the uk they use large language model chat bots to optimize your cardiovascular health so basically simple things like checking your blood pressure prompting you to take your medication just chatting keeping staving off loneliness checking on your mood your stress levels um yeah i really loved that i became the chief medical officer uh fractionally so yeah one one to two days a week i'll do that i was loving it uh and then i basically thought you know i want to take the next step and i'm looking around like what companies are there that i would want to be a part of and at that time while i still am living and breathing large language models like i love this part of ai and i know ambient scribes i spoke about in my podcast like this is in my opinion one of the next big things and i thought well you might as well back a horse right instead of just talking about it or or the thing i mean feel free to push back on to me on this but the thing of consulting is i felt like i was always i was there but i was not really on the team like i felt like i was always advising from the sidelines yeah i didn't have any skin in the game yeah i felt like with consulting i was always advising from the sideline like i felt like i was helping the team but i wasn't in the team and for some that's fine and that's exciting you still got to feel the rush without maybe as much risk but i think i wanted more risk and i wanted to be immersed in it so i looked around i was like well ambient scribes what companies are there and they came across tortoise and i thought wow this seems like a good company it's very science focused um and it's a small startup and it's based in london perfect for me right so i basically just sent a code email to dom uh i think it helped that he kind of knew me peripherally from my network and linkedin yeah i sent a code email and i said hey uh i know what you guys are doing uh i really like the idea and the products that you're building i think that i can really help with this because you know this is what i've been living and breathing for a long long time you know the crazy thing is when i went there for an interview uh the developers now told me but when they saw me they're like shit is that um so i shouldn't swear but that's okay you can swear away they're like crap is that uh doc from dev and doc oh fame at last so it turns out that they were working on different projects and and one of them i covered on my podcast and they shared it within their group um and it also helped that i did an ambient episode and i gave them a shout out yeah and they basically caught wind of that like someone reached out to them from that and said hey we heard you on the podcast yeah so um i think for the listeners this is where the point is like building a network is super helpful and you know you don't have to do a podcast but just like getting your name out there getting your reach out there whether it's on linkedin or in person in conferences I mean, Keith is a wizard at this, right?
51:30He's got his fingers in like a billion pies. But I think that is something that you have to learn and something you never learn as much as, like something you don't have to learn if you're a doctor, right? You can go through the national training applications. You don't have to network and that's good and bad, like it's a meritocracy, supposedly. But anyway, this is one of the side points. So yeah, I reached out to them. we had a lot of talks had an interview and they basically said look just instead of doing like part-time which i initially offered they said just join full-time and actually own this yep and i said okay like i'll join if you give me you know i but i there'll be no point if i join as someone who is just working like i have my opinions i have my ideas i'll only join in a if i can lead and yeah that's happened so i've joined as the clinical lead leading the clinical team uh love it um so maybe if i just tell the listeners a bit about tortoise as a company uh so it's a it's a startup in london um so they raised i think three or four million with Cosla Ventures.
52:46There's about 15 to 20 people working in Tortoise, and the mission is basically to build a co-pilot for doctors to eliminate human error. The current iteration of that is an ambient clinical documentation system. You may have seen these already. This is the next big pledge that the NHS has said to invest in. So these models basically listen to a consultation between you, the clinician and the patient and instead of traditionally where you after the consult you dictate a letter or you write a letter it just drafts a letter for you for you to edit and check so the idea is instead of spending I don't know like half an hour to 45 minutes doing admin after a secondary care consultation for me I can just spend maybe 10 minutes or to 15 minutes editing the letter itself um and you know i think it's a great use case and it's super fun because you know i talk about this on linkedin but like the job is of a clinical lead is super diverse and super fun like i always say there's four pillars to a clinical lead and these pillars may be of different heights in different companies but you know you've got the research side which you know I love machine learning research, my bread and butter.
54:11You've obviously got the clinical side. So that's clinical use cases, regulation, et cetera. You've got clinical engineering. So this is sitting with engineers to build models that matter. Basically, my whole thing with Dev and Doc. And then there's the product side, right? So how do we make this product into something that's sticky that doctors will use and understand? because it's like Keith said earlier, you can build the world's best AI model, but unless you get workplace buy-in, unless you get education, unless you get clinicians actually enjoying the interface and using it, it would amount to nothing.
54:54So all four of those things are crucial, and obviously the regulation bits as well. So super fun. I get to wear many different hats in my job, sit with many different people. it's honestly like one of the most fulfilling jobs I've had for a long while yeah and you could be just saying how I felt when I was at Babylon 2 it's that mixture it's all those things there is one more pillar that you may be putting there as well that you will undoubtedly get and that is leading and managing teams which is you know the one ring to rule them all kind of thing and in there but yes you're in that now and I'm delighted for you doing such great work and the team had taught us and the like can I just add to that to that like on leadership I feel like we're taught to lead in the NHS in a very specific way um you know we're taught to lead in the way that I can write in the white box of my interview oh yeah I led to the medical team but it's very task specific right oh yeah I led the team for an arrest I told this person to do that i delegated like it's a very task specific kind of leadership it's like okay i've got a million jobs can you do this can you do this come back to me and like that is a form of leadership but it took leaving the nhs to learn what i see as a more rounded type of leadership right a leadership where you're leading with your vision and ideas and people are following because of your ideas and vision and you know you can actually look out for people below you and they will look out for you as well like i feel like i'm actually mentoring people and we're actually building towards a goal like it's a very goal and vision driven type of leadership which i think you know for me this is how my brain works and it's it's amazing yeah well you've been unlocked It's wonderful to hear.
56:50And I'm sure everyone listening to this will be feeling equally charged up. Look, Josh, thank you so much for that. It's been really, really interesting. And who knew we would be discussing the great and little Orm from Laduno alongside large language models. So you don't often get those things together. I wonder what the likelihood of predicting that next token would be. Maybe it's slightly higher after today. Yeah, non-zero, non-zero. Non-zero, yeah. Fantastic. Thank you very, very much. and thanks to everyone for listening as well to how to get into health tech and we will see you again next time.
57:24Thanks Keith, it's been a pleasure. Hey everyone, thanks for listening and making it all the way to the end of this episode. Remember to subscribe, rate us and leave a review and you can head to the description of this episode to follow me on all of my social media so you don't miss out on any of the latest health tech content.
From the publisher
On the fifth episode of our "How to Get Into Healthtech" mini-series, Curistica's Dr Keith Grimes is joined by Dr Josh Au Yeung. Passionate about both biological and artificial neurons, Josh is a neurology registrar at King's College Hospital NHS Foundation Trust and a clinical lead at TORTUS AI. He also hosts the Dev&Doc podcast, where he brings together doctors and developers to unlock the potential of AI in healthcare.
Connect with Josh: https://www.linkedin.com/in/dr-joshua-auyeung/
Get in touch with Keith: https://www.linkedin.com/in/drkeithgrimes/
Apply to be a guest: www.thehealthtechpodcast.com
Subscribe to Healthtech Pigeon 🐦: www.healthtechpigeon.com
Learn more about SomX for your healthtech company at somx.health

