#453: Why "can AI replace doctors?" is the wrong question. Shravan Nageswaran, Atman Labs

47:56So that's kind of one spiritual aspect of company building and team building, that the way you actually construct, especially for a company that's inventing something net new from first principles, and it has to really be okay, you know, staying out of the public eye for a while and be okay, you know, when most of Silicon Valley is saying, oh, LLMs are going to be able to do all of these things and not falling for like that fear of missing out and being really high convicted in the path that we're taking. The spiritual undertaking that guides how we work as a company is personal legend alignment.

48:32And everyone in this team from the founders to the new team members that we've hired has some degree of personal legend alignment. And that is the true incentive. And you have this environment of working where it doesn't really feel stressed, but you have complete activation every day because you're confident that, you know, the way in which the company is evolving is going to require you to evolve in a way that you know that that's what you want for your life. But you also have confidence that everyone else on the team recognizes what your personal legend is. And they know that by being able to support you and your personal legend, that will in turn help their personal legend as well.

49:09And so you have this, like, degree of nervous system safety. And so I know I'm going between, like, very abstract ideas and very tangible ideas but and and it might be a bit of a tangent um but yeah to your point on spirituality no i love it and even even that final phrase nervous system safety i think what what you'll people will find a language for this right they'll just go oh okay you're just aligning incentives you're aligning motives you're aligning purpose and all of that is true like yes but i think the way that you're explaining it is that you've got this kind of understanding if there's something beyond the physical there's a there's a metaphysical layer to this that you're trying to a line which is deeper than perhaps what even we have language for i think that's just a very philosophical way of just approaching the world and i think it's a nice way of approaching the world in that i think people like yourself are aware of the unknown there are unknown unknowns that we are going to discover and i think that's but that's a basic fact like we we don't know what consciousness is we don't we don't have a framework or a language for that but we do know that on some level of the universe x plus y equals life we don't know what we don't know what that is even so and even if you go bigger than that like the whole thing is just ludicrous that we're just in this like vast open space like on a rock orbiting a like nuclear reaction it's all very bizarre isn't it but i think an understanding and appreciation of that will give you a way of aligning this um that frankly works for you yeah i mean one other framework that we have this idea of thriving right is um how to balance if we want to expand and have the most um fulfilled conscious experience, it comes from balancing two dimensions.

50:46The first is nervous system safety. And the second is surprise, basically being able to, you know, rewire our, um, our like neural compositions and discover new things about the world. Right. Because as you said, there's so much of the world that we don't know. So really our pursuit should be, you know, through creation, through discovery, like being able to expand our knowledge of the world. And some of that can be done through creation. Like self-expression is not just a medium of expressing yourself, but it's also a medium of being able to like discover new things in the world. And so when we think about even our internal environments and who we want to hire, like can they help create surprise and do so in an environment that like basically enhances the nervous system safety of everyone on the team?

51:34Can they create surprise in the direction that aligns with which the company needs to go and create a truly exponential impact. And if that impact is not surprising, then they're probably not a fit. And I respect that that's probably more of a factor in an early stage where you're building something from zero to one versus like a public company where surprise is actually jarring. But kind of at the stage where we're working at, you know, if you're able to balance between surprise and nervous system safety, you have a truly compounding interaction with the world and with others and that's what we optimize for for our internal um for for our internal dynamic and it allows us to invent um things and do so in a way that doesn't feel stressful and doesn't actually suppress um the things that we can create and being able to actually think about things from first principles with a very clear mind um that's something we prioritize awesome i realize we haven't we haven't actually talked about the text we do need to do that on the health tech podcast because it is fascinating um i've just indulged myself for a bit too long on this um on topics that i enjoy that perhaps the audience might not be so keen on so let's talk about the tech and if we've got time at the end i want to ask you what you've learned from sam altman but um from working with him um but let's let's talk about the health tech now so um the technology required the sophistication required on the level that you're building to emulate doctors that messy complex history taking that i you know just mentioned a psychiatric history in an inpatient with an acute problem physically going on like it's tough right but what i've learned in what i just wrote last week for pigeon insider this week for pigeon insider is is that there are ways of using llms within other frameworks and i mentioned in the intro you know you're moderating on the way and you're moderating on the way out and that's kind of the next level that i'm understanding and have wrapped my head around now what's the advance on that that you guys are doing what does this actually look like what is the difference between I mean, I'll say it just to be inflammatory, but what is the difference between this and just a decision tree?

53:52Sure, sure. I mean, I think kind of taking the first principled approach and zooming out and understanding what is diagnosis as a task, as a task that requires reasoning under uncertainty, which is what we kind of established that LLMs aren't really good at. LLMs can't form a strategy to gather the right information from their environment to explore a decision space. what do you mean exactly by that because can they not because opus 4.7 or whatever it's called will argue that it can uh i'm talking very personally here from just what i like like my experience that they they will say that this is really good at strategic stuff so what's the difference between what they're saying and what you're saying to think about what like diagnosis is, right?

54:40Like a patient walks into a doctor's office and the patient says, I have a headache, right? And what does the doctor do? The doctor first off kind of maps out like a decision space of all possible diseases based on, you know, their training in medical school that could connect to a headache. A headache could be a migraine or a tension headache, something that's like, okay, don't come in and clog your waiting rooms, like take some paracetamol and go home. or a headache could be something completely on the other end of the spectrum, like meningitis, subarachnoid hemorrhage, like a brain believer, you need to come in right now.

55:16What does a doctor do? A doctor asks a sequence of questions and they form a strategy to narrow in on that decision space. So if I find out that the headache was acute, for example, like it literally just came on in the last like hour, like it started off like a little twinge and then it felt like a dam broke in my head. Like, okay, then it's probably likely that because it's more acute, it's not, you know, something like a migraine. And so I'm exploring then the decision space of more severe headache causes. And I'm continuing to form a sequence of questions or a strategy to explore that decision space until I reach just one disease in particular.

55:56And the reward of that interaction or what guides the strategy is also like multifaceted, right? It's not just can I get the most likely disease in the least amount of questions, but also being able to sweep up all possible red flags for this thing to actually, you know, for a clinic to say, hey, we can deploy this on the front lines. It's not enough to say, hey, I think that this is very likely to be a migraine check. But if someone is like, you know, in their 50s and a little older, I need to also be able to not just say, hey, I think this is confidently a migraine. But this is also definitely not meningitis or a brain bleed or because I've asked these questions.

56:35And I can, you know, stand up in front of anyone or in a jury and tell them that I did a thorough history taking. And so this history taking is strategic and it needs to be engineered to be safe. It needs to be engineered to be accurate and it needs to be engineered to be efficient. And these are all things that are very challenging for an LLM to balance. First off, you know, an LLM like fundamentally is designed to generate answers from questions, right? So it can't really think about, it's not designed to think about what's the right sequence of questions to ask. And it's not designed to, in line with what you said in chess, take an action because it thinks, you know, 35 moves later, or in the case of maybe a medical consultation, two or three questions later, it's going to ask this question.

57:24LLMs have struggled, and now we're starting to see LLMs have demonstrated poor clinical performance, not just in making triage decisions with complete information, but also figuring out the right next question to ask a patient. There's this famous study that was done by the Mount Sinai Health System in February of this year, 2026, that tested chat GPT on a number of triage vignettes as well, right? Or a number of clinical vignettes. And it found that in 50 % of emergency cases, chat GPT under triaged. And the, which I think is even as bad in over 60 % of non-emergency cases, chat GPT over triaged.

58:07And so this idea with LLMs, like they can't actually like explore a decision space in a nuanced way. They kind of normalize everything. They lack the clinical reasoning to like a doctor think from first principles in terms of this is the best question I should ask, because either it eliminates like a large amount of diseases and allows me to narrow in on the right one, or it allows me to verifiably rule out diseases that could be a high emergency, which I can't, you know, I can't be okay with letting that slide. And so LLMs are actually not designed to take a complete history or to form a right strategy to explore a decision space.

58:49And so how the tech works. And, you know, you can, you know, LLM supporters, so to speak, who might not be the most first principles, but they're just kind of operating from the order of magnitude of, hey, LLMs can do all these amazing things, so surely they can do diagnosis. Actually, the separation that we talked about is LLMs can do all of these amazing things when it comes to tasks with complete information and where there's high certainty into a solution. 100%, I think an LLM will be able to solve the most complex math problem in the world. But actually, I think to reinvent a system from first principles where we can confidently deploy to talk to patients on the front lines and say, this thing is engineered to take a history like a doctor.

59:35It's engineered to ask the right questions to narrow in on the most likely diagnosis, but it's also engineered to make sure it doesn't miss any red flag symptoms and can do the interaction as efficiently as possible is not a task for an LLM. It's a task for a much more sophisticated decision maker. And that's where the power of reinforcement learning comes in. And it's this, you know, reinforcement learning, as long as you can model the task appropriately, which then becomes the next challenge, as long as you can figure out a way to how do you model the task of being able to ask the right questions to explore a decision space safely and effectively.

1:00:12That, really, we think is the gold path. And how we actually model the task of being able to explore decision space, you know, we're obviously inspired by AlphaGo. AlphaGo figures out the best. And LLMs, like, if you compare an AlphaGo approach to an LLM where you're kind of prompting it to say, here's the game board, what's the next move to play? like alpha go will win a hundred times out of uh you know out of out of a hundred um because that's how the that's that's what the algorithm is designed to do well just like a human um and so we look at alpha go and alpha go you know sees as its environment as its task a fixed game board of different pieces um and the actions it can take are how to move a piece on a game board and so we asked ourselves, how could we model diagnosis similar to that?

1:01:01And what we realized, it's this idea of exploring a decision space. We asked ourselves, how can we actually model a decision space effectively? And this idea of grafts, we realized that grafts are the solution. And so when I was mentioning the example of a headache and how a headache connected to both meningitis and a migraine, you can model that in a graph. You can kind of have a node that says headache and you can connect headache to meningitis and connect headache to migraine. And then you can also expand that graph. If you think about what does a doctor learn in medical school, they learn the ways to group different diseases and the likely symptoms that cause those diseases and things like genetic predispositions, things like referred pain as well, so they won't get confused.

1:01:47and we can model that decision space and all the information that connects you know different diseases in a knowledge graph and then you know what we're doing is instead of deploying alpha go like models on a fixed game board we're actually deploying reinforcement learning models to explore this graph and when you start an interaction you know where you are on the game board is you're not actually connected to anything in the graph until a patient says they have a headache and then you're kind of closer to diseases that have a headache. And then the strategy is how can you sample the right information from an environment?

1:02:18How can you ask the right sequence of questions to move that position in the game board until you get closer and closer to just one disease with high certainty? And we can talk more specifically about some of the tangible limitations of trying to prompt an LLM to do this, but not only A, is it like a beautiful solution algorithmically that we think aligns from first principles. But as we started to see, and as we've started to test, this leads to interactions that clinics say is more in line with the reasoning of how clinicians do a history versus something like an LLM being prompted to masquerade as a clinician.

1:02:56And so that's kind of how the tech works in a nutshell. And we can talk about how we benchmark it and some of the more explicit limitations of LLMs and the observed limitations but yeah that's how the tech works yeah which sounds deterministic right so in terms of a regulatory route that sounds relatively straightforward exactly and we had this conversation earlier where we need to if and you know the grand goal is if we can run our own ai native clinics right where this thing is you know just talking to a patient and it can make the decision even without a doctor in the loop on you know what what um prescription to prescribe what specialty clinic to route them to and and make those decisions and that reinvents care and to get that you need regulation for autonomous diagnosis and there's three things in order to like um that we think are important for regulation the first is determinism and repeatability if the system learns what is the most optimal interaction to take, and it can verifiably do this interaction again and again and isn't something that's stochastic.

1:04:04That's really important. We talked about the second thing is explainability. LLMs are a black box. So you can't fundamentally explain why they asked this question or why they asked this disease. That's been the problem with LLMs. But, you know, because we kind of have this knowledge graph to ground our system, we can say, you know, here's our world model of the patient. This is what we know about the patient where they sit in this graph. And we asked this next question because verifiably it reduces the amount of diseases that are close to this patient in the graph by 50%. We asked this question because we think that it's likely that the patient has this disease.

1:04:42And by asking this question, it moves the patient closer and increases the probability of meningitis. And then the final thing is safety. And this is what everyone gets wrong about what safety actually means in a clinical setting. People think that safety is just about, you know, if you're talking to, and maybe in a mental health case, it's important, but people think that safety is like, oh, if I'm talking to the system and the system says, or if I'm talking to a user and the user says, oh, like, I want to, I'm feeling really depressed and I want to end it all, then the system jumps in and says, I need to route you to a mental health professional.

1:05:20Like that's obviously important from safety, but safety in a clinical history setting isn't about stopping the interaction. What safety actually means is that if you're not missing any potential red flags, that you're not sending the patient with who you think has a migraine back home, unless you can verify that all of these red flag symptoms that would indicate that their brain is bleeding or that they might have meningitis, all of them have been checked. and safety that definition of safety is really important to ensure that hey these systems could eventually interact with humans without a doctor in the loop and we can explicitly model that as a reward and so for repeatability for explainability and for safety these are things that llms will never you can maybe patch an llm to try to emulate this logic but they'll never actually be able to verifiably prove all of these things just a quick question so why hasn't this been done before if if and again like if if it's if it's agnostics for llms llms are not part of the stack then then then conceivably this is an extension of thinking from a decision tree really and going well let's just make the tree bigger well let's just make the decisions better and kind of just going down that route until you get to this so is it the fact that llms came along and stopped that level of thinking or is it is is this in part benefiting from the revolution that we've had in ai like what what's talk to me about that a little bit i think it's a combination of both and i think as you alluded to this idea of how to build ai that thinks in terms of decision trees not dialogue that's the important thing right and rl is actually a very useful training and algorithmic approach to generate decision trees like for alpha go right but yeah to date you know no one has figured out the world actually hasn't been no one it's it's kind of an unintuitive thing to be able to model real world interactions as like a game board and so algorithmically it's quite a novel thing no one has thought to combine reinforcement learning to explore knowledge graphs yeah that's kind of like algorithmically it's a it's a it's a breakthrough so frankly it's the it's that your it's that your system is building the decision tree on the fly like that's the that's the difference it's not like there is a decision tree that this technology runs through on on a fixed board of potential diagnoses is that it asks the first question it builds an idea of a board then it's going to figure out what the best next question is in order to sharpen up the view of the board or take a direction towards something and it's going to do all of that on the fly that's my understanding 100 and and you know it can it can as you say like it can form a hypothesis like you know this this patient probably has meningitis it can ask a sequence of questions but then if that hypothesis isn't valid and this is also something where llms can't do and really struggle with it can zoom out and explore the hypothesis because it's rewarded to kind of play a game and explore this game board until it hits pay dirt versus um just come up with the the most likely uh question to ask based on you know token distribution yeah so algorithmically it's it's a breakthrough no one has thought to you know we're actually inspired there's this discipline in ai and the put you hit the nail on the head where the breakthrough is how to get systems to form decision trees on the fly but grounded in in knowledge yeah and there's this discipline in ai even before rl that people forgot about called expert systems from the 60s expert systems preceded the neural network and the idea by an expert system was if you give a machine a set of hard-coded decision trees like then it could emulate, you know, how experts ask the right questions and, you know, eliminate options to reach a goal.

1:09:11But the challenge was that was because all the decision trees were hard coded, it was incredibly brittle, and it wouldn't adapt as more knowledge came to light. And you could inform these decision trees on the fly. And so, you know, our mission to emulate human expertise and software is actually an homage, like those words are actually an homage to how we reinvent expert systems in 2026 through a combination of reinforcement learning agents exploring structured knowledge representations to figure out on the fly what is the right piece of information to get to update my world model on this patient to narrow in on a on a on a on a the right solution and so exactly as you said it's about how do you form these decision trees on the fly.

1:09:57And then also, you know, the reward in which forming these decision trees, not just to get to a likely disease with the fewest amount of questions. That's not what a doctor is thinking about. But the reward of the system is not just increasing accuracy, but being very certain that you're not letting anything slip and you're not letting any red flags slip. And so incorporating both of those things into a reward is intuitive for reinforcement learning, but it's something that just can't be modeled if you're trying to get an llm to solve this task fascinating the extension of the thinking obviously then goes to what you said initially which is can we run clinics without humans and that being a real unlock to the issue that we have of a lack of uh clinicians and this therefore plugging the gap that's very confronting for someone like me to hear and actually alarm bells are going off left right and center and the sort of thinking fast thinking slow is like there's the alarm alarm alarm alarm alarm and then and then i i have to like actively saddle back into like okay let me think about this slowly what does this actually mean this isn't going to be an overnight thing this is going to be fixed into a few specific areas first there are going to be trials there are going to be tests there are going to be checks and balances it might be like a one-to-many where you know a human supervisor let's say there's like a route to it but 100 % and that's the thing is like I know that when you have said that people would be like oh that you know the rage is starting and understandably it's an emotive topic but I think yeah that's the that's the thing that I have to remember is like this isn't going to be an overnight thing but it's a direction of travel and it's can we do it safely can we do it appropriately the other thing that I do want to mention on this as well again another really confronting thing that I saw was a piece of research and I don't I don't want to quote it word for word because i'm going to try and find it um before i talk about this again but basically they looked at young people and what they wanted from the health care system and yes it's only one study and blah blah blah but a lot of people didn't want to see a human they just wanted the problem solved and i think when it comes to primary care and it comes to community care and it comes to people even outpatients and it comes to you know people wanting like something where they're not judged where they don't have to queue up where the problem can just be solved they feel a lot more informed the younger generation about their health as well that they don't need the kind of like advice and guidance that we once did particularly if you look into the past and compare that to even now and you extrapolate that into the future what what What young people want for the next generation want from their health care system is very different.

1:12:44And it's and it's really important that people like me understand that and explain that to people as well. But that is what's coming through. This is a direction of travel. This isn't us forcing a new view of the world upon people that don't want that. This is actually also in response to what people coming through are wanting. I think that's really important to know to understand and to kind of keep front of mind and I'm saying this as much to myself as anyone else that my view and I look I'm on record saying that AI should never be placed at the point of human suffering alone I've said that multiple times I reversed that decision on the grounds that a lot of people very kind nice people got in touch with me and basically convinced me that that is an indefensible position given the state of health care uh and where it's going and the complete unsustainability of health care in its current state and to hold a to hold a kind of a romanticized position that well only humans should look after humans because that's what we want from care is all well and good but in order to hold if you want to hold that position so that when you could do something else so that you force the system into an economic model where that is possible well a lot are going to be a lot of people are going to suffer in the meantime and i was convinced otherwise and so actually i'm far more open to ideas that you've talked about in a safe way and being that kind of okay well can we ask the right questions in order to make this happen what would that actually look like and what it generally looks like is well it will be supervised first and it will be in a very small area where we know it's the most safe first and then we'll expand from there and it may look like this wild difference in future where it's just ai clinics of people going in and coming out with the right diagnosis but there's a there's many many steps in between um my question though is that where is the where's the tech now like where what have you built and whereabouts are you with this first off i'm really glad that you uh from for what you just shared as well right this isn't like a overnight thing.

1:15:00And really to the point on like the end mission is how do you democratize care in a really fast and very high quality way? Because there's so much of the world, you know, in rural populations, even in non-rural populations, right? It takes 30 days. 1.4 million GP appointments take over 30 days to schedule, which is crazy. And in the US, the average wait time to see a primary care physician is also 30 days. And so there are really important challenges that we have to solve in order to be able to deliver care of high quality and high speed to everyone if we believe that that is a birthright for humans.

1:15:42And I think AI can be a very important thing to amplify our ability as humans to deliver that care. um but yeah where are we right now i mean we are definitely still in an early stage and um you know we've designed and we're seeing the first versions of our models internally and we're pretty excited with their um with their with their progress we've actually been showing it to a bunch of clinicians and they've been pretty impressed at that like the sequence of questions that the system asks and its ability to conduct like a complete history um at least like qualitatively passes the vibe checks for clinicians.

1:16:21So we're really excited. The last couple of years have been honing this research direction, building the necessary technology infrastructures, and now testing it on medical knowledge and on this task. And so we're really happy with how far we've come. The first version of the models are alive, so to speak, internally. And then the next opportunity is to actually test and evaluate this, not just on synthetic patients, but also for real-world patients under the supervision of a clinician, which brings me to the biggest opportunity, which is, to date, there are no agreed-on benchmarks for conversational diagnostic interactions.

1:17:05And so when people are like, oh yeah, AI can or can't emulate how doctors interact with patients, there is no agreed-on benchmark to do that. As we talked about, you know, MedPOM can solve medical exam questions, as well but there is no compelling benchmark that proves that in a real world setting these systems can run diagnostic consultations and make a correct triage decision and so we see a huge i was just gonna say do you know what travel is like this this is a bit of a passion of mine because let me phrase this properly like the presence of ai is forcing us to figure out what is it exactly that we do as humans that is valuable and it's so interesting because had had ai not come in the way that it has we wouldn't be asking half of these questions like you just asked what is it about a diagnostic consultation that actually is good or bad or better or worse or higher scored or lower scored what is that score what is that based on because in order to try to get a machine to do it we have to ask that question i think what's fascinating is that then you can actually benchmark humans doing it and i think yes there's an objective truth at the end did they get the diagnosis or did they not did the patient feel good about it did they not and you know you can sort of measure that qualitatively through through you know reporting patient reporting i guess of like how did that person make you feel and all that sort of stuff um but i i i think what you've just said is is fascinating so i'm so interested to hear what you found yeah and the exciting thing is we're not the only ones that are looking at this problem and i think for a primitive like conversational diagnosis you want lots of people looking at this problem and you know from a research perspective scientists can amplify each other i believe we are the only ones who are looking at this problem from a non-LLM-centric perspective, which is exciting.

1:19:14Because if you look at it, it's not just about the outputs that are important, like how do we actually grade the system on these metrics, but also the inputs. What are the range of real-world situations with which we're testing it on? Patients are messy. Patients have high degrees of medical literacy. Patients have high degrees of ambiguity. A patient might come in with a referred pain like in their shoulder and you know an llm might really focus on that but not be able to zoom out and ask you know questions to paint a broader picture of the patient and then narrow in on somewhere else patients lie like in the real world patients lie so much right and so there needs to be you know a design not just watching house yeah i've been watching house right um you know there needs to be a design not just that shows how the the system is going to be evaluated on what dimensions.

1:20:07Naturally, it's going to be evaluated with a clinician supervising it 100%. But that also needs to take into account what are the right inputs. And when you look at other people building in the space, obviously, we have huge respect for DeepMind and AMI. Ironically, DeepMind is taking more of an LLM-centric approach, like the conversational diagnosis, as most of the world. There's a startup, Curai Health, which was founded by Neil Kostla, the son of Son of the Node, Kostla, and obviously have a lot of respect for what they're doing. Cure Eye Health was, they actually ran a real world study that was testing their LLM based agent for conversational diagnosis.

1:20:48But, you know, the findings represented a lot of self-selected triage patients, where there was low degree of ambiguity, right? And by that, I mean a patient that comes in and says, I have a chest pain. It's radiating, you know, through my left arm and my grandpa died of a heart attack. Right. Like that's obviously low, low uncertainty. That's that's that's that's an MI. Right. That's that's a heart attack. And you're triaging that as an emergency 100 times out of 100. But the real opportunity is to be able to design studies. And we think that will expose the limitations of LLM-based agents. And also, you know, we're not coming into this cocky at all.

1:21:30We want to be honest about where the vulnerabilities in our system are, too, in real-world examples. And we want to do a larger-scale evaluation against frontier LLMs and our approach in a real-world setting, supervised by clinicians, of course, that evaluate, that first off can interact with patients with a range of medical literacy, with a range of complete information levels. So, you know, some patient might be able to share everything, whereas other patients might not share everything. And you really need to push the patient to, like, ask the right set of questions. You know, with a degree of noise, whether that's, like, patients reporting referred pain or patients maybe even, like, making up symptoms.

1:22:16right? The real world is super messy and there need to be studies that can evaluate the messiness. And then how we actually evaluate the system, I think there's three major components. Obviously, like accuracy is number one, like on these messy symptoms, like can you actually get the right diagnosis? And, you know, also like the right diagnosis could be a diagnosis that like after a clinician would do a similar history, they would approve. Or the right diagnosis is actually after you do the blood test and you confirm that it is, you know, this disease, like that probably has a higher degree of confirmation.

1:22:50And so designing the system to get accuracy in the right way is important. Then safety, as I talked about, which is one of the most important things to not only define correctly, but measure for. And, you know, in Curi and D-Mind have done their studies. First off, D-Mind's study was also only on synthetic patients with low, and they excluded emergency cases, right? But when you look at safety, safety isn't just about being able to escalate emergencies. Safety is, and I remember we had this amazing conversation when you said, for this thing to be useful in a clinical setting, it doesn't just need to find the needle in the haystack, but it needs to safely remove all the hay.

1:23:30And that analogy you said to me sticks. And what does it mean to safely remove the hay? It means not just say, I think that this patient doesn't need to be escalated. And I'm right about that. But it's also that I've also, if like a clinician says, hey, are you sure? Are you confident in this decision? I can say yes, because I've looked at all of these adjacent emergencies and I've asked all the red fly questions that have ruled that out. There is no publicly available definition of safety in that direction. And naturally that means there's been no system that's tested for safety in that direction.

1:24:04And then, of course, conversational efficiency is another one. And so we are early days. The models are alive internally and they're continuing to improve. But our goal is in the next few months to figure out and design these real world studies in a clinical environment that can be a publicly recognized benchmark on what it means to test a system for conversational diagnosis. And we welcome, you know, collaborating on that design with other people that are building with LLMs. We're naturally going to test our systems against frontier LLMs, as well as like with clinicians to paint a complete picture on not just what performs better, but also in what scenarios, you know, are there limitations in our systems and LLMs, even in human oversight.

1:24:48Right. And so we welcome kind of collaboration and designing those studies, even though right now we're taking the lead. But then once we can, you know, confidently evaluate our systems on these new benchmarks that we're building, then kind of the way that we were going to bring this to market, we're not going to try to design an AI native clinic on day one. We still need to understand how primary care clinics work. And that also requires regulation. we actually want to amplify the amount of patients that primary care clinics can see. And so we will then deploy our systems as the digital front lines to primary care clinics, digital health companies, to be able to do those interactions and for triage.

1:25:31And there can be a doctor in the loop that verifies the interactions after each consultation. But the hope is that this will vastly scale the amount of patients that a clinic can see, improve the quality, improve the speed of care, get the clinics to see the patients with emergencies faster without needing to hire more staff where they're clearly bottlenecked. And that is a multi-year vision, which we think also has a large revenue potential that can compound the research as well. And it's all about, you know, getting real world interaction data that is validated to then be able to keep improving our models.

1:26:04And then, you know, building an AI native primary care clinic, that's obviously the holy grail because of what we think it unlocks for humanity. And if you think about what primary care is, it's almost like a marketplace, right? It's a marketplace that connects humans to pharmacies, to like specialist clinics. And, you know, right now, like those marketplaces are run really inefficiently and they struggle to like cater to all demand right because they're bottlenecked by human labor and so we're figuring out a way to like completely change the margins of delivery by being able to um basically like run these these more efficiently with ai that can route you to the right place and so yeah that's kind of roadmap amazing i think it's really interesting man one thing before i start wrapping this up one thing that i do just want to mention is that when you're building this one thing that i would encourage you to think about is that separating diagnosis and treatment is fine but there is always going to be an overlap of the two you can't do a diagnostic process end to end without doing a bit of treatment at the same time what i mean by that is within that diagnostic process and and you can wrap in writing the prescription or coming up with the diagnosis communicating the diagnosis to the patient is all part of that really that you can't get away from the fact that one huge part of that is the is that very end bit the communicating of the diagnosis to the patient i think the rigor that you're going through in determining what is a good benchmark for a good diagnostic process should equally be applied to what is the most what is the most rigorous way of assessing how that information is delivered to the patient because there are multiple different things there of like there's a very big difference between diet you know communicating a diagnosis of bacterial tonsillitis to then hiv i mean that's an extreme example right but it's right it's there it's there to make the point the other thing that i that i would say on that is um well first of all i would just say that i don't i don't know if appropriate like uh allocation of resource is is applied in that area specifically with the way that people are thinking about this i think it's an incredibly important part and not necessarily just because there's a difference between an hiv diagnosis and bacterial tonsillitis in the way that that's explained although that is obviously a factor even you know bacterial tonsillitis viral tonsillitis uh just the sore throat associated with a common cold uh someone that's immunosuppressed with a sore throat that says oh it's fine like you've done everything it is just viral like it is just a common cold and sore throat because of that but that needs to be communicated very differently if someone's immunosuppressed because they're going to be worried and anxious about stuff so there's a huge amount there's a huge amount of nuance in the way that a diagnosis is given and the other thing as well is that i think every clinician when they deliver a diagnosis is always optimizing for a couple of different things like not spiking their anxiety and also increasing their adherence to whatever the treatment plan actually is and I think that is also worth bearing in mind there's going to be other factors as well but that's just me picking a couple that that come to mind in the I'm you know if I'm delivering a diagnosis oh it's you know this is what it is but it's very treatable or this is what it is but we're going to do these things then you're going to be okay like all of that is part of it to reduce their anxiety make them feel better because that's part of what good healthcare is um and also like you're you're gonna like that is your moment to really make sure they adhere to the treatment as well like that's so so so important because like just saying tonsillitis amoxicillin four times a day for seven days and just saying that like that being a text output to the person or voice or whatever it is in whatever version of the vision of the future we have in an AR clinic is fine but there will also be if we're going to do public health properly and we're going to do primary care properly and we're going to do community care properly there's also going to be an optimal way of delivering that to increase adherence in the community which actually would also solve a massive problem that already exists how many people actually do their physio exercises to the volume and level that they're expected to when the physio says do blah blah hardly anyone but like if everyone did well actually what's that going to do for the amount of people that present with the next injury so it's super interesting what you're doing and i think that it it would be remiss of me not to mention that because i think that you're doing so much of the work already that actually i think if you're going to solve a problem in health tech do it end to end if if you can capture that bit as well and apply all of the thinking that you're doing to the the information exchange of what the diagnosis is and everything that you can get the benefit from at that moment in terms of delivering for public health i think yeah it's um yeah i think it's super interesting that's a great point yeah thank you definitely i think it's this idea if you want to build systems that not only can gather information uh effectively but then can communicate like information, whether that's the diagnosis, the expectations of the consultation with empathy, like that's, they both compound each other, right?

1:31:58Like the patient is on a degree of trust. They're going to take the treatment. They're going to volunteer more information. And so I actually think that's something that in our system, because like the interface of our system is an LLM, right? Like is that kind of, you need to generate dialogue somehow. The R kind of AI, RRL exploring a knowledge graph is like the brain of the system that determines the question to ask. And the LLM is like the mouth, like how do you generate dialogue and then pass it back into our system? And I think that's actually an exciting opportunity and things that LLMs do well is they can generate, they can be quite empathetic.

1:32:34They can generate the right dialogue and kind of in conjunction with our system, you could have something that not just asks the right questions, but is actually like quite empathetic and gets you to take the treatment and allows you to build trust in it to where you're confident. Or gives that advice to, like, the interim step would be, and based on that history, gives the advice to the, like, let's say you've got a one-to-many model of a clinic, right? You've got this AI across 10 different consultation rooms and a single person. Their job is to do the communicating because we're in the interim of going pure AI.

1:33:06Let's say we're in the middle, right? Like the AI is going to do all the diagnosis, but it's going to leave it to the human to do the communicating. But the AI actually, based on that history, based on the knowledge of how all the information was communicated, is going to give advice as to the how. It's going to actually just tell the human how to do it. And eventually it can then do it itself through whatever weirdly humanoid robot is going to do it. But ultimately, I think that is an amazing and super interesting interim step of, well, that takes a huge load off the person as well. because it's not like, yo, you just got to go and deliver this diagnosis that's really difficult.

1:33:40It's like, and based on what I know, here's how delivering that is going to optimize for these things and make the patient feel better. Right. Like, I think that's really interesting. Or based on what we know, like here's how to get them to adhere to this medication. I always use the example of, um, I always, I always use the example of, of a GP trainer that I had, um, that on a specific patient didn't follow nice guidelines and i asked him why and he said because they pray five times a day this will make the adherence better and i was like wow okay interesting yeah really interesting and it was like one of those like formative moments in my medical career of going like ah i understand what joint decision making is now i understand what what you know why you would veer and why why you are a single practitioner and why it is guidance and why like why it's a multifaceted thing like decision like that like loads of things clicked in that moment for me that i still remember it to this day even though it's like 20 years ago right so yeah it's it's it's that essentially which um yeah definitely sticks in my mind yeah i mean there's no there's no one size fits all with medicine right there's no like one one guideline it's all you have to yeah well this is why we end up in the philosophy pretty easily and pretty quickly right it is a spiritual discipline medicine it's both scientific and spiritual um you're dealing with that kind of rawest aspect of the human experience which is health um so navigating that like safely accurately and empathetically like that's like the the north stars of any technology should be so i completely agree 100 um but listen man i i love the ambition that you have and the reason that i sort of explain my position on it and and stuff is that i think you of all the people that i've met um particularly recently i i just think there's something there's definitely something different about you guys obviously about your co-founder as well and um i think the ambition that you've got the the experience that you've got the the frankly you have the ability to build something here that's like i think generational that that will make a very big difference in healthcare i think we're all very uncomfortable with llms and particularly i think the lack of determinism is what makes us all pretty like cagey around them in healthcare like we're very very i don't know we we don't like the idea of them close to health care close to the patient sorry like it doesn't feel there's something doesn't feel right and that's not just me that's that's people that i speak to and people that i know anecdotally as well as even looking at a lot of the research that's coming out like we're just not that comfortable with it whereas something like this that's based on modern principles without you know taking advantage of every everything that's happened with ai and the advances but actually not using the llm portion and actually going back to what can be truly deterministic and safe.

1:36:33Honestly, I think it's fabulous. Before I let you go completely, you worked with Sam Altman, which is really interesting. You're part of a co-founding team with him, which means you must have worked with him very closely at certain points. There's a lot at the moment here in the news, in the press, about you know people are allegedly saying he's very dishonest out for himself like all these different things that aside um obviously a heck of an operator and and doing what he's doing what what does what have you learned from him is there anything that you think about now or anything that you took from your time working with him that you that you apply now into into i guess becoming great at what you do one thing with sam is he's he's intrinsically like a generous person and and i don't think much of the world sees that like he's generous with his time obviously like he's generous with with his capital as as as you can see um and naturally like obviously like he he has the principle it's not like a hundred percent selfless right but it's the principle of like these things being generous like ultimately compounds right you can empower really smart people to do really ambitious things and and guide them on the on the journey and then also So, you know, Sam, like in this idea of like personal legends, right?

1:37:52I don't think, I think Sam is very intentional about what his personal legend is. And that's why he chooses to work on the projects that he chooses to work on, whether it's building data centers or even with WorldCoin. The idea behind WorldCoin was, obviously, I described it almost like a social experiment, right? Where it's like creating this world currency. but that one of the core principles behind world coin is it's very important on the internet to see who is a unique human and and who isn't right especially in the world where with ai and there's going to be tons of bots producing slop and so this identity product we're actually building an identity protocol that not only allows you to distribute a world currency but verify yourself on on the internet it's like a captcha to um vote like safely and and kind of uh different different online elections.

1:38:41And so it was quite this idea of a personal legend and seeing how things are like uniquely connected was inspired by Sam and the ability to intentionally work on big ideas from first principles. Like that's the mental model that I've learned a lot from Sam. And then kind of how to find the right people and empower them is something that I learned a ton from him on. And I think Sam also technically this idea of building a platform company versus building a product. WorldCoin is a platform. The orb, the biometric devices that we built, is a platform for proving unique humanness at a global scale.

1:39:20And that unlocks a number of products and applications, just like we've built a platform to emulate human expertise. And in medicine, that can be deployed on a number of primary care specialties just by expanding the knowledge graph the system can reason over um you know and as new diseases and treatments the system can can um can expand to to to emulate a number of expertise across domains even eventually beyond medicine and so you know another thing i learned from him is that you know the most ambitious founders work on uh platforms not products and i i carry that with me um and yeah i i definitely learned a lot from him also just like his kind of attitude and his demeanor he has this level of intensity but also warmth with which he with which he carries himself which i think is really important because you want to attract people to to especially if you share big ideas you know if you embody a bit more of a cold persona people will naturally be skeptical but if you're quite warm quite intuitive um and quite intense like people will believe in the future that you share and i think sam does a great job at that from a communication standpoint that i want to emulate building teams and and and yeah i guess the one thing i would maybe disagree with him on is that like llms aren't the path of super intelligence and that's been what brought me to to starting this company but honestly you know oh a lot of my career my mindset my ambition to to the world coin team as a whole and and what i learned from sam so super grateful for that experience amazing um dude it's been an absolute pleasure um thanks so much i know i've kept you for longer than i do most of the guests but I really enjoy talking to you um I imagine people listening might want to get in touch with you or learn more about what you're doing what's the best way for them to do so email LinkedIn um I can give I can give you my WhatsApp number but maybe that's I I definitely um like love love people reaching out like if you work in healthcare if you work in AI and are excited about you know the team that's working on conversational AI diagnosis and the team that's going to start running these experiments and trying to bring this to the real world and going beyond llms like whatever opinions you have i want to hear it and i want to um and if this aligns with your personal legend in some capacity like let's let's chat um so yeah i i can share my email um or my what yeah just say your email out loud that's that's all good okay uh shravan s-h-r-a-v-a-n at othman labs a-t-m-a-n-l-a-b-s dot a-i shravan at othman labs dot a-i perfect just reach out over there perfect um i need to read the alchemist again you've just reminded me i love that um i read that book actually when i was in vietnam like that long ago so i'm definitely i'm definitely going to read that again so i've written that down um the other thing is obviously you've got uh you mentioned investment and you're going to do a round soon yeah i imagine your seis allocation is long gone so i feel that that sucks or else i'd be trying to get in on that but um what is the next round that you're doing and uh yeah is there a is should should investors get in touch with you yet or is it an i'll call you type thing yeah we we just started the the next round actually we've been talking to a number of um investors not even from the context of raising, but just kind of evangelizing our mission.

1:42:47Investors that are working in healthcare and have a distribution network of clinics, they saw the demo and they said, this thing should be talking to 10 ,000 patients at once and asking how we can invest. And yeah, we're at the stage now where we've hit an inflection point. We've built these models and the next two years are going to be about testing, evaluating, benchmarking these models, starting a revenue stream of launching these with clinics and building a hopefully billion dollar business there. And then without fear, going towards the path of ultimately truly democratizing healthcare. And so we have an exciting trajectory.

1:43:25The right capital can amplify this trajectory. We're very intentional about the capital that we raise because the systems that we design are incredibly, we're quite precise about our capital allocation and our team. and we operate like a really well-oiled machine, I would have to say. So yeah, we've opened up a kind of larger seed round and really looking for the best, most ambitious investors across healthcare, AI, anyone that's both non-consensus, first principled and just excited by this and wants this to be a part of their personal legend, reach out. We're opening a seed round. So if you're a lead, you're a follower, you're an angel, happy to chat with you.

1:44:03Do you have any EIS or SEIS left? I think now we're allocating everything to the new round. Ah, shame. We'll chat, James. We'll chat. We should. We should. We should. Shraab, it's been a pleasure, mate. You've obviously been a big part of the Monto model. Perfect. It's been a pleasure, dude. Thank you so much. Thanks, James. Appreciate it.