Are AI Agents Going Rogue? | with Robin Matthews, PocDoc

0:00They're not actually self-aware, are they? Even that process has taken our backlog of errors and issues like that and pretty much eradicated it completely overnight. What you're seeing now is they're hitting over 80 % of the equivalents of a really experienced senior engineer. Any form of centralised regulation is probably not as well. It'll be out of date before you've finished it. It will, exactly. Still using their taste and intuition that they've developed over the years to learn how to write great software, but applying it to a different set of tooling. And I think if you look at it through that lens, for me personally, it's a super exciting time to be involved in software.

0:40Every day, just taking a step forward and learning something new.

0:45Steve Roest: Hello, and welcome to this week's Health Tech Hour with me, Steve Roost. Each week we bring you the best news, views and interviews with the CEOs, leaders, clinicians, journalists, investors, the list goes on, who are changing the face of healthcare in the UK and beyond. As regular listeners and viewers will know, I am a CEO and co-founder of a health tech venture myself, which is called PocDoc. And as of last night, PocDoc has been announced as one of the fastest growing European startups and in fact, the fastest growing diagnostic startup in Europe, thanks to Sifted, which is part of the Financial Times Group.

1:23Steve Roest: Our flagship product, the Healthy Heart Check, is now the number one tool the NHS uses to screen for cardiovascular disease outside of a GP surgery. And it's available from PocDoc, that's P-O-C-D-O-C dot co, or from Boots, or from Pharmacy2U, or from Amazon. So if you want more information, head to pocdoc.co, that's P-O-C-D-O-C.co, or to any of the major retailers, and we'd love to hear your thoughts on the product. Thank you if you are listening to this live on UK Health Radio. We love UK Health Radio. It's an incredible station with tons of incredible presenters and content. We love Johan and the team, so thank you so much for the live platform.

2:03Steve Roest: Thank you also if you're listening on any of the podcast channels. We get downloads now in over 50 countries every single month. It's absolutely insane. And if you're watching on any of the social platforms, TikTok, YouTube, Instagram, where they're all at Health Tech Hour or at Steve Roos. Thank you so much. So we're currently running at about a million views every 30 days at this point across all platforms and streams, which is absolutely mind blowing. We love that you love what we do and we love having you here. So thank you so much. Now onto today's show. I have been really excited about this show for a while because it speaks to a huge amount of what's going on in the world today.

2:42Steve Roest: And we're doing a show on AI today. Now, we've got a very special guest on today's show, Robin Matthews, who is widely acknowledged as a thought leader in the UK on how to integrate AI, agentic AI and other aspects of AI into medical technology, diagnostic platforms, digital diagnostics. Robin also happens to be PocDoc's chief technology and product officer. Now, this isn't a setup. It's genuinely that Robin is one of the most widely regarded thought leaders in this space. And this space is moving so fast. And I realized we hadn't done a show on this and I wanted to get Robin on. Now, I've mentioned this on PocDoc and on other channels.

3:25Steve Roest: Sorry, on LinkedIn and other channels. PocDoc, we believe, is one of the leaders, if not the leader in regulated medical technology businesses in terms of adoption of a Gentic AI. And it's unlocked unbelievable growth and acceleration for PocDoc. And in my role in the industry and in Robin's role in the industry, we do our best to try and help other leaders and founders understand the benefits of all technology. But in this instance, Agentic and actually later today, Robin is hosting an ever increasing number of UK health tech founders to help them understand how to unlock the benefits Agentic offers in a safe, clinically secure environment and manner.

4:09Steve Roest: So Robin, welcome to the show. How are you? Thanks. It's great to be here. Really shocked to finally be on the show. I know. It's a big moment. And for those of you, if you're listening, you won't see this. But for those of you watching, we've both got tech bro black on. That wasn't coordinated, by the way. Right. So let's let's there's a lot to unpack in the show. And there's also every single day. There's loads of news stories about AI. Everyone I know, I'd be shocked if anyone listening has not heard AI and the names OpenAI and Anthropic. and claude and gemini and all of these different bits and bobs we're a broad church though so i want to just kind of get down to the definitions of what we're talking about and i wondered whether maybe you could kind of take us through a very a little bit of a potted history about the evolution of this thing called ai and where we are now versus where we might have been before because this term's been thrown around for a long time yeah yeah absolutely and i think you know what What's happened in the last year and a half to two years is what we've seen is a massive increase in the capabilities of large language model-based AI systems.

5:15And essentially, a large language model, what it is doing is predicting text. So when you ask a question or you type into the input box in Claude or OpenAI, ChatGPT, what the model is doing is it's predicting what the text should be, what the answer should be. And what we've seen over the last year or so is a massive increase in the capability of those models because they've had a lot of human reinforcement learning put into them to make their responses so much more real and useful to us as everyday users.

5:51Steve Roest: And why has the world and these businesses and humans discovered, maybe discovered isn't the right word, but fixated upon the ability to predict text as this unbelievable unlock of capacity? Yeah, I think my personal take, and maybe it's because I am a sort of a software engineer, really, you know, that's been my careers around technology. My personal take is I think this has grown out from technologists experimenting with lower level AI technologies like machine learning, computer vision, that kind of thing. And I think technologists have been able to unlock this capability that means that they can actually develop software so much more quickly using large language models to predict what the code should look like.

6:40So I think it's born out of that technologist building tools to solve technology problems is my belief. Obviously, it's broadening out now so that a wider range of people can start benefiting from that. But I think that's probably where it's come from.

6:55Steve Roest: And is it true, I said I wouldn't ask you a yes, no question, I'm going to ask you a yes, no question, but I think it's an easy one because I heard this. Is it true that the sort of origins of this, the current agentic AI or sort of large language model AI comes from the same sort of core technology around the predictive text stuff? Yeah, absolutely. It's on that continuum. It is, absolutely. So there's kind of a, you know, some people call it maturity ladder or, you know, I prefer to refer to it as more of an adoption ladder because I think it's not really fair to sort of say, you know, some organizations are more mature on this journey than others.

7:29But yeah, it all started off really with kind of word prediction. So within, say, a software development environment or even inside, maybe it's a word processor, for example, being able to predict the next word that you're trying to type was the starting point. And from there, we went from sort of predicting a next word to predicting a whole phrase that might be useful. And from there into more fully sort of agentic kind of approaches.

7:57Steve Roest: And how, because I mean, I remember when predictive text came out, you know, and, you know, the little paperclip thing in Word, you know what I mean, was trying to help me write some letters and stuff like that. That feels a really, really long way from where we are now. So what was happening in the background? Yeah, I think the unlock has been a couple of things really to unlock. I think one is the development of hardware devices that are able to actually process so much more data so much more quickly. So GPU development over this time frame has really taken a hold, I guess, in terms of the speed with which models can be trained and with which those models can be deployed.

8:36And then I think the second thing is really the accessibility of data. so so much more data has been available to be able to train models you know across literature across you know code um i mean i think i think a lot of this started with you know github actually you know built one of the first really capable uh software engineering um products and that was because they were able to you know scan millions of open source software libraries and to be able to learn you know from those um what what good code should look like for those of you listening who don't know what github is what is github so github is a uh a software repository the leading right the leading software repository um service um where yeah huge amounts of code both publicly available and privately available are stored right and at what point did you

9:31Steve Roest: in your role as cto ctpo either at pocdoc or before you joined although you've been at pocdoc for a while thank god but um when did you start hearing this because i'm guessing when it was still predictive text it was like oh that's kind of neat but it wasn't like oh my goodness this is gonna completely change the entire planet yeah absolutely i mean obviously in my role i you know and throughout my career really i've been following technology trends and things like that so i've been obviously aware of this for a long time um you know some of the engineers on my team now and and previously have been using the sort of co-pilot style approach to sort of AI support.

10:16So where they're working in their development environment and getting hints and help to build out modules within their coding environment. That's been going on there for a few years. And some people like it, some people don't like it very much. So adoption was mixed, I would say. but I think really what changed was the the launch of um you know chat gpt and claude when they started to become really useful at predicting um how software could be built and how to resolve a a problem that's been written in human language and turn that into code I think at that point for me it started to become really super useful and I started doing quite a lot of experimentations around that um not really introducing it within the sort of the commercial world but kind of just checking these things out really and figuring out how they might be useful and then i think probably for me it felt like a bit of a toy um and i think it was only really in the last six to eight months where the usefulness has really just kind of gone off the chart now where if you look at a lot of the evals that are done against some of the latest models what's an So this is an evaluation of a model using a certain set of test criteria.

11:28If you look at some of those evals, and particularly the ones around sort of the equivalents of a senior engineer, what you're seeing now is they're hitting sort of over 80 % of the equivalents of a really experienced senior engineer, being able to pick up a problem defined in human language and being able to turn that into a software solution. I mean, that's then becomes really super useful, I think. and that's when we started to to look at adopting it within pod dog and when but not everyone's a

11:56Steve Roest: fan right or well or rather maybe that's too simplistic there's there's different views if we just take within software engineering where it sounds to me what you're saying is that the the adoption rate was highest within software engineering because the individuals within that sector were the most likely to be able to understand it clearly yeah and then they had a highly applicable problem that they were trying to solve which was how can you help me write code better faster yeah absolutely is that right yeah that's fair um okay and but not everyone in that sector is is on board or like there's a different views on this right yeah absolutely and i've been i've been sort of spending a lot of time thinking about that problem actually over the last 12 months or so um i've i've been going through a process of building out our internal software engineering and capability adding new people to the team and going through that interview process has been a real eye-opener because I've spoken to probably hundreds of software engineers from sort of mid level to sort of very senior principal level engineers and I was actually really quite surprised at some of the resistance that people were demonstrating towards the use of AI but when I thought about it it's kind of like it's understandable really because people have built their career based on the things that they've learned, the experiences they've had, the craft in writing code.

13:19These are the things that have been valued by their employers. These are the things that have driven promotions and salary increases and things like that. And I think give them their sense of self-worth, really, in the work that they're doing. Now, almost overnight, that's been challenged and almost removed from those people. so i think we're looking at a real schism sort of if you like where all the things that previously they valued about themselves and people are valued in their capability are no longer relevant and you

13:54Steve Roest: you but you're also in that category absolutely right that's you're you're in the same boat absolutely well i think you can look at that problem in in with two lenses i think you can either look at it and say right okay well you know everything that i've done is now you know gone and I've got very little value and I don't know what I'm doing and I'm not going to get involved in AI. I'm going to tell everyone it's really bad. And there's a lot of that still out there. Or I think you can look it through another lens, which is actually what's happening is my role is changing and I've got a whole new set of stuff to learn.

14:27How do we actually embrace AI? How do we make it agentic? How do we turn it into a software factory? How can I support that within my organization? So for me, it's shifting capabilities, still using those skills, still using their taste and intuition that they've developed over the years to learn how to write great software, but applying it to a different set of tooling. And I think if you look at it through that lens, for me personally, it's a super exciting time to be involved in software. And I think that's, you know, I've hired people and we're encouraging that kind of mentality within our organization.

15:04Steve Roest: And we're super grateful for the people that have joined in the last 12 months, just absolutely insane level, which is awesome. Let's just dial it back a second. We've jumped around and I myself as well have used different terms, AI, large language model, agentic. is there a is there a framework or a way of helping everyone listening kind of understand the different buckets or types or evolution because they're not all the same and yeah but they do tend to get used interchangeably a little bit even by me yeah they you're right they do and i think language is evolving really rapidly in this space as well i don't think there's any you know really sort of preconceived notions around these things because i think a lot of these capabilities are relatively new so i think you can think about it really as things like you know machine learning and then building large language models as the foundation so these are the kind of the core of of what sits behind the things that we interact with and machine learning again i might butcher this but my understanding of it just to check with you is effectively that it's a fairly deterministic set of algorithms based off of a certain set of data is that is that fair so so it's but because there is a difference between ml and ai yeah well yeah there is and you know it gets blurred i guess it gets blurred i think is that is kind of the issue and and you can use ml in different ways for different you know different outcomes um so i think um what we're really talking about here is the use of those uh capabilities to build large language models that can then be used to make predictions, which can then be developed into products.

16:45So kind of that's your base layer. Sitting above that, you've got then sort of the ChatGPTs and Claude's of the world where you can interact with those models directly through a chat interface. And that's kind of...

16:57Steve Roest: And that's when you go into ChatGPT.com. Yeah, absolutely. And you can get responses. You can get it to make you a PowerPoint presentation or to give you some thoughts on an Excel spreadsheet or whatever. those new tools are being added to that layer all the time so the ability to use a file locally on your computer or to generate a file in the cloud these are new these are new layers of tools that being added to that layer then sitting above that i guess is the ability to run an agent so where you're effectively running a piece of software known as a harness which is kind of accessing the large language model and the tools to be able to perform independent work on its own so that's kind of the next layer up so like creating almost like a sort of human ai yeah a capability you're giving it some you're giving it some skills you're giving it some routines you're giving it um some some uh sort of um sort of safety measures if you like things you do and don't want it to do um and then letting it perform its job autonomously and then sitting above that now you know you can have what's known as a factory i mean this again this language is evolving but kind of the i think the the most popular phrase is a factory whereby actually there's a queue of work and the agent is effectively continuously running against that queue able to pull work down able to perform that work and then able to pass it done back when it's completed back onto a queue for humans to review and i remember when we when we've had a lot of talks about about this area and some of them felt like i was in the matrix to be honest with you but the idea here is effectively that there is no cap on the number of agents that one can have yeah there is no limit to how large that factory can be so yeah so so capacity constraints have sort of hypothetically gone, right?

19:00They've shifted, is how I would describe it. So you may have removed some of the capacity constraints around software engineers writing code for sure. But that puts more requirement on defining intent and work that needs to be done at the front end of the system, if you like, of the factory. And more at the back end of it as well. So being able to actually review the code that's coming out of the back has become a new bottleneck for people. So I wouldn't say it's necessarily removed. And I think that is a common sort of misconception is that now we can do almost anything. But actually there are still some limitations around there and you still do need human involvement.

19:41Steve Roest: And that's what you referenced earlier about how the role of a software engineering team in terms of human shifts. So it's not necessarily the stuff in the middle, but it's at the front and the back. Yeah, absolutely. And it is also improving the stuff in the middle. So kind of working with the agents, improving the agents, you know, for the first time ever, software engineers are really able to get really good quality information back from the agent in terms of what it's done and the success of what it's done and over what time frame. And they're then able to improve the way that that agent works over time.

20:12So, you know, there's a role there for them as well in improving the factory over time.

20:18Steve Roest: And let's, and we're going to go, we're going to stop for a commercial break in a sec, but we'll try and get this one in here. What about all of this do you personally find the most exciting? What really gets your engine running? Well, I think I'm fundamentally a product and a technology person. So for me, it's the ability in a startup business to be able to address more of the problems that we want to solve for our customers. historically you know startups are both fun and enjoyable also massively constraining because you know you've got a limited set of resource and there's only so many things you can do you almost always you know just prioritize one or two things with the ability to be able to try and develop more software products you know we we can release more things and kind of address more customer problems okay and beyond let's talk for a second about what what happens beyond software engineering so what's your view about those things because i i think from my take there are there are some people that that still continues to disagree with the impact of ai within software engineering but i feel like on the macro that argument is sort of done a little bit you know i mean i've personally i think it's completely done i don't i mean i'm not in the it's not much but i mean there may be some very very niche areas where you know a human is still the best option but i think for generalist kind of software development development i think you know that comes down to using using ai and but what about beyond software engineering yeah well it starts to get a little a bit more subjective it does and and and actually this is a new area that i'm exploring as well so you know as you probably know we've got grok what being released you know a few months back we've got um open ai i've just released dot which is their kind of equivalent i guess in early development um before that you had open claw and hermes as well is this like muse like the muse yeah from from from that they have a name and i can't it's not an assistant is it what do They call them an agent, right?

22:30They call it a personal assistant, a personal agent. All of these things, what they're trying to do is wrap up and develop the capabilities into a product that non-technologist people can work with. So, for example, right now I'm running GrokBot and I've got bots that are scanning my inbox and helping me to prioritize what I need to respond to, maybe drafting responses to emails for me, archiving things that perhaps aren't important, but being able to give me a summary so i'm already sort of experimenting with a lot of those things so yeah you haven't had a real reply from me no no no we'll dig into it this is the issue we've

23:06Steve Roest: got to go for a commercial break and then we'll dig into this thing so and then at some point we're going to bring it back into health but i wanted to make sure that we set the scene before just zeroing in on on healthcare um we will be right back with uh the next part of today's show the health tech hour with my guest today robin matthews chief technology and product Officer at PocDoc and one of the most widely regarded leaders in thinking on agentic AI and how it can scale digital diagnostic providers. We'll be right back.

23:41Steve Roest: Hello and welcome back to the second part of today's Health Tech Hour with me and my guest today, Robin Matthews, who is the Chief Technology and Product Officer at PocDoc and an AI agentic thought leader so robin before the break we were kind of talking about this whole what happens beyond software engineering and we've got crockbot and these agents and unlocking things and doing all kinds of different things where does that go what you know what are the upsides what are the risks well i think what we're going to see in terms of where it's going to go i think what we're going to see is i think more generalist thought work will increasingly be able to be performed by software agents so what has kind of happened at pace for software engineers because that was almost like the first use case if you like is is slowly beginning to happen in other businesses and you can see particularly for sort of young startups where you know there may be one or two founders involved initially the capability that these bots are bringing their ability to run their go-to-market strategy or to and automate that through the use of bots be able to sort of run their social strategy through bots etc is really enabling these businesses to grow super quickly with very lean you know startup costs and i think we're going to see more and more of this coming into sort of the enterprise over time there's definitely a lot of resistance i think you know obviously we operate in a regulated business and i think you have to obviously be extremely cautious but i think there are also opportunities where you cannot isolate use cases where actually it's a safe thing to do So let's talk about health then, seeing as you've sort of segued us into that.

25:18Steve Roest: Not that we planned that. So when you were, because you're a technologist deep at heart, which is one of the many reasons why I love working with you. You're not necessarily a health technologist. Absolutely. So when you were learning about this technology and obviously working in scaling Pockdog, at what point did you realize that these things could merge together? And at what points did you feel like, no, that definitely can't? Like, what was your sort of vibe on the whole thing? I mean, you're absolutely right. Coming into PocDoc, I had no real experience of working in certainly the health industry and kind of with the regulations that were involved there.

25:54I have worked previously in other regulated organizations, but this felt quite new to me. I'm a bit of a geek with these kind of things anyway, so I spent quite a lot of time kind of reading the regs and really understanding what actually sits behind the requirements. Because I do think one of the issues I think this industry and probably other regulated industries suffer from is kind of there's a little bit of kind of lock into how people perceive the regulation and what they perceive the regulation means rather than actually going back to the source material and really trying to understand it.

26:28So I spent some time trying to do that and that enabled me then to think about, well, actually our product is various different things. Some of those components are definitely medical device. They're part of the healthy heart check that you mentioned earlier.

26:41Steve Roest: Or the diabetes health check. or the kidney health check. You know, some of those components absolutely are fundamental to the clinical safety and, you know, that we take obviously very seriously in the delivery of that product. But there are lots and lots of other elements to our platform that do not form part of that medical device. So for me, it's about taking more of a risk-managed approach to what we're going to do. There are some elements where we still don't feel like we're actually comfortable enough to really kind of let agents loose over the code and we would rather kind of have humans do that.

27:13And I think that's kind of, that's okay. I think it's about taking a risk managed approach to the adoption, which is why I kind of do prefer the term adoption ladder rather than maturity ladder, because I think one approach is just not right for everyone. You've got to really look at your...

27:27Steve Roest: Yeah, maturity implies a kind of judgment, right? It does. And I don't think that's really helpful because I don't think it's helpful for everyone to be at the sort of software factory end of that spectrum or that ladder. um you know some people will will be better off positions you know somewhere else along that um along that journey and we ourselves you know we're we're at different stages for different parts of the business and different parts of the product um and i that's what i think you know other leaders and sort of in my sort of area should be thinking the same way in my opinion yeah i think i was i was surprised when i was speaking to a number of other ceos and ctos of health technology businesses in the uk that's that that work with the nhs at the the the the high level of awareness of ai yeah but the low ish level of adoption yeah or or willingness to sort of try it or even a pathway to try it because and each one of these businesses has ctos has technology teams has an engineering team i think i think there's a few things mixed up into that and and you know i i kind of understand that as well i don't i don't think anyone should be sort of you know being judgmental about that i i fully understand where they're coming from i think one of the issues is that there's a misunderstanding about what we're talking about when we talk about ai and i think very often when you talk about it in a health tech environment people think that you've instantly gone to having ai making clinical judgment because that's where they feel most sensitive probably and most concerned about its use And I think that is absolutely right to feel concerned and have those thoughts.

29:10But that's not kind of what we're talking about here. What I've been talking about is adopting AI and agentic AI to help us build our software products. We still have human control at all elements across our software development lifecycle. And I think that's really important for us and probably many other businesses as well. So I think that's one of the confusions is, you know,

29:32Steve Roest: what yeah what are you actually talking about adopting well when we when we um when we arranged this agentic leadership session that you're hosting later for for ceos and ctos of health tech businesses that work with the nhs we we put some stuff out on linkedin and it was really interesting to see the comments where almost all of the comments that were negatively inclined there were many positive but the negative ones were exactly what you said which is ai shouldn't be used to make clinical decisions yeah and that's immediately where i think a lot people's brains go in health and then it sort of skips this bit where actually you can just use it to massively increase the speed of output for things that have got nothing to do with making a clinical decision 100 and i think i mean there are other areas of sensitivity as well so i think obviously clinical decision making is one i think you know patient data and patient health data is another of course i mean again you know in our environment we don't have any agentic work around any of our data stack at all so you know we're very um you know very sort of um focused on using it for very narrow reasons at this point in time um and i think those things will come i think we will unlock those in the future but i think there's a lot of work that needs to be done before we get to get to that point um but i definitely think you know there is benefit from having um some of the AI tooling applied in these areas but I just think it's going to take a lot longer.

31:02And why is it going to take longer? I think it's because we don't really have all of the tooling yet to be able to perform evals and to really understand whether clinical decisions are the right decision. It's actually you know some of that is going to take a long time to actually understand the impact of some of those decisions. And also that depends on I think that regulation

31:25Steve Roest: has to catch up here really i i really i've got a huge amount of respect for everyone that works in regulators around regulators and i really don't envy them right now i really don't it is so hard it's so fast shifting and i think you know literally we've got to the point now where almost every every month we're getting new models new capabilities new tooling coming out trying to keep pace with that with with regulation i think it's extremely difficult extremely hard so i think And again, I think that's another reason why people are just like, let's just stay as we are for right now. Hypothetically, just as we're all a broad church, I've talked about this on the show before, but hypothetically, a medical device should be deterministic in theory, you know, broadly speaking, however you might define that.

32:13Steve Roest: But if you make the same 10 inputs in theory, you would be able to get the same 10 outputs. And that, as I understand it, is quite hard, if not impossible, for a large language model to be able to guarantee. Yeah, exactly. Being able to prove that, being able to guarantee that is a really hard thing to do. And then knowing how to deal with other situations when it doesn't reach the same conclusion. Because you're, is it true, I mean, this is where we're getting a little bit geeky here, but if you're running off of OpenAI or Claude, you, and you're building a medical device, for example, that uses those models to provide information back, you won't necessarily have access to how that model works, which.

32:53Correct. Yeah. You don't know how to reach this determination. Yeah. Exactly.

32:58Steve Roest: You're just sending data to the model and then receiving it back. Yeah, so I mean we, in PogTalk, we do use machine learning models in some elements of our medical device. But we are very, you know, we've taken great care to train those with a very specific data set and to test them and to validate that, you know, the same result is achieved. Yeah, well there are models as well. And there are models and we understand them and we can draw. There are proprietary models. Yeah, exactly. It's a bit different to just, you know. yeah it's i just i really don't envy regulators i just i'm finding it hard to understand how they can ever get back on top of this which which i don't know it just i mean i think they they tend to like most of the regulations are written from a kind of a principles perspective so it's more about how do you demonstrate accountability how do you validate so i think we can apply the same principles to the way in which we're working.

33:54So an example of that is whilst we use agentic and agents to build our software, our test suite, so the things that we use to actually validate that the software works as intended and against the specification, that's all deterministic. So you can mix and match different approaches within a software factory. You don't have to go 100 % agentic or

34:18Steve Roest: 100 you know other other ways so makes sense we are going to stop for our last commercial break now and then we're going to come back and continue the discussion in and around health because we're the health tech hour um we'll be right back with the last part of today's show with robin matthews ctpo at pocdoc and thought leader in agentic ai and health we'll be right back

34:45Steve Roest: okay welcome back to the last part of today's show with my guest today robin matthews ctpo at pocdoc and agentic ai thought leader so let's before we kind of go circle back around with health let's just get some stuff out on the table so we've seen with just with adopting agentic ai within software engineering in our group led by you we've seen massive speed and delivery improvements yeah absolutely so i mean we we started off with some relatively small scale kind of tests initially so around things like maybe bug fixing and stuff like that we applied we built some agents that were monitoring our logs and then looking for events that were happening that were you know represented a fail state and then we had an agent kind of looking at those being able to diagnose them being able to replicate those errors and then be able to write software and propose a change that a human could then review before putting live even that process has taken our you know our backlog of kind of errors and and issues like that and pretty much eradicated it completely overnight which is crazy you know that's clearing tech debt overnight clearing clearing tech debt yeah to some extent i mean certainly certainly the consequence of tech debt anyway i would say um but being able to do that i think you know i've worked in tens of you know startup businesses over the years and you know everyone's carrying carrying those sorts of issues with them and you know the priority is always on producing new features and things like that so it kind of gets kind of ignored to some extent being able to do that um in that way is just kind of game changing overnight yeah and overnight you know is really game-changing so kind of there are things like that um that you know you mentioned tech debt again so yeah being able to actually take large chunks of our software and be able to completely re-architect it around kind of so we're not changing functionality but we're re-architecting it to remove some of the you know previous decisions and things like that that needed to be updated you know be able to do that in a matter of a couple of weeks for example is kind of like crazy i've been able to have conversations with you that say well actually steve we're going to completely rewrite this piece of software but i would never have had you know the nerve to be able to say that to a ceo in the past um because they think i was crazy whereas now that's entirely possible to be able to do that and that's due to kind of these these new new ways of working well yeah i mean we're at poc dog we're launching in 12 eu markets in 10 weeks yeah which would otherwise be absolutely impossible absolutely you know that's translation service across all of our different you know localization 12x i mean it's it's it's amazing so within health let's focus within health for a second and let's like hypothesize to the future where do you think the ai in health train is going in general open question um yeah that's a good open question so i I do think what we're going to start to see is more and more organizations adopting software development practices like we have at PocDoc.

37:55So starting to apply agentic AI. So what I'm hoping that's going to bring is a fresh set of software that is more capable, more able to solve problems within the system. So I think that's a wave that I think is probably going to happen, I'm hoping, fairly soon, really. I think beyond that then looking more at the sort of clinical decision making and stuff like that that's a wave that I think is going to take several years to come through I'm still hopeful it will because I do think there are a lot of benefits there but as you know as we've talked about previously there are obviously risks and I think you know we need to tread very carefully and we need to do you know proper evaluations of those those moves but I think that will come through over the next sort of probably in the five to ten year sort of um time frame okay next question what throws this off right what the genie won't get

38:51Steve Roest: back in the bottle but what what happens that means that this thing just goes completely off the rails and that future doesn't happen so i think you know one of the challenges is these systems becoming more and more capable and we're hearing more and more news stories about actually you know our model could be could be dangerous um you know we've almost got into this kind of ratcheting up of who's the most dangerous model at the moment which is quite amusing but i think um you know putting aside maybe some of that is marketing but um i think there is definitely we've seen examples where agents have kind of got loose if you like into the wild and and um been able to um perform security checks against systems they were never supposed to perform security checks again for example um i think those things will will form risks i think we all need we've all got an awful lot to learn about how we can control this capability what we need to do there's a new surface if you like a new a new security surface that has emerged in the form of agentic ai tooling which the security kind of industry really needs to get behind and really understand how we can control that i think like this is going to get a bit existential so apologies but at what point do we think an agent so so when when i've read those stories around agents rogue agents on the loose right and then the the i don't know how apocryphal or true this was about the agent setting up their own social network to talk to each other and all these things they're not actually self-aware are they or what no yeah well not in my opinion i mean i think you know you probably need to sort of talk to some computer scientists around around that but i think um uh from from my perspective no they're not self-aware they're they're trying to you know you've given them a task to perform you've given them a goal um you've set some parameters around how that goal should be achieved but there are lots of parameters you probably haven't set for example you may have asked it to perform a security uh test against a set of systems but maybe you haven't limited it to just that set of systems you haven't explicitly told it not to go out further than that and what happens like why does an agent decide to what what actually in the code i mean i'm not a technologist in that sense an engineer so i don't know but what prompts it to it's been told to evaluate eight things and then it just decides to go off and evaluate a bunch of others like why what actually makes it do that because i think it's trying to find the best answer for for you so depending obviously what the prompt is and what you've asked it to do and how you've scoped it you know it's going to try and resolve that to the best of its ability so that's where you know it can go off rogue is the term it's not rogue implies that it's maliciously you know thinking about this which is clearly not the case i think that also doesn't help the debate it doesn't help and i and i think again i think some of the language is evolving really quickly around this and obviously you've got the media who want to maybe sensationalize some of this a little bit yeah of course i mean it's amazing it's like the stuff of like terminator yeah absolutely and you know so i think i think there are kind of lots of those sorts of terms being applied that are not particularly helpful um and give the general public and and others who are kind of involved in this around the the fringes the wrong idea about kind of what those risks are and i don't think that's helpful to you know to regulators or operators trying to implement this no it definitely isn't it because it creates so much noise and it obfuscates the signal yeah yeah which is sort of yeah it does it does um so i mean so i think though that's kind of that's where the i think the where we could get derailed here i think i think we could enter a period where you know as these models become more widely available there's also a huge growth in open source models as well where people can run these things on their own infrastructure structure and without all of the constraints that are being applied to you know open ai and anthropic models etc tell me more so so you know you probably will have you know read a while back you know when um anthropic launched fable 5 it was withdrawn from the market it was um and the reason behind that was that um it became very clear that that model had new capabilities and was able to perform mass sort of security evaluations on kind of any infrastructure essentially and that obviously presented a huge risk to both commercial and public sector businesses so it was withdrawn whilst some controls were put around the capability okay so if you try and make it run maliciously there are controls in place right that will attempt to prevent that from happening that's fine and that's been in sort of enforced on on the big kind of yeah but that doesn't exist in open source are you saying that doesn't apply to people who are obviously being able to like download an open source model um and to be able to run those on their own infrastructure that sounds risky it is it is concerning and i think you know there's been a lot of communications coming from central government here in the uk around this to expect a wave of the security kind of issues coming our way in the next sort of six to 12 months um so we're all gearing up and getting ready for being able to protect against that wow that that's a massive step up in the arms race on cyber yeah definitely because you're no longer you know reliant on having you know one or two extremely knowledgeable people trying to expose you know risks and issues in systems you've now got you've now got agent platforms that could do this on scale almost limit this Across infinite companies all the time.

44:44Across infinite companies all the time and looking for exploits all the time. So, yeah, it's a really big increase, I would say, in the kind of the risk.

44:52Steve Roest: Okay. And how do you think governments should be trying to manage this risk? I'm not going to say regulate the space because I think that's already too, you're leaping to the solution. So how, if you were the governments, what would you be doing? I think they need to work very closely with the frontier models, but also kind of others in the sector as well who are producing tooling in this area. I think it has to be a collaborative effort. I think any form of sort of centralized regulation is probably not going to work. It'll be out of date before you finish it. It will, exactly. The market is moving so quickly that I think it has to be a collaborative effort.

45:31And I think that's kind of what we're seeing a little bit in the US right now. it may not be kind of as sort of smoothly deployed as perhaps we'd hope but i think that's where kind of some of the interventions there from central government um it that's where they're coming from i think they're trying to work more collaboratively um with with with those those organizations and i think that's the model that we kind of need to employ i completely agree so in

45:58Steve Roest: the last few minutes of the show i ask everyone this which is um you know during tough times you know and over the last 12 months you've performed miracles as we've migrated across into the sergentic future but what kind of mottos or self-talk or talk do you give to the team when when you've been asking them basically to take these huge leaps of faith yeah i think it's a really good point and a really good question because i think it's easy for people who are beginning this journey to look at kind of all the material that's available you know on youtube or whatever to kind of see how to implement these things and it feels almost like it's really straightforward and it's just a simple process i think what we've found is that actually it's a very complex process because there's implications for people we've talked about earlier in terms of kind of their motivations and their thoughts towards it there's implications towards process and we're really changing the whole operating approach to how we we make and build product and then obviously there's you know big technology changes happening all the time as well so you can take a step forward and several steps back it's kind of been our experience and i think in that environment it's about resilience it's about giving people the freedom and the backing to know that we're backing their development and their learning in this space and that we know it's not all going to go smoothly and accepting that and being able to kind of give them that space to learn feel comfortable with making mistakes give them that kind of reassurance that it's that's kind of part of the journey that we're on um so for me it it is all about kind of every day just just taking a step forwards and learning something new um in that journey amazing robin thank you so much for coming on the program and i know that you write regularly about this stuff on linkedin right is that correct yeah i'm gonna be i'm gonna be uh starting to write a lot more regularly um i'm gonna my long form kind of essays are going to be on something called unlearning in progress.com uh but i will be referencing those from my linkedin profile as well so if you'd like to learn more about that or follow follow my journey um then please go to linkedin and and follow me or connect there be happy to continue the conversation great and thank you to everyone for listening we'll be back again next week with a new show