Fast Enough to Matter, Careful Enough to Trust: How Leaders Are Governing AI

0:07Welcome to Data & AI Mastery, the podcast where we sit down with the data and AI leaders shaping the future of business. I'm your host, Raul Gabriel Irma. Today, we're doing something a little different. One of the things I love about hosting this show is what happens when you step back from the individual conversations and look at the bigger picture. Patterns start to emerge, themes that keep coming back across different industries and different companies, no matter what stage of the AI journey they find themselves on. This episode is built around one of those themes. Today, we're focusing on strategy and governance, the decisions that happen before AI gets deployed and the frameworks that make sure it keeps working once it does.

0:55Because there's a real difference between an organization that is experimenting with AI and one that has genuinely thought through what that means for its strategy, its accountability and the people it serves. And that gap, in my experience, is where a lot of the frustration lives. You're going to hear from Luke Pearce, Chief Data Officer at Santander UK, who has one of the clearest frameworks I've heard for presenting AI governance at board level. You're going to hear from Miriam Salah at Vodafone 3, who took one of the most deliberate approaches to building an AI strategy that I've encountered.

1:34But also from senior leaders at organizations including Aviva, Oxford Site Business School and Intact Insurance. All grappling with the same fundamental question. How do you move fast enough to stay relevant, but also carefully enough to stay trusted? There are no easy answers here, but there are some really good ones. Let's get into it.

2:01Completely different from anything that we've done before. We sat down as a team and we challenged ourselves to say, what we know is artificial intelligence is really going to transform our business, but we can't see it, obviously. We don't have a crystal ball, unfortunately. And we know that it's going to impact the core business model as well as our operating model. So let's start, but what we feel comfortable, we know. We know that we're going to have a business and we want it to be thriving. We're going to sell, we're going to build, we're going to run. It's one of the key pillars of what we do as a business and any business.

2:35In fact, you can sell either the selling part is going to be to a consumer or to B2B or wholesale, etc. It's not as important. But these three key pillars we thought were a good foundation to start. And then from that, we thought about where do we see kind of the homeostasis between artificial intelligence and human intelligence? because we do believe, and we're seeing it now as we're deploying AI capabilities organically, we know that there's going to be a working space between the two. And then from that, we wanted to come up with percentages in the self space. What do we expect the percentage of human versus AI in the built place and in the run place?

3:21And then we expect that there's still going to be some supporting and central functions. This is everything to run your business, your HR, your finance, your legal, etc. Again, in that space, we wanted to come up with that percentages and how they work together. That was our starting point. Then the second part was, and now we need to figure out what our customer experience is going to look like. I spoke earlier about that digital transformation. The anchor of a digital transformation is personas and customers and what they expect from us as a business. Now, our customer can become an AI, an agent, as well as a human.

3:59We never had any thought, any thinking around this. So that's something that we need to take into account. But now also some of our employees are going to be agents. So agent to agent, how does that fit in within, you know, the customer experience, which normally drives the outcome of any transformation? And then we had an idea of what we wanted to achieve in the next few years. That was the first part of our AI strategy. So we decided that the right thing to do is let's go on a roadshow. We have amazing partners, big tech, consultancies, advisory. Let's go share a view of where we are with our AI strategy and let's gather some feedback.

4:40Let's make sure that we get to a point where it's never going to stop and we need to continue being agile and evolve it. But at least let's get to a point where we feel like it's best in class because we have 20 of our partners support us through that and give us feedback. So we've done that. And now what we're doing is making sure that we focus on the actual execution plan. Now, the execution plan is, as you would imagine, firstly, the operating model. So how are we going to be set up as a business to support the human and AI working together? And I believe here that the change is going to be the biggest, right?

5:23Because not only do you need to bring people on the journey, you also need to actually rethink which roles will be augmented through AI, which roles will disappear. There will be roles that are going to disappear, which roles are going to appear.

5:41It is a balance is the first thing to say, but you set really clear guardrails. So actually the outcome that you're trying to get to, so when you say we want to get a customer benefit, we want to do a really improved service or value proposition for a customer, but it's always with a, and we need to do, make sure that we do that safely. Because if you go for a sort of a quick buck on something, which isn't what we would do, it's not what we're about as an organisation, it's about that long-term value, then actually you won't get that if you do it without kind of the right controls and the right guardrails being in place.

6:13You might potentially lose customer trust. And that's something that we would never be comfortable doing. So you architect right from the start and you don't look for an either or trade-off. You look to say, how do I reach that customer outcome with the right guardrails and controls and protections in place? And actually, if I can't do it with the right guardrails, controls, protections, then it's possibly not the right thing to be doing. And I think when you're very clear in terms of what's the outcome you're trying to get to and what does good look like, then actually it's not a trade-off because they're one and the same thing.

6:47They're all requirements to get to a quality outcome. If I turn around to a customer and said, we've done this thing really, really quickly and really, really cheaply, but actually we didn't protect your data along the way or we behaved unethically along the way, then that's not a good customer outcome. So actually it's a total failure. The good customer outcome is we've delivered something to you really quickly and we've delivered it to you at a really good price point. Your data has been completely protected throughout that journey. We've managed it in a sustainable and an ethical manner. That's the customer outcome that they actually wanted.

7:21and if you define it in that manner right from the start and you architect in that way right from the start then actually you're generally in a you're generally in a good position there's not a trade-off that that's how you do it well so what does it take in in practice to you know implement a sort of guardrails the sort of red lines how does that cascade through in in the organization when you know individuals are implementing projects would love to understand what does it take? Yeah, so you have to think broadly. So technology, people and process, you need to think about all three. Certainly for us, it has been core that you architect in the right guardrails and the right controls from day one.

8:04And so when you build out whatever the technology solutions are that you're using, how are you going to use the LLMs in your organization? Actually having those guardrails in right from the start is really important. The way that we approached that was actually that we built our own platform. We didn't build LLMs, we utilized kind of the industry-leading LLMs, but we built a platform that meant that all of our scale productionized solutions are built out from that platform. And that inherently comes with the guardrails associated. So we made sure that, you know, we worked with the right third parties.

8:38We made sure that actually we put in, you know, transparency controls, hallucination controls right from day one. So you do have to architect in that manner and you do have to invest and make sure that you set up in that way. You then need to think broadly. I've mentioned kind of process and people. They are all part of the solution. So how do you educate the people who are going to be using the different technologies or the different solutions? How do you make sure that you are helping them to understand a little bit about AI? How do they use these things? What should they see? if they see or think certain things, what do they do?

9:16Making sure that you've got processes aligned to that. So you do have to think broadly around it, but I don't think it's any different necessarily to delivering any other key transformation. You still have to think about people and process and technology. This is just a different technology that you're working with that comes with some different challenges and some different risks that you need to understand. and you need to make sure that you factor those in right from the start.

9:48Yeah, I think it's an interesting one, right? And I've been reflecting on this in the last few months. I've worked on programs where they were very waterfall and I've worked on programs where I pushed a lot on agility because I think it is really, really important. But I do believe, interestingly enough, that in the era of AI, we need to create a new way of managing change and governance that is a bit of a mix between both. I think the waterfall space, and I'm not talking about like really rigid waterfall, but kind of a little bit more gateways when it comes to anything that matters more, like the foundations, security, privacy, ethical, responsible AI needs to be a little bit more waterfall and very thought of and managed appropriately.

10:34And then on the rest in the implementation, in the actual POC and before kind of production, it's good to be agile because you can test, learn, and change things as you go. So I think a little bit of a hybrid of way of thinking and doing things is going to be required because we can't deliver either or for sure is what I see. That's pretty cool, like a governed agile approach. Exactly. Yeah, we talked a lot about agile, which is like waterfall and agile. Agile. I like that. Yeah. Is it because AI presents risk in terms of, you know, it can get it wrong, decisions could have an adverse impact on the customer?

11:14So I guess it kind of raises the question, when do you decide that, you know, an AI system is ready to be in front of a customer versus we're going to keep putting guardrails around it, you know, this waterfall approach just to minimize risk, but as a result, you might not capture the whole customer experience, right? that you could get out of the technology? Like, what do you think about this dilemma, I guess? Managing risk, right? We manage risk all the time. It's not just with AI. Throughout my career, we've managed risk through committees where we'd look at the pros and cons, the risk level.

11:47And then from that, if you have the proper mitigation in place, mitigation would need to be delivered within a specific period of time. And in general, it's a committee where you have proper representation from privacy, from security, et cetera. But what's for sure is that you never take any risk when it comes to a customer and impact on customer. As much as possible, if you can do it for things that you can manage internally, test and learn and evolve it. Absolutely. Definitely. That's fascinating because you've got the FOMO and the FOMO, right? There's the fear of messing up. There's a fear of missing out.

12:21And when you're a large organization with obviously lots of customer and regulated.

12:30it is interesting because i think even at the the board level there's an appetite to use ai and to start seeing how those things can be used now obviously that's that in itself comes with a lot of challenge but i think it means the conversation is easier because this is this is something and i've talked about this this a lot with with people on yeah ai i believe will will become like digital so you know when when we started putting things into the um into the app space people were on their mobile phones. It was a thing in itself. And we all set up kind of digital capability. And then it just became part of the everyday.

13:05I think AI would be exactly the same. And because people are engaged with it outside of the working environment, they come in, you can have a conversation with people and they start to understand it. But they also start to understand things like hallucination and what can go wrong. So I think the conversation with the board on the one hand, which is, yes, we want to get more involved. We want to start to use it. On the other hand, it's how are you showing that control? And that's a really difficult thing to answer at the moment. Without having to understand how models work and what you're doing, you've got to break it down into something which I think is consumable by people working at that level.

13:43And what we've tried to do is look at, because for me, I think the idea that you keep talking about you know the the hallucination the one-offs and the outliers is is the wrong conversation to have and i've said many times that if you have people will go on to our you know chatbots on various things i was on a chatbot the weekend trying to get um you know some stuff so with a um with a like a mobile provider the chatbot was awful and it gives the wrong answer but no one really gets you know because eventually you get put through to a person no one really minds on it as soon as you get into an ai chatbot which gives a funny answer then everyone says oh it's hallucinating it's going to give like all sorts of bad things and and so there is a panic almost setting in that it's a very emotive subject and i think we need to move away from that guardrail discussion on you know we will manage that it will produce a good outcome as in it won't produce something that's that's hallucinogenic but we also need to show you the controls that we have in place across the flow to make sure you understand that and you understand that actually we're monitoring these things on an ongoing basis.

14:48And the way I try and break it down is, operationally, we need to make sure that we're calling these things in the right way. So you can look at, if you're linking AI together and you're calling a model in a number of different ways, if it starts to do it, the patterns start to evolve and start to do things differently, then that's worth an investigation. So we call that the operational metric. We also need to make sure it's being fed with good data. And I think that's one of the most fundamental parts and the crossover between the governance side and the AI side is if you put bad data in, it's almost exponentially worse than what it used to be of what comes out the other side.

15:22So make sure you have your data controls. And then the third part, which is let's look at outcomes and let's make sure there's an oversight in terms of making sure that the data we're feeding in is producing the outcome and we're looking at the outcomes and checking it. And that could be, I mean, it could be some partial manual checks that people are doing. There could be AI as a judge and those things will start to evolve over time. But I think those three key parts of making sure that you can describe the way the oversight is working is a much better conversation to have with a member of the C-suite, a member of the board, so that once they understand that, then they can see, okay, well, how many things have we got live?

15:57How are those controls working? And do we have confidence that they're working across the organization? I think that's the only way that you scale. And that's the only way that you then start to get into a conversation around agentic or something a little bit further, which is, okay, can we put this now in front of a customer? Do we trust it enough? But it is really that core conversation, I think, is so important at the moment.

16:25I think the approach that Oxford and the Business School has taken has been a very conscious, deliberate one. And it's almost been our strategy comes with a starting point of being equitable, inclusive, and providing access. Because our students, our learners, our professional services staff, our academics, our faculty, and our researchers need access to these tools. And part of that strategy is acknowledging that if we don't embrace and provide them in a supported, maintained, secure way, people are going to use them anyway. They're out there. So you can either take a head in the sand approach and pretend it's not going to happen, or you can get on the pitch and adopt and embrace it and work with those personas to work together, acknowledging with the fast-paced change of technology, we are all going on a journey together.

17:17So that's a strategic starting point. How do we choose? I think the honest answer to that, Raoul, is we've made a conscious decision almost not to choose. We've adopted a portfolio approach because how quickly things are changing i can't sit here today with my hand on my heart going this is the right tool for the right job and i can't with my hand on my heart say this is where this tool is going to go versus where this tool is going to go so we have deliberately consciously made you know a portfolio decision to give all of our personas access to a range of tools and knowing that we've architected it privately securely within our own domains and tenancies try to promote an approach of please feel free to experiment, innovate, and play.

18:03Yeah, that's a really refreshing perspective. And I'd love to deep dive a little bit because I speak to many organizations and more on the enterprise side. And they're struggling with this idea that if I put all my money into Chagd OpenAI, risk of losing out on Claude and Gemini that looks like they're also making good progress. But if you wait six months later, the picture might be different. So I'd love to hear from you. What do you think other organizations can learn from what you're doing at Oxford, but also balancing the capital investment, right? To give so much choice to so many stakeholders like you are surely comes with a big investment, right?

18:47and that's a real consideration and balance of that decision and that approach Raoul and you're absolutely right but I think that's where the shift for us at Oxford and and let's be honest if you're Oxford or Cambridge there there is a certain brand association and value with those names that is very attractive to partners so I think we should be honest and upfront and acknowledging that that said we are not immune to the financial pressures and cost pressures the same as everybody else and value for money is a really important aspect. Financial sustainability is a really important aspect and even as Oxford we would not be able to enter in and support the more traditional models of individual licenses per user per month.

19:34That's just not financially vulnerable or feasible. So we've attempted to be slightly more creative in terms of moving more towards access by consumption models as opposed to licensed models. So basically you can have access and if you don't use it, it doesn't cost us. And then we know that we are only paying for what is being leveraged, used, exploited, adopted, which is hopefully driving benefit. And that has allowed us to approach this portfolio approach to our AI offer. And so far, even though we're still continuing to learn every day, every week, every month, that has proved successful.

20:18I mean, we did take quite a good approach at the start. So I think a lot of organizations, you know, the AI is there. They say, let's find some use cases. And you get a list of a thousand use cases, and it's very difficult to digest them all. I mean, I think probably everyone goes through that first process, and then you're having lots of conversations. You're not actually progressing anything because you're trying to work out what that killer use case is. I think we took an approach early on to bring a domain approach together. So we effectively, rather than say, individual, anyone can bring a use case, we said, let's focus on a domain.

20:55Now, as we all know, when you build a model, that's only part of the story. So if you're going to build a model, you have to integrate it into a front end system. You have to then bring it into an operation. Those are fundamental parts and they can often take longer. And equally, you've got the governance running alongside that. So the point of that approach was to say, we probably have a limited number of core systems in each of the domains. And we have an organization that where if we start to build an understanding of how the AI will work and how it will affect them, it becomes part of that cycle that they can get used to doing training, understanding the models, understanding their role to play in the process and really building teams that cut across the different parts of the organization.

21:45So we're talking about pivot to agile at the same time, but it really doubled down in terms of the AI flow. So we had the business, the architecture teams, the technology teams, the data teams, including the data science and the engineering teams, all working together and effectively building out a roadmap for how we would land AI. And so then it becomes much less of a surprise because you have teams that are ultimately, and this is where I think AI is really interesting, teams that understand the regulation, if it's like a financial crime, people who understand the operation because they're working on it day to day, they are key parts to shaping the way the model works and bringing them closer to the data science team.

22:32And the technology teams means when it rolls out, then it rolls out in a much better way. So I think that was a key part for us. It's really about making sure this isn't something that is driven by either technology or data and us assuming that the answer is going to be perfect. It's about making sure we have that joined up team approach and really being challenged probably by the business and the operations teams from start to finish to make sure that what finally gets implemented is going to be successful. And equally, I would say that even when you think you've got the perfect solution, And, you know, doing that first round of testing and making sure, you know, it's probably rolled out with a few really good experts in the organization means that then you really start to do the live testing, you know, with oversight, with challenge.

23:25And so when you get to the maybe the scale of rollouts, you really are in a good situation. And it's only with that skilled individuals who are business SMEs that that really works effectively. And so I think that was probably the best part of our success.

23:46What does success look like, you know, fast forward a year's time? Actually, this is very interesting. I asked the same question to my manager, the CIO, right? You know what he told me? It actually resonates with me so, so well. Basically, he said, if in one year's time, if we can tell a data story so simple that the business will be able to say, yeah, I want more of that. Can you do more things for me? Right? Rather than saying, oh, I don't understand what do you mean by this data management? What do you mean by like, like ownership? What do you mean by stewardship? What does this tool do? Instead of that, if people can understand the data story simply and can understand the benefits we're bringing to the table tangibly, and that's, that's making making data simple.

24:34That's success for me and the team in the next 12 months. Super clear. That's great. It's nice to think about it from, I guess, an outcomes point of view rather than kind of internal complexity and technicalities, you know, which often people don't have to appreciate anyway, right? No, absolutely. It's not just Excel sheets and spreadsheets, right? Data is beyond that. Data is not building dashboards. More often, we actually put more data in the hands of underwriters, claims processors, pricing analysts than necessary. And then you are actually in a decision paralysis, right? Like, how am I going to make the decision?

25:14So having the right data at the right time with the right people in the right possible ways, you know, everything has to align for you to be able to make those, you know, informed decision making faster. And I guess, how do you balance this desire for simplicity, but also knowing that the insurance industry has a decent level of compliance and you need to have responsible governance? So one could argue that those are kind of like separate concerns, simplicity versus making sure that things are done the right way, which adds a bit of complexity. So how do you think about balancing those two views together?