BONUS: AI governance - how business can adopt AI safely

0:04Hello, I'm Katie Prescott and welcome to this special episode of the Times Tech Podcast, which is brought to you in paid partnership with ServiceNow. Since we launched this podcast a couple of years ago, we constantly discussed the twists and turns of AI's development. The tech is developing at such a rate of knots and that has huge implications for all of us, trying to understand its potential, trying to use it, trying to make the most of it. But when it comes to businesses, one of the biggest issues that we hear about is navigating this very rapid pace of change. Leaders frequently say it can seem overwhelming in terms of implementation, making sure spending is in the right place, getting the best outcome for the company, making sure everyone is on board with it.

0:46But also we keep hearing time and time again, it's an enormous challenge in terms of governance. So today's podcast in partnership with ServiceNow is going to focus on that issue of governance, looking at how companies can avoid the pitfalls thrown up by AI when it comes to legal issues, copyright, ethics, security, and embrace the opportunities safely. To help us explore this, I am delighted to welcome Matt Hyam, Chief Digital and Technology Officer for UK and Ireland at ServiceNow. ServiceNow is an enterprise software company headquartered in California, which employs 30 ,000 people. And essentially, it sells tech to companies to help manage their back-end processes, anything from HR to IT support.

1:31Well, welcome, Matt. Thank you very much for joining us on the podcast.

1:34Matt Higham: It's a pleasure. Thanks for inviting me.

1:40Tell us a bit about your role at ServiceNow and what do you do?

1:44Matt Higham: So I tend to spend most of my personal time with the boards of organisations across UK and Ireland and global companies as well, trying to help them unlock the difference between hype and reality of AI to help them overcome a lot of the challenges that they're seeing and really benefit from the true value of AI at pace and scale. I talked a bit in the introduction about what governance means and what some of the pitfalls are, but how would you describe it? So governance around AI really is making sure that you know what the AI is doing, what business logic it's executing and the value that you're getting from it with trust and transparency throughout the entire process.

2:23Matt Higham: Maintaining that both to your employees, but also to your customers and to the regulators. And what sort of things are your clients worried about when they talk to you about governance? So what are the big warnings or fears that they have about it? Well, we're essentially shifting our business operating models from very human-centric. What we're now moving towards is a genetic AI that is able to make autonomous decisions, which is really removing the agency from the human. So we have to have oversight to make sure that when it takes those actions, that those actions are done in an ethical, moral, and regulatory correct way.

2:57So that's the governance side. You want to make sure that the AI is not going rogue, essentially. But companies obviously want to use it to make themselves more efficient, perhaps take people out of the loop. How do you juggle that dichotomy?

3:11Matt Higham: I think that's the problem. We keep looking at this as a balance where actually it's a high dependency. If you look at governance, it helps you go faster. It helps you drive growth. It helps you build trust and transparency with your employees and your customers. I wouldn't use an organization's products if I didn't have transparency in how it was leveraging my data or making decisions based upon me. It would be very difficult. In the same way as the way with employees. Yes, we are going to be taking tasks away from humans that are very repeatable, but it doesn't necessarily mean that the role disappears.

3:42Matt Higham: We just move the role into a much more productive kind of capability for the benefit of the organization, the employee and our customers. Can you give us a tangible example of where you've seen this working really well? Yeah, at the moment, agentic AI, unfortunately, is really pointed at the kind of unsexy part of business, so HR, finance, IT, and we're starting to see massive consolidation in that space. So we're seeing cross-functional AI-agentic workflows that are taking all of the pain points out of kind of legacy homegrown processes and create incredibly smooth, frictionless employee experiences for things like onboarding into a new company or a customer experience to understand if something happens in the supply chain, what the effect is on that customer the journey and whether they're going to get their product on time.

4:30Matt Higham: There's thousands of use cases in those things. We're seeing it in health now where it's allowing the NHS to be able to increase the operational efficiency of being able to book appointments. We're seeing lots and lots of use cases that should just remove the friction from everyday life. But how do you make sure that those use cases are well governed to come back to that point of governance? Have you seen some good examples of where those have been implemented, but implemented safely? Yeah, lots. So we make sure that the data that is being accessed is the right data with the right system of record in order for the AI to make a decision on what it's going to reason upon in order to move the action further along.

5:06Matt Higham: Then we create a layer with the actual agents themselves. So we give the agents a set of tools, a set of skills, set of capabilities, and we put a ring fence around that to make sure that it can't go off and do something rogue that we don't want it to. And we only allow it to access that foundational data that we've given it access to as well. Then we make sure that the human asking the question has the right to be able to ask that question in the first place. We don't want them asking something they shouldn't have access to. Give me all the salaries to an entire organization or something along those lines.

5:35Matt Higham: And then the final part of that is we have to control the large language model. So it's okay saying the agent is able to do certain actions, but what we need to be able to understand is the model that's been trained underneath, how is that reasoning at every single stage? And then we append all of that into a database where we can then look at it from an audit perspective to ensure that everything's fully explainable end to end. And then we apply that philosophy, that sort of architectural philosophy and principles to every single use case that we deliver into market. It sounds like you're saying control, control, control at every level, sort of an understanding of what's going on through the whole stack.

6:14Matt Higham: Yeah. And it's the only way you're able to have true governance. If you're only doing one component of that, then you don't know where or how the AI is really reasoning. If it's going off finding spurious bits of data, or if it's going off to the internet doing something that it shouldn't be doing, or it's operating outside of its guardrails, it can very quickly do huge amounts of harm and create an enormous blast radius. And that's where, from an organizational perspective, if you think about the damage this could do to your reputation, to your customers, at mass scale in a very, very short timeframe, it's much more different to when we're operating these businesses just through humans.

6:51And I'm always interested in this. How comfortable do you think companies are diving into the world of AI and using AI?

6:59Matt Higham: It depends on geography, but also on industry. I think we're seeing in the US a much more gung-ho approach to the adoption of AI, a much faster approach to AI. In the EMEA markets, particularly in Europe and the UK, much slower because you've got the regulatory compliance elements there that makes it a bit more fearful. Obviously, the EU's AI Act and the Data Act is very clear, and the UK is following pretty much close suit to that. Organisations in those regulated environments are taking baby steps and they're moving forward in the right way. But also, there's an awful lot of the unregulated part of the business, essentially, that could be driving much faster value from.

7:39I said in the introduction, AI is just moving so quickly. It It feels like a week in AI is a decade in any other industry. Looking back now at the early days of generative AI and ChatGPT, which launched the public in 2022, has the world of agentic AI, as you described it, these agents that go out and act autonomously, come more quickly, you think, than organizations have been ready for or we all anticipated?

8:09Matt Higham: it yes and no i think look generative ai was was great in the first instance and there was that hyperscale adoption we've never seen before as you said but that was because there was still a human in the loop it was a human asking a question write me a poem generate me a picture whatever very controlled almost controlled but then there was still a human decision as to what to do with that afterwards agentic ai is the next generation of that where you've laid on top a set of tools skills and capabilities and access to other pieces of data to be able to actually do some action. That's the bit that really is the high value.

8:42Matt Higham: You've moved the engine into a technology capability, but it's also super high risk because you've now transferred the risk. You haven't got human in the loop anymore. You can apply human in the loop or human on the loop to give governance, but that accountability framework is only going to come from transparency of what the heck's going on within that agentic world. And that's why that governance layer is so key. It doesn't architect you into a corner. It doesn't over apply too much governance to prevent innovation. It drives the innovation faster. And do you think within companies that needs to be then one individual who's responsible for AI governance?

9:18And does that sit with the CTO?

9:21Matt Higham: No, this is one area I love talking about, which is we are still using a 2D hierarchical organizational structure to challenge a 4D world that's operating in real time. So we have to fundamentally look at the organizational structure and the accountability layers within the business. And it absolutely sits at the heart of every single role within the leadership team, because they're ultimately the ones the regulator wants to pin down if something goes wrong and potentially sent to prison. The areas within the business as well, you're going to have people that are going to be governing potentially 5, 10, 15 ,000 agents.

9:58Wow, yeah.

9:59Matt Higham: Operating on behalf of them in their role and they become the governance layer, protecting the organization, the customers, the employees for the regulatory compliance. So you're not managing people anymore, you're managing AI bots. You're managing a bunch of agents. Yeah, exactly. You could still be managing other people because there may be other people in the human in the loops and you'll be applying human in the loop governance as well at certain points during those stages. And that's the difference. But that's a fundamental shift in operating model, organizational structure accountability reporting lines but what we're still doing in most organizations is trying to trickle this stuff in but maintain an old hierarchical 2d model and it doesn't does there need to be one person who oversees ai and who you know is ultimately sort of it'll it'll the buck stops with that person from a technical perspective potentially but from a governance and accountability it's a tool so it's executing business outcomes on behalf of the business.

10:55Matt Higham: So the roles that would sit within risk, finance, operating models, all of that, the chief operating officer, they have to be accountable for the outcome and the impact that that has both in market and to their customers and to their employees. And you're in a really unique position here with the number of clients that you have and speaking to so many boards about this. How many people are making those sorts of moves towards becoming, as you describe it, a more 4D organisation? Surprisingly, not enough. Doesn't surprise me actually. Unsurprisingly, not enough. This is moving so fast. I've never seen a technology, certainly in my career or anything else that has really held the conversation for this long.

11:35Matt Higham: Most boards aren't technically adept, really. In fact, such a low percentage of boards actually have a CIO or a CTO or a CDO on their leadership team. They usually report somewhere else in the organization. It's a technology problem. They'll deal with it. That can no longer happen. This is a fundamental shift in that. So actually, what I am seeing is more boards are becoming much more adept in the knowledge and understanding of what this technology is. And now they're trying to figure out how they apply that in a highly governed way within the market within which they operate. And that's where we can support a number of my colleagues.

12:12With this pressure that I hear all the time from boards to speed up, implement the technology as quickly as possible, you know, install these efficiencies, which is a quite difficult circle to square.

12:23Matt Higham: Yeah, because it's the competitive advantage. Your ability to be able to respond to the market, grow your market opportunity is now becoming at AI speed. You've got unicorn small companies that are able to do the same capabilities because they're built from the ground up on AI to operate like they're a large scale organization. So it's no longer that your threats and competitive landscape is coming from the next biggest organisation similar to you can be coming from anywhere. And that's what's really making organisations make decisions probably slightly in the wrong way without that governance foundation in place means they can't go as fast as some of their competitors.

13:04And you've talked about the lack of understanding of the technology. Do you think there's an education piece here? Do you think boards need to invest in making sure that they do understand what's going on, even if they're not going to the extreme geekery of it?

13:19Matt Higham: Absolutely. And I think the skills problem is a big one. The skills and understanding, but not just at the board level, in every single layer of your organization, you have to have an open and transparent conversation with your business, with your employees about how you're going to use AI, why you're going to use AI, build a culture of AI in your organization on how they can maximize and leverage it, but put it in the right place. Put it in the point of where work is operating rather than just help you write emails. That's not going to make a paradigm shift to your business, but do it with a governance framework that's hard and is capable inside your organization.

13:56Matt Higham: It's not just a policy document that's in a PDF that people can read. That's not going to help you govern AI when it's operating systematically across every aspect of your business. So how does ServiceNow come into this? How do you help companies write governance structures? So our platform is the governance structure. We have built a capability that allows us to be able to give you transparency and data provency over your data ecosystem. The platform that allows you to be able to manage any AI in your enterprise on any platform and govern that, and then have the capability to be able to demonstrate transparency and trust to every member of your organization so that you can springboard forward and unlock the value that you want to.

14:40Matt Higham: And we place that right in where the work is being done within the organization so that you're able to trigger the value that you want. So if you had to, I don't know, give three pieces of advice to business leaders who are thinking about governance when it comes to AI, what would you say? Get it in first. It's the first foundational capability that allows you to be able to go faster in the long run. Without it, you'll just be doing lots and lots of pilots and hitting walls and hitting roadblocks. It becomes incredibly scary. And also the accountability of this thing sits actually at the heart of the board.

15:15Matt Higham: This isn't something that you can just layer in as a technology delivery capability. This is a fundamental shift in your operating model. So treat it like that in the same way that you would any other major operating change in line with the regulatory policy under which you address the market to. So, yeah, it's a fundamental foundation that allows you to be able to springboard faster and leverage the value for both cost extraction of your business and operation efficiency, but for the growth opportunity that you can go and address in market as well.

15:49Matt Hyam, thank you so much for joining me on this special episode of the Times Tech Podcast, brought to you in paid partnership with ServiceNow.