Your AI agents need performance management, too

0:00Kate Smaje: As you start to think about your performance of your agents, if you're not having a conversation about how many agents do I actually have, who is responsible or accountable for the performance of those agents? What value are they actually creating? God, I start having those on our non-human workforce because it's real and it's big.

0:21Senior Partner: Hey, everyone. That was McKinsey Senior Partner, Kate Smaje. She's here to talk about what leading through this AI moment means for the relationship between us human talent and our digital co-workers. This is McKinsey Talks Talent. I'm your host, Lucia Rahui, with talent experts Brooke Weddle and Brian Hancock.

0:47Senior Partner: Kate Smage, welcome to the podcast. Thank you for having me. And congratulations again on the second edition of your book, Rewired. Takes a village, but appreciate it. Bergen, Brian, pleasure as always. Great to be here. Great to be with you. Let's get into it and start with some context. Leaders, frankly, have so much going on in today's world of, you know, head spinning disruption. direction, if I am a leader, how much do I really need to upskill myself and get into the nitty gritty of AI with everything else I have going on?

1:25Kate Smaje: Look, I think you have to, right? I'd love to be able to say, hey, don't worry about it at all. But it really isn't the case. And the reason is, you know, you kind of got to put your own oxygen mask on first if you're actually going to help your organization embrace this moment. And it is intriguing to me, actually, even if you look at at CEOs, they'll often spend a lot of time saying, my organization needs to do X or Y. I need more people to be more tech proficient, more fluent in what they do. And my first question is always, that's great. Agree. What are you doing? Like personally, what are you doing?

1:59Kate Smaje: What's been your growth? What's been your learning journey? How do you learn? Do you learn differently to how you did before? And it's intriguing to me that, you know, there are, of course, some leaders are like, well, let me tell you. And you have this amazing conversation for an hour. But there are many who say, oh, well, I didn't think it's about me. As with any topic, if you're really going to lead your organization on it, then you've got to lead from the front on it.

2:24Senior Partner: And we've heard again and again, speaking of leading from the front, that this kind of transformation is not just a tech transformation, but also a business and specifically a people transformation, I believe the way you put it in Rewired, was that the heart of every tech and AI transformation is a people transformation. If that's the case, why are so many companies falling short at building the in-house talent they need to turn that tech into real and meaningful business advantage? What are they doing wrong?

3:00Kate Smaje: Yeah. I think the first part of the why is because it's hard, right? Technology is the easy part here. The hard part is, can you create the organizational capacity and capability to absorb that level of change? And that is the hard stuff. You know, what are they getting wrong? I'm going to say maybe a couple of things. The first is a pure mindset. So exactly as you say, Lucia, if you think about every AI transformation being at its heart of people transformation, it's a very different ingoing mindset to assuming that this is the next rack and stack of all the technology spend that I need to get in place and I need to roll out onto my organization.

3:39Kate Smaje: It's just a different mindset. So number one is actually flipping the paradigm on that already puts you at a significant advantage, I would argue. And then the second thing that I think people get wrong is spending the time on the talent transformation means how many new people can I bring into my organization? And we've all done it. You celebrate the number of hires coming in, the places that you managed to poach them from, and that's the kind of high five moment. But actually, the high five moment needs to be, have you made these people wildly successful? Have you got the friction out of their day-to-day so they can actually thrive in the job, not just survive in it?

4:23Kate Smaje: And that's sometimes things like, if every time I need to spin up a new instance, I've got to go through five layers of procurement red tape. that's a problem, right? If I need to be able to pay my incredible data engineer three layers below me more than my own salary, that's a problem. It breaks the comp model. It breaks the career pathways. And so rather than focusing on, you know, how much new talent am I getting through the door, focus on, am I setting that talent up to be wildly successful once it's here?

4:54Senior Partner: How do you think about talent that's within the traditional tech organization versus talent in the business that now needs to be even more tech-enabled and technologable than ever.

5:07Kate Smaje: I think one of the other mistakes or misnomers is forgetting that some of the most valuable talent you have is actually your core business owners, your domain owners when you're doing this change, right? The folks that really understand a level of depth, how does this place actually work? What's the subject matter expertise, the wisdom, right? The learned pattern recognition that you need to kind of harness. And so this isn't all about, again, not all about new talent coming in and also not all about the technology talent, but actually how you transform that business talent. Because, you know, as one client put it to me, it is far easier for me to teach my metallurgists AI than for me to teach my AI specialists metallurgy, right?

5:50Kate Smaje: And every organization has their own version of their metallurgists, right? and figuring out how do I actually make sure that that incredibly core, intrinsic, un-repeatable capability is actually thriving.

6:08Senior Partner: Kate, what have you seen as the role of what I would call ways of working, but think like the rituals and the routines around work? Even in the context of McKinsey, we have changed the ways teams operate to be able to embrace new profiles, as you're talking about, and AI overall. What have you seen really work in that context?

6:27Kate Smaje: Well, I think this comes to the core of it, right? Ultimately, it's not about sort of putting a technology lick of paint over, you know, the business that you have today. It's about saying, actually, what's the new? Let me reimagine for a second, what is a better way of being able to do this? Maybe a way where human capacity is no longer my constraining factor when, you know, What was top quartile is now hygiene, so I'm going to raise the ceiling again. And as you start to reimagine your business or parts of your business, domains within it, that forces you to change the way that you fundamentally work.

7:04Kate Smaje: So for us, even if I think about our team rooms today and the way in which we problem solve, we are doing that with AI. It's not being done by AI. And it's one of the core mantras of our product teams is not AI that does it for me, but AI that does it with me. And the reason for that is we're spending more time thinking about, well, what are the rituals that we have? What are we actually trying to do? What's the capabilities that our people need that, yes, is amplified by tools, but the point is not the tools, it's the capabilities. And if you start to rethink those rituals or those habits or those workflows in a more corporate context, that's how you start to unlock value because you've got to redesign those for the role that the technology is going to play rather than just assuming that the technology creates that change itself.

7:56Kate Smaje: It doesn't.

7:56Senior Partner: I think this is why we keep using the word reimagine, right? Reimagining a workflow, reimagining a domain, reimagining, to your point, an entire organization. and I think and some people might say wow that sounds exciting but it's actually really hard and so what we're seeing now is how do you actually reimagine so in the context of a workflow how do you get the right kind of cross-disciplinary cross-functional talent to get in a room and really start with a blank sheet of paper so you don't tech enable a bad process how do you use red teams right to really challenge and say but could you think about it this way yeah and also

8:32Kate Smaje: I think what belies sort of or is underneath the hood of that is, you know, organizations aren't static anymore. I mean, arguably, maybe they never were, right? But they certainly aren't as static as we have treated them in recent years. And so even your reimagined moment and the descriptions you had there, there's an implicit assumption that there's some kind of from to. And actually, the two, I think, is a lot fuzzier and a lot kind of, you know, malleable, more malleable than it has been previously because, you know, the technologies are going to change. The bar will keep rising. You know, what we understand about, you know, cognitive load and brain health and so on will continue to shift, right?

9:14Kate Smaje: All of that is going to keep moving. And so in that context, I think, you know, I just think it's a much more dynamic way of thinking about organization.

9:23Senior Partner: Exactly. And to that end, you know, in several of the HR clients that I'm working with, the way they're thinking of the two is moving from centers of excellence or centers of expertise to actually, you know, employee journeys. And so what they're doing is they're loading their organization with people technologists that can understand and partner with the deep technical experts, but can actually on their own do using low-code, no-code tools, do a lot of the rewiring, not once, but constantly as the businesses change and as the priorities change. Is that kind of shift to having the business owners, in this case, HR being a business owner, taking on more of that ongoing design once you've got an agentic platform in place?

10:11Senior Partner: Is that something that you're seeing broadly across organizations?

10:14Kate Smaje: Absolutely. It's particularly in the more successful ones, right? Because what I think you've really hit on there is it's about building the muscle of the organization to do this again and again and again. And that was kind of the core of Rewired is, you know, in many ways, we're going to be transforming for the rest of our lives. This isn't a one and done thing. I actually don't like the word AI transformation or the term AI transformation, because it conjures up that at some point we're going to be done, right? We'll be transformed. And we'll all high five or walk off into the sunset and life is good.

10:45Kate Smaje: And it just doesn't work like that. The other thing I would say as well is, you know, I do think, and I know that I'm speaking to a converted audience here, but, you know, the role of the CHRO is really critical. And I'm struck and I probably drive our head of people crazy, but I was reflecting the other day on where am I spending my time? Am I doing the right things? Am I spending my time in the right places? And I found that actually the person that I spend the most time with and the team that I spend the most time with was people. No question. And I think there's something in that of that fusion and that relationship, actually, of people and technology is more important, I think, today than it's ever been.

11:29Kate Smaje: You know, on both sides, I think your technologists need to really understand what does it take to move people? What does it take to bring technology that changes organizational capacity? And at the same time, your people people need to understand how does technology change what the art of the possible really is and does? And what does it mean for the way that our organization is going to work very, very differently at multiple different layers as we look forward?

11:54Senior Partner: To that end, I was with a group of private equity CEOs, kind of a portfolio meeting. And one of them said that they've renamed their chief people officer the chief performance officer. And the reason for that is because AI was going to significantly change their business. And the hard part about it was actually the people side. But if they got the people side right, they were going to be able to, in very specific, tangible ways, improve performance of individual managers, of teams, of units. And so they relabeled the chief people officer, chief performance officer to really drive home the point that you were just making, that people is key to the new way of working.

12:39Senior Partner: The other thing that brings up for me, Brian, is one thing I've heard you talk about, Kate, that I love is when you think about performance management, HR is no longer just managing the performance of humans, but also digital workers. And there is, I would say, a bias today that any agent is a good thing. And it's simply not true, right? Agents need to be kind of fine-tuned over time and in some cases maybe sunsetted given business need and given the way that work is happening. I wonder if you could talk a little bit about that and this concept of performance management, not just for humans, but actually for digital workers or agents as well.

13:21Kate Smaje: We know how to manage the performance of our human workforce. We know how to get the best out of them, right? You know that you're going to hire them for a certain set of skills. You're going to develop them over time. You're going to make sure that their emotional and mental well-being is strong. You're going to make sure they're fed and watered. And when the time comes and maybe they're ready to retire or maybe the performance hasn't been good enough, you've got a process for that as well. What is interesting is actually when you ask the question of your organization of when did we last think about the performance or manage or have a conversation around the performance of our non-human labor.

13:59Kate Smaje: There aren't many organizations that can stick a hand up and go, oh yes, last Tuesday, we were all in a room talking about the performance of our non-human I don't come across them. Maybe you do, but I don't come across them. And when you really step back and you think about it, your agents, for example, they will need good engineers around them to fine tune how they work and so on. In many ways, that's their management. They will need access to compute because that is their energy. That's their fuel as they think about it. They need to have strong lifecycle management around them because actually they can't just exist in perpetuity.

14:35Kate Smaje: And what was a wonderfully fit for purpose agent six months ago is a whopping great liability possibly today, right? In a post-mythos kind of a world. Or my new favorite term that I will steal from a colleague of mine who talks about the abandonware in our networks, right? Things that have been created, but actually now no longer should frankly exist, or someone's sort of forgotten about, and you end up with this abandoned where everywhere. I just loved that term. And so as you start to think about, you know, your performance of your agents, if you're not having a conversation about how many agents do I actually have, who is responsible or accountable for the performance of those agents?

15:14Kate Smaje: What value are they actually creating? So in the same way as, you know, for years, we've had wonderful conversations about talent to value, wonderful conversations about the ROI of human capital. God, I start having those on our non-human workforce because it's real and it's big.

15:32Senior Partner: And who do you see taking responsibility for those agents as they start to proliferate across organizations? Who's the right person to take ownership if there is one? Yeah.

15:43Kate Smaje: I would love to see the business owner who is using that agent for whatever workflow they have should be, you know, just like it's, they're the manager of the humans for that part of their business. They should be managing their agents as well. I don't think that this is a technology problem, right? I don't think that the CTO suddenly needs to manage half the workforce in the organization. Let's say you're in finance and you've got a bunch of agents there that are, you know, that are helping you with revenue reconciliation, helping you with month end close, helping you with, you know, data reporting and so on.

16:16Kate Smaje: Then actually that the head of finance that is looking after the humans in that process should be looking after the agents as well, is my humble opinion. Now that means you'll also need guardrails and standards and lifecycle management that's clear across your organization so you have a shared risk posture around it. But I think increasingly, this will be federated to where it is needed, just like we've seen that shift in other forms of technology and data and so on to be federated to where it is going to be used.

16:46Senior Partner: And to that point, in HR in particular, what we're seeing is some of the people that are deep HR experts, the org psych PhDs, they keep us out of trouble in our assessments, need to understand agents so that they can supervise them and make sure that the guardrails are in place and that, you know, hey, how this thing is actually, you know, working, is that in line with practice? And we need the folks that traditionally have done that in a non-agentic world be able to be comfortable with managing agents that are potentially putting the organization at risk.

17:24Kate Smaje: Yeah, I mean, the rise of AI does not mean that we all suddenly devolve all responsibilities for quality control and critical thinking. And that's not the world we're in. If anything, those capabilities, I think there's a premium on them now. And your ability to pass apart what that agent is doing. How have I set it up? What's the thought process? What information am I actually giving it access rights to?

17:52Senior Partner: How have you seen business owners collaborating with the technology team to kind of manage cost of compute or some of the agents that they've created?

18:00Kate Smaje: For me and many of the companies that I work with, I will always ask the question when a great idea comes up, love it. What happens when it's wildly successful? And it's amazing how often you get people, you know, look at their feet or shuffle in their seat a little bit and say, when this is wildly successful, well, I'd have to refactor the whole code base because I didn't build it for scale. I built it as a POC. Or, you know, oh, when this is wildly successful, well, we'd have to change the way that we actually do it because it shouldn't call on an LLM the whole time. That's going to create huge latency problems and huge token costs and so on.

18:33Kate Smaje: So my first thing I want to say, because I don't think this problem is just the purview of tokenomics, is you've got to ask yourself what happens when this is wildly successful. Because I don't think that there's been enough thought, if I can humbly say, about those second order, third order consequences of AI. We can come back to unpack that if that's helpful as well. On tokenomics specifically, I see a number of different things. firstly I think it is just starting to understand and be able to model and forecast where is this actually going and I'm seeing a huge amount of time and effort often triggered actually by finance departments sort of working in collaboration with technology very effectively by the way you know but the collaboration there to say can we get underneath the skin of how we should think about this over time and what do we actually you know want in terms of outcomes that's why I see it be good.

19:30Kate Smaje: Where I see it be bad is it's just like, whoa, we've got to pull back on this, strangle everything, get rid of all the licenses, pull it back. Well, that's not a sustainable solution, right? The sustainable solution is to say, what is our appetite around this? How do we want to think about it? This isn't a reactive thing. It's got to be a proactive thing. So really thinking about better forecasting, better modeling of where this is going to end up. I see a lot of folks thinking about what does this mean for their vendor relationships? not just kind of, you know, better negotiation of contracts and so on, but starting to think about, you know, token marketplaces and ability to trade off against your high volume users versus your low volume users, because there's a huge tail in every single organization, right?

20:16Kate Smaje: You can't just solve for your super users, because that's actually not the way that the whole organization uses AI. And then I think there are things of, you know, in the design of the way that you build some of this software is saying, look, there are some things that we do where you don't need to drive a Ferrari to the grocery store. And so let's actually just allow you to use the base model for that. Because the base model, by the way, will be faster. It will be a heck of a lot cheaper, orders of magnitude cheaper. And if I'm doing something basic like transcription services, I do not need to use the best model out there to be able to do that.

20:52Senior Partner: Do you have anything to say about adoption generally and how to inculcate adoption across an organization and get to that level of fluency and learned behavior, given that we're all kind of working at the speed of AI without necessarily being fluent in these AI tools?

21:13Kate Smaje: Yeah, so I think the real challenge is that we're having conversations around tools. and uh you know where i get nervous on some of this is when you see scorecards and the adoption scorecards are just you know what's the adoption rate of tool a versus tool b versus tool c you know market by market blah blah no right that's just a hiding to nothing it doesn't really tell you about um either how the the technology is being used let alone how it's actually creating value. So I think mistake number one is that we use the word tools too often in these conversations. You and I were with a group of CFOs yesterday.

21:52Senior Partner: And one of the things that kept coming up from the CFOs themselves was we are living through a period where there's lots of fear. There's anxiety. We talked a lot about psychological safety. So as I'm thinking about that employee journey, and I'm thinking about what are you seeing as leading organizations doing to create the conditions for people to really embrace AI in thoughtful ways that aren't just adopting a tool, but rather changing the way that they're doing things to drive towards an outcome in a way that we think is consistent with, you know, a great employee journey, right? Just wonder if you've seen anything that's really working in this, in today's world, whether it's this heightened level of anxiety.

22:33Yeah.

22:34Kate Smaje: And by the way, I think there really is a heightened level of anxiety, both both, you know, within organizations. So that's sort of the FOBO concept, right? Fear of becoming obsolete. You know, I'm worried about what this all means for me, whether I can keep up. I get a lot of questions kind of when the, you know, when the door closes and the, you know, and you're just, you're just in the sort of truth room and so on. I get a lot of questions about, I don't know if I can keep up now, right? The world just feels like it's moving so fast. And I, and I get that, right? I totally get that. So I think you see it inside organizations, But I also think you see it outside of organizations.

23:08Kate Smaje: You know, there is a bit of a backlash right now on, you know, AI is bad. AI is bad because I don't want a data center in my backyard. AI is bad because it's taking jobs. AI is bad because it's entry-level jobs, right? What are my kids going to do? AI is bad because my winter fuel bill is going to go up because there's pressure on the grid, right? There's all manner of these different things. And I think the onus really is on, you know, at least in the corporate context, on leaders. And I say leaders maybe in the broadest sense of the word, right, whether that's corporate or otherwise, that leaders need to think about, well, what actually, how would we make a good outcome from AI?

23:46Kate Smaje: Like, how can we actually do something here that is going to be net positive, net positive for my organization, net positive for those that are, you know, operating within it, and net positive also out for the communities and the, you know, society in which they operate. And I say that because I think some of the fear is because we're not having those dialogues. And we all know that fear sort of amplifies in a vacuum, right? And so I think we need to fill that vacuum. And I think we need to be able to have an open and honest conversation about why are we doing all this stuff on AI? This isn't AI for AI's sake.

24:21Kate Smaje: It's not, you know, because we all want to be cheaper, less interesting, you know, less culturally diverse versions of ourselves, right? That's not why we're doing it. So why are we doing it? And when you really ask, or at least when I ask CEOs and CXOs that why question, I get some fascinating answers. You know, I want to be able to solve, you know, genuine kind of disease problems that couldn't be solved otherwise, because I can speed up my R &D pipeline faster than it could ever be. You know, I want to, you know, make sure that every person, wherever they are, gets the insurance backing that they actually need.

24:59So when something goes monumentally wrong, that they're actually protected in a way that's fair and just.

25:04Kate Smaje: You know, I want to solve a way of squaring the circle of how you get, you know, cheap and affordable and green energy, you know, all together. And there's all different versions of these, but the why of it, when you actually boil it down, I think in many cases, it's a super interesting why.

25:23Senior Partner: Yeah. The example that I always give, which is very analogous to that, is my friend who's an ER doctor. And AI is freeing her up to, in her words, be a better doctor, be a better human, because she's not doing all the transcribing and all the paperwork that any medical professional will tell you is the least favorite part of the job. And I think we need to lean into the human parts of what AI enables. And there's actually interesting research that has come out in the past year or so that shows in medical training, medical training that is done with AI that allows the student to try first. And then the AI comes and says, hey, have you thought about it this way, but provides a little bit of the coaching actually creates better outcomes in terms of learning than the traditional way of doing it.

26:09Senior Partner: So if you integrate AI thoughtfully in the apprenticeship process, you're actually able to accelerate how people learn. But if you're not thoughtful, you can end up with some of the worries that people have. But I'm on the optimistic end where I think AI has this incredible ability to free people and get them up to their top capability faster.

26:31Kate Smaje: But I think there is a second order of that as well, which is when I see the people who are working most intensively with AI right now, the natural inclination is, oh, great, this frees up so much time. You know, we're all going to have, we're going to be able to walk the dog more and we're going to have all this free time and it'll be wonderful. We'll have, you know, margaritas at four o 'clock every day. And actually what you find is that group is exhausted. And it's not because the AI is failing them. Actually, it's because the AI is working, right? So it's taking out all of those rope work kind of tasks, right?

27:07Kate Smaje: Taking a whole ton of stuff off their plates. But what's left is a level of kind of cognitive engagement, higher cognitive engagement than, you know, than we're actually used to doing. So it's the things that require real judgment to move from, you know, do I go left? Do I go right? It's the things that actually the AI couldn't do, you know, and I have to make a, you know, make a call on it. It's the difficult conversation that the AI can't do that actually is draining emotionally, right? It's all of those things. So what we're actually starting to see is the cognitive load on the AI super users, if you like, is going up, not down.

27:45Kate Smaje: And so I do think, you know, optimistic as I am, and I really am, you know, there is a second order point here around how we think about that brain health, how we think about cognitive load and start to, you know, plan for a workforce that is going to have a very different set of demands on their cognitive ability.

28:06Senior Partner: Are you seeing anything new arising either in your client work or in your research that leaders should have on their radar that they may not?

28:15Kate Smaje: For many things that we're starting to do now, your ability to discern the difference between the last model, the next model, et cetera, for the long tail of users. I'm there's still so much more that's going to be interesting on that level. But for many, many people, it's getting harder to discern. And so part of me, oddly, hopes that actually some of this cadence of feeling like the world is moving so fast around me that's causing a lot of that fear might start to settle a little bit. My optimistic self says we might see a little bit of that for that group, not for everyone, but for that group over time.

28:53Kate Smaje: I think the other thing that is coming is, I hope that we're going to spend more time on the humanware and maybe a little less time on the software and the hardware. Because the thing that is making this hard for folks is, you know, what actually is the answer on how we're going to reimagine how this workflow is going to be different? What is the answer on how I need to be showing up differently in my hybrid workforce of, you know, carbon and silicon employees to bring that together in a way that's actually going to work? Those are the questions that actually are vexing people. And I think if we can use this moment to lean into that human where part of the equation, I actually think that will help massively.

29:36Kate Smaje: Kate, Brooke, and Brian, thanks for joining us. Thank you.

29:39Senior Partner: Thanks for having me. This was fun. Thanks for joining us, Kate. This was awesome. Yeah, thank you so much.