277. What REALLY happens when you let A.I. run your workday, with The Economist's Boss Class Host, Andrew Palmer

25:37And that would just mean a whole bunch of invoices suddenly landing on the finance teams in each territory who weren't prepared for it. So work was just sort of like a sort of waterbed effect. It was just popping up somewhere else. So managing all of that is difficult. So what do you do? I mean, I think some firms experiment with kind of quite blunt carrots and sticks, right? I mean, so your performance review might specifically talk about AI usage. um there's a you know firms which have um i think quite crude incentives like you know if if the firm as a whole has a million queries to chat gpt that triggers a cash payout um so of course people are just presumably just writing in you know does this count does this count over and over again and and they trigger it so designing that is is hard i think a better way to approach is on the adoption front, obviously setting the expectation, role modeling from senior leadership is important, but the metrics matter more.

26:47So you just go back to quite basic ideas in management, right? This is the thing that we are trying to do. This is our priority. And you kind of try and get something measurable. So it might be speed to market for a new product, or it might be a sales measure or whatever it is, and then let people work out whether AI is the best way to do it or not. Give them the tools, give them the encouragement to experiment with AI, but don't force them down that route because it may not be the right route. So I will give you – there was quite a nice little sort of five-step ladder that a guy called Bryce Chalamel, who is now running kind of enterprise AI at OpenAI, the maker of ChatGPT.

27:33But when we spoke to him, it was at Moderna, the big biotech firm. And so he just had this sort of quite nice sequence. It was access, which is like giving people access to AI, paying for them to have access to the model, making sure that they have the tech. And then it was adoption. So making sure that, you know, they were being encouraged to try it. And that could be incentives. It could be role modeling, could be all sorts of stuff. Then proficiency, how do you get good at it? Because you have to practice and then you get into new ways of working and reorganisation and that's when you start to actually change the inside of a firm.

28:12But those first three steps are kind of unavoidable. You can't jump them and you need to be measuring usage but I don't think crudely incentivising usage alone.

28:23Andrew Palmer:Is this where we get this kind of, I don't want to be irony or a paradox where it's in the world of AI, it's as much about great people management, it's as much about great change management and the basic principles of that that have been around for decades. Is that what's going to be the difference between successful AI adoption and unsuccessful? I think a lot of that is right, that quite standard techniques from change management are really going to matter. So, you know, we talked a little bit about workflows, but making sure that the AI is part of an existing workflow is critical to adoption in the first instance.

29:05Later, you can worry about reorganizing absolutely everything. But if you want to get people to start playing around and working out what the potential is, you've got to make it as easy as possible. for them. The difference here is that, you know, traditional change management is a kind of, you know, you sort of unfreeze things, you start to, you basically train the organization to work differently, and then you freeze them again. And the critical thing with this technology is that it's moving all of the time at pace. So you never get to that freeze or refreeze point again. You're constantly having to change.

29:44That's a slightly different feel to the technology and quite a difficult one. So you're constantly re-evaluating where the models are, what can you do. I could imagine in time that it might settle down, but for now you're never done. And that's different from the internet.

30:03Andrew Palmer:And you got quite hands-on for this series, is experimenting with AI yourself. How did you decide where to test it in terms of the tasks that you have? And is there anything that you found it particularly brilliant at? Again, to reference Ethan Mollick, who I had one of the first conversations with, and I asked his advice, like, what do I do? I'm an unsophisticated user. I sort of often just forgot that AI existed, you know, sort of like just wasn't really using it much. And his advice, which I think was useful, was, you know, tomorrow, go to the office, everything that you do, try and get AI to do it.

30:42And it doesn't matter what it is, just try. And you will quickly work out where it's helpful and where it's not. So that was where I started. As it happens, I mean, all of us will have different jobs. So you can't sort of, it's not very transferable, but there are certain bits of my work, and I suspect that they map a bit to yours, where, you know, if you're researching something there's just some really useful um use cases there immediately with with ai or if you're kind of thinking about like who are the top five people in the world to talk to on something um this is a this is a more sophisticated version of a google search it's not cheating because you'd be doing this anyway it's just a better way of of of doing those things so that was all that was all very helpful i think you know to the extent that sort of what what surprised me or what what jumped out and where I was immediately taken into a totally new space was coding.

31:40As witnessed by my love of printing, it may not surprise you to know that I am not a coder. I don't know how to code. It all looks like gibberish to me. But the moment there was a task that needed to be done within The Economist to build a checker of our internal style guide, and someone had basically been waiting for developer time for over a year. And there's a scarce resource within many organizations. And just by sitting down with Claude and entering English language prompts, within an hour and a half, I'd built something which worked. And it didn't end up being exactly the thing which is now being rolled out to the newsroom, but it did give other developers a kind of sense of what this thing might look like and how to go about it.

32:29So that was a genuinely magical kind of experience, actually. It was like, I can do this thing and I could never do it before. But there's then also a bit of the kind of pullback because there's like so many journalists now saying, oh, coding, look, I can code. Isn't this amazing? Actually, you've also got to be very careful. So, you know, people who really knew what they were doing had to go away and think about how to implement this. I'd given some useful clues, but I was also nowhere near producing something that was able to go out to a sort of live environment. And other conversations that we had in the season made it clear that vibe coding is wonderful for kind of accelerating, sort of prototyping um i'm about to use the word ideation which is a kind of rule of mine i should never use the word ideation so but you know that kind of process of thinking through um ideas um it can accelerate that but you don't then throw it out to market you've you've got to have people who really know what they're doing to review the code um and to to write it and to test for security so there's a nice little phrase which is demo don't memo um which is going around in tech circles and that's like don't write powerpoint slides don't write google docs just use a natural um use vibe coding platforms to give a sense of what it is that you're talking about but you don't then just throw it out um to a production environment there's still work to be done so there was a little that was both magical and a kind of like well hold on don't don't lose your head over this because it can still foul up in fairly drastic ways.

34:17Andrew Palmer:I mean is there anything that that you think changed your view of how humans might work given its current capability and bearing in mind that this is as bad as it's ever going to be and where it could it could move forward? One of the kind of my favorite conversations was with a kind of unheralded person actually so someone called hillary gridley who's a product manager whoop which is a wearable device company and um hillary's not not wildly senior person not kind of like out there kind of above the parapet um but she's just super interesting on how to use ai as a manager and has built a bit of a reputation for herself in that that field and she so what she's doing just within her team is sort of supercharging herself as a manager.

35:08So this is kind of coaching and feedback, I guess is the sort of use case here, where she says, as a manager, I have limited time to give feedback to everyone. And I find that I'm giving the same feedback over and over again to people as new people come into the team, or because her advice is not landing for whatever whatever reason. So she's just built a ton of custom GPTs where she is basically codifying her feedback. So she will take a first draft of something and then a finished draft of something, ask the AI to turn that into a sort of feedback machine, basically, so that other members of her team, when they have a first draft of something, can put it into the custom GPT, say, what does this look like?

36:03And a kind of quasi Hillary, a sort of shadow version of the manager is there saying, look, you need to work on this and that. And that doesn't remove the entire process. She still needs to look at something, but what's coming across her desk tends to be better quality than it was beforehand. And the person on her team is not waiting for feedback. That seems like a kind of low stakes and really helpful way of a manager improving their team when you know and we all know the constraints around metal managers etc when time is limited when lots of people want want feedback so I really I just really like that and I played around with with making sort of versions of that here and it was it was useful just as a critiquing tool it was really really useful so i would encourage people to play around with that i really enjoy about boss classes

37:02Andrew Palmer:how you're the people that you speak to and the organizations that you get insights into it it's such brilliant real world examples and honestly i think i'll retell us the story you told us about to the last time we spoke to you probably at least once a week to somebody that that we come across because it's just such great lessons that we can take forward with us you spoke to so many people Brett Taylor, Mike Krieger, brands like Pizza Hut, Indeed, Lovable. What's the standout story, takeaway for you? Well, if we're talking Pizza Hut, we've got to talk about takeaways, haven't we? So the standout takeaway, I think, would be that we're in this phase where there are all these kind of barriers to reaching the kind of the imagined AI future.

37:51And so you have to think all of those through. And some of the behavioural, and we've talked about them, employees are fearful or they're over-enthusiastic and they don't think things through. So that has to be worked out. Some of them are workflow related. Like, is it natural? Is it a natural part of things? Some of them are technical. Is the AI good enough? Does it hallucinate? And some of them are organizational. So how does the organization as a whole make sure that things are fitting sort of smoothly into the way that things are running? So I will pick on Pizza Hut, not because it is the most sci-fi version of the future, but because it is, I think, a useful window on what's happening right now.

38:40So I went to a place called Plano, which is north of Dallas in Texas. Apologies to your Plano listeners, but I do not recommend going to Plano. There's almost nothing to recommend it. But it is a place which has the kind of experimental pizza hut, basically. It's just a laboratory for innovation within that brand, which is owned by a bigger company called Young Brands. So at that pizza hut, basically AI is infusing absolutely everything. So you've got customers making orders and it's an AI chatbot. You have machine learning AI working out which orders should be done first so that a pizza can arrive with a customer piping hot if it's being delivered.

39:28You have generative AI pulling on social media feeds, rating sites, direct feedback to give a sense of if there are any problems, and so on and so forth. And in future, they think that there'll be computer vision there as well so that people can kind of like be sure that they're putting the right number of pepperonis on the pizza or whatever it might be. But all of it is kind of it's in the workflow. So you've got this technology is there. It is incrementally helping to make a process better. They are not going wild with it. It is experimental. the chatbot the drive-thrus that they have um elsewhere in that group have humans listening in and stepping in if things start to go go wrong so it's one of those things where you sort of i spent a couple of hours there and sort of at the end of it i was sort of asking you know guys this is this is this is the super technology and you end up with you know the right number of pepperonis on your pizza it feels a little bit kind of disappointing but actually i think it's a really good example of how playing around with processes, working out where it fits, where it doesn't, thinking through the guardrails is something that every organisation has to go through.

40:50I don't think pizzeria is going to be transformed very fast, in fact, but I think those principles are quite useful to think about. And you did say takeaway.

40:59Andrew Palmer:I did. I did. Fair. I mean, was there anything that you saw that kind of scared you? I don't know if you've heard about this and i'll probably forget the name it's called something like molt book or and it's basically yeah yeah you heard about this with the the ai agents have basically got their own social networking site and i was telling you about it and i was like this is it's amazing and it's cool but it's kind of terrifying is there anything you saw that that kind of made you step back and think oh i'm not sure there are things which are really frightening again if you go a sort of long enough time horizon out and start to extrapolate, you could imagine the AI being really, really good at almost everything, better than humans.

41:43And so Maltbook is a sort of, I guess, one example of that. Although to me, that's maybe less frightening because it's a bunch of agents kind of interacting with each other based on what they're being trained on. I think you'd expect some sort pretty weird behaviors. But there is, you know, people behind these models are worried about, you know, there's behavior like blackmail, for example. You know, in the right circumstances, an AI has at Anthropic, the maker of Claude, basically sort of gone around the back, found evidence of an employee having an affair and attempting to blackmail that employee in order to reach its target and to perform its task.

42:34So there are behaviors there which are like, bloody hell, that's potentially frightening. I just wanted to quickly interrupt here. Andrew did message us later on and say, just to be clear, that this was an experiment and they were kind of trying to get Claude to blackmail the users. So we're not quite in the terrifying universe just yet where Claude will start blackmailing you. But it is a great example of what potentially could happen without guardrails right now i think the most disturbing moment for me was was was less that it was i wrote a i wrote a column um and then asked uh one of the models to do a version of the column with a simple prompt and in 30 seconds it had written um you know a column which i thought was like materially worse than mine i was really really um certain that one was obviously better than the other but if i showed it to my colleagues it ended up being perilously close so it was actually three two in my favor but two colleagues just thought the ai had written the column the column that i had written and that was you know briefly very very unsettling it was you know what is what's the point of me writing these things if people can't tell and i kind of got over it because you start one value in ai is is like it forces you to be very introspective about what's the thing that distinguishes me where do i add value how do i continue to stay ahead of the machines and you know maybe i'm just fooling myself but i kind of just i sort of in my head thought that through and felt felt like okay i don't think i need to worry just yet um about an AI doing my job.

44:18But it was a bit of a wake up call. So I just thought there's no way my hugely discerning colleagues are going to think that I wrote the thing that the AI did.

44:28Andrew Palmer:Yeah, it's tricky, isn't it? I was talking to somebody who works in AI saying that, who's a podcaster as well, and how obviously now you can take text and it can change it into a into a podcast read out by two different AI hosts and how, yeah, it can start to mimic voices by going into the back catalogue of kind of your podcast and think, oh gosh, you know, maybe someone couldn't tell the difference if it was me and Al or an AI. But until that day, we will continue on. I guess what I want to ask you about before we wrap up is around the ethical side of AI. In my world, in the world of psychologists, there was a lot of concern, a lot of discussion around the ethical responsible use of AI, the fact there isn't much legislation at the moment.

45:14Andrew Palmer:Where have you seen a difference between AI being used carefully and perhaps carelessly? You know, the world that we're both in, which is sort of organisations and managers trying to get to grips with this, tend to be better. You know, there are reputational risks, often these are regulated entities so they tend to be thinking about these kind of things a bit a bit harder so you know generally speaking I thought most of the people that I spoke to within that world were pretty pretty thoughtful or at least they'd had the experience early on of like experimenting and things had gone wrong and they pulled back a bit so we seem to be in a bit more of a thoughtful phase.

45:59I mean, there are obviously plenty of examples of unethical AI in the sense that bad actors can use AI to try and hack into systems, for example. So it can be put to bad use. It can certainly be put to thoughtless use. There's a lot of news at the moment about Grok, which is Elon Musk's AI chatbot and how it is being used. There are definitely unethical usages. I would just say the enterprise, companies can definitely get things wrong and job worries are going to center on the enterprise. But in terms of that kind of you know hell for leather doesn't matter at all that doesn't seem to me to be the place where where the real worries are most organizations have processes around governance and data so it's much more like solving solving those barriers um rather than the sort of hell for leather unethical stampede seems to be the problem i think a lot of leaders i speak to business owners who

47:10Andrew Palmer:haven't quite figured out how to feel about ai yet they're sitting in a space where they're excited by it they're slightly concerned about it they're overwhelmed by it what advice would you give to to a leader listening who might be at this kind of this weird bit on the fence where they're they're excited but they're also a bit uneasy i think that's completely the right place to be um so that is if you're not feeling that i think probably you've you've got it wrong um because it is this strange mix of great risk and potentially existential risk and then huge opportunities. And we don't quite know how it's going to fall out.

47:50And the technology itself is very unpredictable. So you can code something without knowing how to code, but you can ask it to spell strawberry and it will get it wrong. So it's this totally baffling mix of capabilities. So I think confusion is totally normal. And sort of leaning into it is the answer. So, you know, every leader who's banging the drum for AI, who isn't, you know, playing around with AI is not doing themselves a favour. You know, they need to develop their own intuition for where this might work and where it's actually not going to be particularly productive. So I think that would be my advice.

48:29So be happily confused and experiment.

48:33Andrew Palmer:I love that. I love that. Be happily confused. if there is anything else that that you'd say of course everything you've you've learned andrew is there anything that's really really surprised you anything you really want leaders to make sure they understand i guess kind of a key a key lesson from from all the conversations you've had that you really want to make sure communicates to the business world i think it's back to some some a fairly fairly basic thing actually which is the theme of this right now. And that is, you know, being super clear what it is that you're trying to achieve. So, you know, it's classic, classic sort of technology problem where, you know, this shiny thing comes out and everyone says, oh, we've got to adopt it.

49:18But why? So what is it that you are trying to do with this technology is as ever the key question to ask. And once you have worked out what your priority is, then that's where you, that's where you put your, you know, the eggs, That's where you focus your energies. And if AI isn't the answer, then that's fine too. There may be other ways to solve the problem. But I think it is a sort of prioritization challenge fundamentally still. This thing can do everything. That's a real problem. That's sort of paralyzing. It's like watching Netflix, right? You can't like, what the hell do I do? So choose the thing that has the most impact as a leader and go from there.

50:00I think that would be my super obvious, but actually surprisingly countercultural advice. If there's anyone listening who is feeling a bit nervous having heard this conversation,

50:14Andrew Palmer:what advice would you give them from a career point of view to stand out and, and I guess to use your phrase, get ahead of the machines? Well, I think playing around with them is important. um so you know it's natural to feel anxious about about ai and to wish that it hadn't been invented frankly if you're kind of heading into this this labor market um but i you know it's there it's with us and potentially it is it is going to do amazing things as well as damaging things and so i do think playing around with it is even more important for for the kind of you know the younger cohort um it is a way to kind of lean into you don't have other things to bring to the table for entry-level jobs but you can bring an understanding of ai so i would i would say that and then i would say the other thing is and this is this applies to everyone is working out um you know when it is not appropriate to use it so this is something we all have to get our heads around is, you know, even as it gets better and better and better, what are the skills that you want to guard for yourself?

51:27What are the moments when, you know, using this thing actually results in some sort of cognitive decay over time or dependence on the machine? And so I would say, you know, one thing which is very, very noticeable from people talking to people who are right at the cutting edge of this is that almost without exception, they said that they do not use AI to write, that they use AI to kind of critique what they have written, but writing is thinking, and they do not want to outsource thinking. And so all of us have to kind of have that internal conversation. You know, what's the point where we're going to do the work, even if there's this really, really tempting, fast assistant, which is always happy to help.

52:13So that was Andrew Palmer from The Economist. And I don't know about you, Leanne, but I came away from that feeling usefully confused.

52:20Andrew Palmer:Yes, which it turns out is exactly where Andrew thinks we should all be. So let's go back and pick out three key takeaways for leaders. Number one, start boring and start now. The most effective AI implementations aren't glamorous. They're built into existing workflows, they're low stakes, and they don't demand 100 % accuracy. see. Think Glowforge's AI sales coach, not sci-fi. Find one repetitive task in your team's week and try it there first. And the second thing, which is this is so important, don't outsource your thinking. Andrew spoke to people at the absolute cutting edge of AI and almost every single one of them said they refuse to use AI to write.

52:59They use it to critique what they've already written. Writing is thinking. You need to really guard that. And if you're building a team, ask yourself, what skills do you never want your people to outsource to a machine?

53:11Andrew Palmer:And lesson three, be happily confused. If you're a leader who's excited by AI, but also kind of uneasy about it, Andrew says you've got it exactly right. Confusion is the appropriate response to a technology this genuinely unpredictable. The answer isn't to resolve the confusion, it's to lean in and experiment anyway. You can find Andrew's Boss Class podcast wherever you get your podcasts. Just search Boss Class by The Economist. We'll put a link in the show notes along with Andrew's Bartleby column. It's currently back for its third season exploring how AI is transforming leadership and management.

53:45So if you like this interview, you are going to love Boss Class 2.

53:48Andrew Palmer:That's all for today. This is Truth, Lies and Work. We will see you next week.