Why Today's Training Methods are Failing

0:00I was like, this, I found it. I found it. This is what I've been looking for. I can honestly say has genuinely changed the way I run my business. It's changed the results that I'm seeing. It's changed my engagement with clients. It's changed my engagement with the team. I couldn't be happier. Honestly, it's the best investment I ever made. What you just heard was a testimonial from a recent graduate of the Duct Tape Marketing Certification Intensive Program for fractional CMOs, marketing agencies, and consultants. Just like them, you can choose our system to move from vendor to trusted advisor, attract only ideal clients, and confidently present your strategies to build monthly recurring revenue.

0:39Visit dtm.world.scale to book your free advisory call and learn more. It's time to transform your approach. Book your call today. dtm.world.scale

1:03Hello and welcome to another episode of the Duct Tape Marketing Podcast. This is John Jantz. My guest today is Matt Bean. He does field research on work involving robots and AI to uncover systematic positive exceptions that we use across the broader world of work. He's an assistant professor in the technology management department at the University of California, Santa Barbara, and a digital fellow with Stanford's Digital Economy Lab and MIT's Institute for Digital Economy. He received his PhD from the MIT Sloan School of Management. We're going to talk about his book today, The Skill Code, How to Save Human Ability in an Age of Intelligent Machines.

1:47So, Matt, welcome to the show. Delighted to be here, really. appreciated the invite and excited to talk. We're going to do our best to save human ability today. I think that sounds like a noble goal. So the premise, I'll be very brief and let you talk more about it, but the premise of the book is this idea that we're losing a lot of skills that the master taught the apprentice, you know, over centuries of time. So kind of led you to studying that particular gap or space. Right. The gap I wasn't sure was going to be there. I've always been interested in training, learning, and skill development my whole career, one way or another.

2:26And since about 2000, the confluence of that in AI and robotics got my fierce and firm attention. When it came time to pick a dissertation focus at MIT, I knew robotic surgery was a very interesting place to go because it boy oh boy doing robotic surgery is radically different than doing it the old-fashioned way or even the modern way before that which was with straight sticks basically laparoscopic surgery as we knew about it in the 90s and i just looked at the control apparatus for that robot and it's a giant xbox controller basically with a 3d vision goggles that you wear and foot control so you're using your feet and your hands and i'm like that bears almost no resemblance to the old-fashioned way.

3:12How do you learn how to do that? The parts they're looking at are called the same thing, but other than that, right? Yeah, no kidding. Although, by the way, looking at it through this robot, you're seeing it at 10x magnification and an inch movement on the outside with your hands and the controller translates to a millimeter on the inside. So it's a real different game. I didn't know all that to begin with, but I figured the training for this has got to be real different for the training the old school way. How does it work? What are the upsides? What are the downsides? That's the sort of going in question.

3:41After no more than three months in the field, in the operating room, watching real procedures with this thing, talking to residents, talking to surgeons, it was very obvious that everyone was assuming the way to learn how to use this thing is the same way that you should learn how to do the old procedure. And that was a fatal assumption. Turned out to be false because, and by the way, the old school way is something we have encoded almost in our DNA. It's literally 160 ,000 years old. In surgery, they call this see one, do one, teach one. You basically show up, help a little bit, watch, get a little bit more involved as the expert decides you're ready.

4:20And sooner or later, you've got somebody looking over your shoulder. Look at any profession. I defy you. The book is full of examples of this. We just take that for granted as the way that you learn how to do stuff and build skill, which is the ability to do the thing under pressure, basically, reliably. That's different than knowing conceptually, like book smart. This is like, can you do it? So anyway, it turns out that mechanism of learning, this taken for granted thing, it was getting busted in robotic surgery because the console and the controller allowed the senior surgeon to do the whole thing themselves.

4:55So the resident becomes an optional participant at that point. Like I have to make mental effort to involve that person who wants to become me someday. And there's a patient on the table. And therefore, I'm never going to do that because they're going to be slower and make more mistakes than me. So you'd get, instead of the old school way, a four and a half hour procedure, that resident is busy for actually about six hours because they are working from before the patient gets, there's any incision all the way to well afterwards. They're sweating the whole time and they're helping out in consequential work.

5:30Now they might be swimming in this shallow end of the pool, but they're swimming. Now they show up, help get that robot docked to the patient, sit in a separate control console and they watch a movie. That's all they get to do. and so that was how no one everyone recognized that was unsatisfactory on some level but boy oh boy they were just kind of tolerating it and surgeons would come out of programs after six years of big air quotes training and they i have well i'll quote a chief of urology for one of the top hospitals in the states and he said top surgeons suck now that's what he said to me to my face.

6:08They've watched a lot of surgery. They haven't done it. We have to retrain them when they arrive. So that was the, it was about halfway through that study that I realized most of what I just told you. Because really when it comes down to it, you know, learning is reps, right? Yeah. And not reps like on a putting green in with no wind and a perfect Sunday day. Like you got to have somebody throwing rocks at you. And so it's sure some reps in practice conditions is helpful. But you're right. And even in surgery, they have a name for this in the research on surgery. They call it dwell time, which means how much time are you in the OR?

6:44That used to be the old proxy for you're in there. That's the best place for you to learn. Now it's the worst. So anyway, I also made it a point in that study to try to find surgeons who were learning anyway, in spite of that barrier. I was very lucky to get some guidance to do that early. That turned up some really interesting and new ways to build skill that defied the new barrier of novice optional world that we've got now. So is the skill required to become a great surgeon different now? I mean, motor skills maybe are different needs. I mean, or is it really just learning the tools? Sure, there are different skills involved.

7:24And that's much less important than do you know a healthy way to build skill, period, in this context. because the skills that you need to do your job are always changing a little bit constantly. You know, new version of this thing for my iPhone. Gosh, darn it. I got to learn a little something every day, right? Why did they move that button? Right. And big things come along every once in a while. And we've got a really big thing with ChatGPT these days. But for most folks, that doesn't mean you have a whole new job or a whole new set of skills. It just means, say, 30%. But if you have a healthy path to learning new skills, humans love to learn.

8:01We're going to do that. It's no problem. If the way you learn is getting busted, then we have a serious issue. So you have talked primarily about medical industry, but you actually studied a number of industries, didn't you, that this equally applied to? Yeah, over 30 now. So that was the immediate question. I published that study in 2019. I gave a TED Talk in 2018. But actually from about 2016 on, those findings were done. And I wanted to know where else is this happening? Is this novice optional thing happening elsewhere? It should be. That's all I knew. Because I had the hypothesis that the more intelligent technologies, robots, AI, they allow a single expert to get more done with less help.

8:48If that's really true, then let me go to some radically different places in the economy to study things like policing, investment banking, bomb disposal. I have a list of more than 30 now to see different technologies, by the way, different skills, different cultures. If this same problem is showing up there, then it's time for all hands on deck. We've got a serious problem. And that resulted in that Harvard Business Review and TED Talk piece, which came later. It looked to the world like I just trumpeted my surgical findings. But in fact, those were three years in the can. And I had spent the next three years being like, hang on.

9:23So that was the sobering news in 2019. This is everywhere I can look except for one place. I found one exception. It was accidental, where tech was getting deployed in a way that improved skill development, which gave me more hope. And I think it was worth studying. And I keep trying to find and study exceptions like that because we need them. Yeah. But in general, yeah. Once I hit north of 2022 professions, occupations, industries, and so on, I just kind of stopped trying to find it. It's my pleasure to welcome a new sponsor to the podcast, our friends at ActiveCampaign. ActiveCampaign helps small teams power big businesses with a must-have platform for intelligent marketing automation.

10:07We've been using ActiveCampaign for years here at Duct Tape Marketing to power our subscription forms, email newsletters, and sales funnel drip campaigns. ActiveCampaign is that rare platform that's affordable, easy to use, and capable of handling even the most complex marketing automation needs. And they make it easy to switch. They provide every new customer with one-on-one personal training and free migrations from your current marketing automation or email marketing provider. You can try ActiveCampaign for free for 14 days and there's no credit card required. Just visit ActiveCampaign.com slash duct tape.

10:45That's right. Duct tape marketing podcast listeners who sign up via that link will also receive 15 % off an annual plan. That's ActiveCampaign.com slash duct tape. Now this offer is limited to new ActiveCampaign customers only. So what are you waiting for? Fuel your growth, boost revenue, and save precious time by upgrading to ActiveCampaign today. So I want to get obviously a little more into the findings too, but just even your process is fascinating. Did you have any industries that kind of questioned your goals? Like, what are you up to? Like, what are you really studying? What are you trying to, like, how is this going to change my job?

11:22No, because already on the ground. So number one, I should be real clear. All those 30 plus industries I'm talking about and occupations and so on, I didn't do all those studies myself because each one of these studies takes about a year and a half. I got primary access to data collected by other scientists who had done similar studies in their context. So investment banking, for instance, I've collaborated with a woman named Callan Anthony at NYU. She gave me her data set. I gave me her mine. We compared notes. We wrote another paper off of that. But in general, surgery is a perfect example of this.

11:56Everywhere I've gone, everyone already knows things are not well when it comes to skill development. The next generation is struggling, but the productivity boost coming from these technologies is such an enticing saccharine hit up front that there's no one party whose job it is to pay attention to that unintended negative side effect of getting this productivity hit. The expert is happy. The organization, the person who's investing in that capital is happy. And it's kind of a pain to learn now. Can't find a mentor to save my life, says the 24-year-old. That's nobody's problem, really. Well, let me ask you this.

12:35Will that catch up over time? Like, will that 24-year-old who, when he's 48 and that's all he's known, be the mentor? Exactly right. This is why I'm trying to sound the alarm bell as loud as I can. This is why I wrote the book. It's three, five, seven years later, depending on the cycle time for your talent, that the organization, the profession, the economy is going to start to realize that the next generation of talent just isn't prepared for duty. And surgery is noticing this now. So I announced my findings in about 2017, 2018. And I'll just ground this out for you because they went kind of early.

13:09So I think other professions should get ready for this. The next generation of robotic surgical talent that's floating in the hospitals have to spend an extra, oh, between 80 and 180 grand on what's known as a proctor, which is another senior surgeon either inside their own hospital from the outside to come in and retutor this person on how to do the job. They just went through five years of training for it. That's supposed to be good enough. And everyone knows on the street, it's not. And so it adds to the cost base, the time to ramp to skill. That's a band-aid solution. There's no way that's going to work.

13:44Are there other factors besides the technology that are really kind of leading to this, but maybe even culture inside an organization, demographics of the next generation coming up. I mean. Yes. Yeah. And so let's, I think it's the fair thing to say is the problem I'm describing has been around since the advent of language. So about 160 ,000 years old, let's go back to ancient Greece, which I do in the book. There were big disputes about the shift from hand pinched pots to using a potter's wheel. And what was that going to do skill around here? And, you know, an expert doesn't need mentee or an apprentice quite as much.

14:23If they've got a potter's wheel, they can do a whole bunch more themselves. So it's just the intensity and the pace of what happens when chat GPT is free for everyone to use. They're dramatically more self-serve on a much broader span of their work. Their dependence on a novice in different ways is dramatically, the span of management control these days, I've been doing studies in warehousing, for instance, the last few years, you have one manager to 50 to 80 people. And part of the reason for that is technology. You can manage them through their iPhone partially, their schedule, their timing, discipline, pay, all that stuff.

14:58So it's a tale as old as time. It's just so much more intensified now that lots of things are digital, that it's kind of a difference in quantity amounting to a difference in quality, I think. Is there a way that organizations can kind of protect that idea of the, you know, the learned skill and the knowledge base inside the organization? Or are you really actually saying those are obsolete in some ways? Yeah, no, I hear you. And the back third of this book is what do we do now? And so definitely I have evidence to prove from studies that folks are fighting for their skill out there. They're not doing it in an organized way.

15:35They're not quite aware of what they're doing. I'm doing my best to report on things that are working. So individuals can do things every day to protect their skill as they engage with these technologies, for sure. I have a post on my sub stack called Don't Let AI Dumb You Down, for instance. There's some things that you as a user of that technology can do to avoid this subtle slide towards B plus territory if you're not careful. In fact, nudge yourself up in towards A plus. You can, in fact, not just tread water. You can actually use this tech to enhance your skill by the end of every session.

16:08It's just we're not doing it. So that's one level. But tactically, managers in many ways and people who run businesses have all the tools they need around handling this new tech. If you went to a top flight MBA program or more likely learned some great lessons outside of academia about how to mount and run a business and lead people, your tactics for dealing with new surprising events in your environment are probably just about as relevant now. It's just everyone's kind of got their hair on fire running around saying, when you know perfectly well how to run a healthy experiment with something that could be good or bad for your business right now.

16:47And what does that take to make sure that folks aren't just trying it all on their own right now? Which is not a terrible thing, but it's just like yelling fire in a crowded theater. It's better to get folks organized. And what does an organized, healthy experiment in a business look like? By the way, that's aggressive. You've got to be aggressive. Fine. But focused. So sure, there's stuff in the book and also on my sub stack about let's just everyone remind ourselves. We know a lot. We know a lot about how to handle change and technology, technological change. Let's just put it to work. You know what I've been telling people for the last, you know, five, well, probably 30 years.

17:22But, you know, it's really ramped up the last five years is instead of saying, how can we use this technology to do something we're already doing faster or better or whatever? How can we actually ask better questions? You know that that becomes our job, right? Yes. And in fact, part of the way you do that is the temptation immediately is just to use it to, I know how to do X. I'm just going to do X about two times faster. Right. Which is, boy, is that entrancing. It's amazing. You can get it to write a memo in five seconds when it would take you 30 minutes. Jeez. And that's where folks stop. That's the difference between top performers and average performers.

18:01The top performer goes and says, what could I do with this that I would have never even dreamed possible before? For instance, I made my master's students who are most of them fearful of coding, had no coding expertise. And I said, you have three weeks on your own, no help but from ChatGPT to become a data analytics person, analyze this data set and do plots in Python, coding a solution in Python and then post it on GitHub, which is a shared software manipulation platform. And they looked at me like, are you joking me right now? What kind of class is this? This isn't computer science. And I said, go.

18:41And they all did it. And by the end, I post this on my Substack too. In the beginning, I have their ratings on how afraid they were, what they thought. And with that nudge, mandatory, you must go do the thing that you thought was impossible before. and they all did it at the end. That was a shock to all of them. Like hot diggity. I had no idea. I thought this was just kind of fancy autocomplete, like fake my head kind of thing instead of change the world. I wasn't thinking broadly enough. I wasn't asking the right questions. Like what could I do with this that I couldn't do before? And if you can make yourself ask that question and then go for it, try something really crazy, you're going to fail.

19:20It's going to be a waste. You'll in three nights, or maybe 72 hours of your life. But you'll learn a lot about this new world we're entering that most folks will not have under their belt. Yeah, and I think to some degree, what you're describing is not about the end result. It's about the journey, right? It's, and really in your bones skill, right? What you have at the end of that is not just, I know, quote unquote, a little bit about AI. It's like, no, I've actually, I tried to decode sperm whale songs. I know I talking to somebody who just took my challenge and tried to do that. And they got some meaningful interpretation of the data.

19:57It's not as good as this paper that just got published. It actually doing that work by scientists who, by the way, four years ago, got the memo and used AI to decode sperm whale songs. Another person wanted to sub Bruce Springsteen in for Luke Combs in that Grammy performance with Tracy Chapman. They got pretty far. And this person didn't code before. So yeah, that will, you will know, like in the deep sense, you'll have skill that will inform your decision-making and leadership in ways that most folks won't. Well, Matt, I appreciate you taking some moments to stop by the Duct Tape Marketing Podcast and talk about the future, I suppose.

20:33This is what we were talking about, right? To some degree. And strangely, the past. Yeah, exactly. Thousands of years ago. And I'll just toss in for your listeners only, I posted the first chapter of the book online. If they want it, duct tape marketing.mattbean.com. It's sitting right there. They can just go and grab it. Hopefully the book is helpful to them, but the future is now. I mean, it's, but I agree. It's an old dynamic too. Yeah. Awesome. Again, appreciate you dropping by for a few moments and hopefully we'll run into you one of these days out there on the road.