When Agents Make Bad Supply Chain Decisions at Machine Speed W/ Russell Halper

0:01Hi, I'm Kelly Barner and this is the Art of Supply podcast. Every business story has a supply chain angle and those stories deserve to be told. This show digs deep into news headlines and interviews thought leaders, executives, and newsmakers to capture their perspective in their own words. With no agenda to lead us in any particular direction. We'll offer you objective coverage of geopolitics, innovation, and competition. Facts will always take precedence over hype. The rest is up to you. Today on the show, I'm glad to welcome Russell Halper as my guest. Russell is the founder and manager of Insight Kitchen, an AI-native consultancy that turns data science into business solutions that improved supply chain revenue management and integrated business planning operations russell thank you so much for being with me kelly excited to be here thank you for having me yeah absolutely so two questions for you as we get started one i want to learn a little bit more about your background and two explain the name insight kitchen what a cool name for an ai native consultancy.

1:17Thank you. I'll tell you exactly how we got to that name. If I go far enough back in my career, I actually started in applied math. My entire career has been at the intersection of originally data science and supply chain and now AI and supply chain. But if I go far enough back, I was studying chaos theory. And I realized at some point I didn't want to continue with that path. I didn't want to be in academia. I didn't like to teach, research, publish, or be in a math department, which were minor road bumps. But I had moonlighted in the business school studying a field called operations research.

1:54Now we probably, that term still exists, but probably a more modern term would be data science and supply chain or supply chain AI. And really enjoyed that. So I moved over, I finished my degree in the B school. I was working with one of the large parcel delivery companies during that time and another tech company doing some work in the background. Finished. I'm from Silicon Valley, had a chance to go back there or wanted to go back there and ended up being the Great Recession. And I was fortunate enough to land back in California with a large consumer goods company inside of their supply chain and got to spend a few years there doing everything from hands-on work with production scheduling and building the systems around that forecasting dc operations transportation was a great learning ground and thinking about that intersection of data science and supply chain inside of an organization but if you go back then you typically topped out at at director there wasn't a lot of places you can go but if you were in the services world there were a tremendous uh number of really interesting opportunities across a broad range of companies so i left and i joined a little 15-person consulting firm Palo Alto and end-to-end analytics.

3:08Over the course of eight years there, we grew from 15 to 100. I went from consultant to partner. We worked across a number of very well-known companies in high-tech, consumer goods, retail, automotive, and some other sectors as well. And at a certain point, we had a chance to join a larger company. So we were acquired by one of the large consulting firms. I joined there first running a global supply chain solutions program. Then in, let's see, when ChatGPT was released, they asked me if I could help build the West Coast Gen AI practice. So we helped to scale that. Ultimately what, and this is where the name comes from, one of the challenges I felt like was it was always about what technology do we need to put our customers into.

4:01It was never about the business problem first. And I saw sales cycle after sales cycle where the solution for any problem was some software platform, some type of data transformation, some type of something, but it was more of a implementation than a solution. Okay. And ultimately, when I looked at the things that can be done now with where technology is, where AI is how it accelerates that the technical part of a solution, which isn't the whole thing. I was looking around and thinking to myself, you know, it feels crazy that we can't literally said, like find the right recipes for our customers rather than actually bringing them to, you know, bringing them to a solution.

4:43That's where the idea of Insight Kitchen came from. It's that we wanted to think about that we're here to find the recipes our customers need to be successful with data science and AI and supply chain and revenue management. And sometimes that might mean knowing what's the right type of planning platform to introduce them to. Sometimes it might mean building something completely custom or some combination of those things. But ultimately where we really exist is to think about how do we find that recipe for our customer success with AI and supply chain. And that's where the name came from inside kitchen.

5:21Well, I'll pull back to one of your earlier references and say that if I ever was possessed with the terrible idea of opening a restaurant, Chaos Theory would be an excellent name for it. If I ever offered a cook for you, Russell, just say, no, thank you. It's okay. I already ate. But it's interesting when you talk about the role that data plays in all of this and that people are looking for the recipes, what I tend to hear in conversations is that the desire to implement AI comes from something, okay, I'm going to say emotional. This may or may not be the right word. Maybe subjective is a more accurate word.

5:58An executive has said, what are you doing? You go to a conference and you hear your peers talking about what they're doing. And you think, uh-oh, we need to get on this bandwagon. Then a few steps down the road, you look at your data and you think, uh-oh, maybe I should have started here instead. As a data person, as you see more and more companies starting down the serious AI journey, are we looking at data in the right place? So am I right? Are we starting too much from sort of that subjective, gotta do it, shiny object, and then coming around to the data later versus letting the data advise us when and where to start with applied AI?

6:39it's um it's a really interesting question on where do you start and when you're talking about ai being something everybody's trying to figure out every single level everybody's doing it's to me that instagram moment of you're seeing everybody's highlight reel of what are the things that they're doing but you're not seeing what the real reality is on the ground so you know somebody goes and publishes hey here's the thing we've done but it was a tremendous amount of work to get that right if it is even right. So the data question is a really interesting question. And it comes down to like, can you get good insights from bad data?

7:20Sometimes it is a barrier to being able to do the things that you want to do. Sometimes it's not. I think most organizations, there's kind of two categories generally in the world. One are organizations that they know they have bad data and the other organizations that don't know they have bad data. I think there's very, very few organizations where they have outstanding quality, data quality and maintenance across the board. What I normally advocate is with data, with AI, with any of these things, start with what are we trying to drive as a result from a business outcome? Yeah. And that's not a different conversation than it's been with any other generation of technology in the past, whether we're looking at agentic, machine learning, cloud.

8:09I mean, at the end of the day, it comes down to what are those pieces of addressable spend in my business or the efficiencies that I can find by being able to bring in a certain type of technology. And then if we have a sense of what that looks like, the question becomes, can the data support it or do we need to pull the data with us to be able to accomplish those outcomes? My personal philosophy is generally not to start with data as the primary business focus of what we're trying to do, but more so to start with what's the outcome we're trying to achieve and then trying to figure out what's the gap we need to get to from the current state to drive that outcome when looking at the data.

8:49Yeah. Now you talked about initially working with generative AI, which now at the speed of AI is like, you know, we've all just absorbed it, right? It's part of our daily life. Now there's a lot of focus around agentic. It's all AI. But to what extent have the either data availability or data quality expectations and requirements shifted as we've gone from, you know, help me write this email or business case in the voice of a pirate to actually help me do this analysis and potentially even make autonomous decisions on behalf of the organization? Has the quality requirement or the abundance of data shifted as what the AI can do has expanded?

9:41The way the data, there's a yes and there's a no answer to this question, right? Like there's one world where I can leverage that data for many more things than I was able to leverage it before. and because of that, issues within the data, my data quality that I have, and there's a number of measures around what data quality can mean, can be exacerbated in more ways as I look across the enterprise. The other way to look at it is to say, I have way better tools now than ever to be able to work on my data quality. And some of the most interesting things we've seen recently and how companies are building data ontologies, right, working on better data management practices in the organization are actually leveraging and the capabilities to be able to better maintain, better understand that, as well as being able to leverage agentic capabilities to have some intuitive understanding of where there might be a gap in the data that it's working with.

10:54and surface that in the recommendations or at least understand it's there as it's going through whatever other recommendations, workflow, advisory, whatever work you're asking the agents to help you with. Now, you talked about things being outcome-driven. So figure out what you want to do, right? And then find the way to get there. Can you share a few supply chain specific examples? So what are some of the desired outcomes that companies are looking at and saying, okay, this is what we want. And in order to get there, here's the data we have and the AI technology that we're going to use. So there are, in supply chain, we're ideological and physical supply chain.

11:39But I think more than anything, that Instagram moment in supply chain is really prevalent. So we were at a conference a month ago about speaking with one of our clients, which is a large semiconductor manufacturer. And this is a big conference, 11 ,000 people, 1 ,000 talks. It's an AI conference, not a supply chain conference, but it's cross-functional. So it's a great representative sampling of what's the penetration of AI across not just industries, but functions within industries. If you look at this, of the 1 ,000 speakers, and look at all the talks there, which is less than 1 ,000, but still hundreds, there were three talks on supply chain.

12:28And I think what it shows is that supply chain is still somewhat underpenetrated from the lens of how do we bring Agenta capabilities to bear. And so what we often see when we're looking at supply chains and we're talking to leaders, it's still very use case driven as to what are they trying to accomplish by bringing in AI. And our advice is always to think about, generally, what are the issues that you haven't been able to solve before or solve easily with your current iteration technology? And can AI play a role in helping to address some of those things? the theme that we always come with is we tend to work from the business back.

13:10So often when clients come to us and they're talking about, Hey, we want to help you want your help to build a roadmap. We're starting with what are the, what are the, what's the, where's the business have opportunities? Where's their leakage? You know, areas of address will spend. Where are the places where maybe I need an army of people to go and, you know, validate that every transaction is accurate because I'm a food and beverage distributor and I have many, many low value, high touch transactions. I just can't hire people to go look at all those things. And there's other times when people are going to just come to us and say, we want to think about building a truly autonomous supply chain for planning.

13:55And we have to help them understand what does that really mean? And how do we pull away the hype from that to things that actually can't be done today? I'm not a big fan of that term, different conversation. But we're always starting with our clients with where are the areas of addressable spend? Where's the business value? What are those challenges that we couldn't get after before? And again, then bringing AI to the table in the way that it can help support those things versus say, like, you know, I got to use AI for something. It's like, great. Well, let's start with the things that are challenges, right?

14:32If something's working perfectly, why throw AI at it? It's not going to necessarily add value, right? Go work on the hairy, tough issues and see if they can chip away at those. Yeah. Now, have you found yourself in those conversations where maybe people come in assuming that AI is going to be part of the solution? And you say, actually, AI is not the answer for this particular challenge or desired outcome.

14:58Yes, but also, I don't know if you've seen this, there's also times when like the letters AI have taken on a broader and broader meaning of what are the things that are under that umbrella. And there's a lot of old things that are now being called AI. And so when somebody's talking about wanting to leverage AI, what do they actually mean when they're talking about that? That is a great point. Yeah. Especially when we're looking at, I mean, any category of technology, but procurement and supply chain. In my mind, I think of it as sort of the breakfast cereal, now with more marshmallows, marketing effect.

15:35Like literally everything's AI now. Absolutely everything. It could be like a squirrel running in a wheel. And it's going to be promoted as some form of AI. But you're absolutely right that when somebody says, oh, well, can't we just bring in AI to solve this? I mean, it's almost like looking for, is there a magical, instant, easy way that this can just get cleaned up? And I would bet more often than not, even with the AI, it's certainly not magical. It might just become possible. I think it's got a really great brand, right? If you were to take the letters AI away from it, there was a big company that had a series of commercials and they called it Bob, right?

16:16And it said, Bob, go at it. I think it's a nice framework because suddenly you can get away a little bit from the hype and actually thinking about, what's it actually going to do and how does it help me?

16:29And organizations we often run to want to throw agentic capabilities sometimes at places you shouldn't be using agents for. But there are other great things in the world that allow us to solve those problems with incredible ease. And sometimes there are issues where they want to use AI, but the reality is it's just not there yet. There's a framework I sometimes use, which is, let's pick on forecasting. versus AI, right? So I wanna build a machine learning or statistical forecast that's gonna forecast my demand. Okay, there's a lot of battle-tested, tried and true techniques to do that. There's some newer ones, but there's a lot of ways to do it.

17:24Regardless of which way you do it, normally what we do is we have some measure of forecast accuracy. And what we wanna look at is how good is your forecast today? And how good is the forecast, whatever algorithm we're using is creating and know that we're actually getting better. So I went from 30 percent error to 25 percent error in my lag to DC level forecast. If you think about that within the agentic framework, first of all, it's hard sometimes just to measure our agents doing as good of a job as humans, because a lot of it is workflow or not necessarily numerical. But the second part is today it's often generally accepted where you may have agents that are doing 90 % as good as a human, 95 % as good as a human.

18:18And that's a good, that's not a bad bar. That's good. So what does that mean though, right? We have forecasts where the bar is they have to do better than people. And we have agents where the bar is they have to do nearly as good as people. And what does that now mean on how I'm going to use them in my supply chain? if I'm using agenda capabilities to say make decisions am I getting to decisions that I might not get to otherwise because I don't have the people so even if I'm doing 90 % of good as a person I'm adding value so I'm looking at stuff that I might not be able to ever look at otherwise and I can cut revenue leakage or increase customer service or you know have better material availability rates or some of those things or am I just making worse decisions way more quickly than I ever would before and they're compounding and driving me down the wrong path.

19:08And so I think it's an interesting question right now as we think about how do we leverage HNT capabilities with where the technology is today and probably realistically where it is one or two iterations in the future. What does it mean when you can spend an awful lot of time on it and it's still not going to quite do some of the things that people in some cases are able to do? Yeah. Now, if we use the example of that conference that you spoke at as sort of a rough example, you talked about maybe there's some underpenetration of AI into supply chain. Why do you think that is? Any guesses? I mean, it's been a space operationally that's gotten a lot of attention over the last few years.

19:48It's complicated. There's lots of data. It's certainly messy. But why do you think AI hasn't made more inroads?

19:58Well, there's first of all, like in the global world of where AI can make an impact, some low hanging fruit, and that's been the places that it's been the most penetrated, right? So if you look at writing code, there are very, very few developers or data scientists or AI engineers who are going to write code without some type of assistant with them, because such an incredibly powerful technology. When you look at supply chain, there are tons of physical constraints and process and things that come into play that are really hard to quantify. And so when you're writing, let's see, I'm going to go to writing code.

20:37I hope I get using chat GPT to help me write an email, right? There's a lot of ways to write the same email. If I'm using AI in the context of physical AI, I have a robot or a driverless car that's getting real world feedback. There's that immediate interaction, feedback of if it's doing good or bad. In a supply chain, you have physical operations, but there's not this real-time physical feedback into what the AI is doing often. Or if there is, there's a lot of system architecture you have to span between the two. And so it makes those implementation timelines longer and more risky. And you have to figure out how does the, I really understand the physical world is interacting with that actually having those touch points in the physical world.

21:27So that's part of it. I think supply chain technology has always been long, like relative to some other parts of the business, have longer implementation timelines, just the complexity of running supply chain, right that's not going to be any different with ai uh at least relative to putting ai in other parts of the business that may be more transactional and um you know in general a lot of the solutions that we see in supply chain are still built on previous generations to technology with ai being integrated into them so you know there's certain uh i won't say names but there's certain uh frameworks on how to measure solutions where to even get into that framework, you have to have existed for 10 years.

22:14So that means the solution had to exist at least five years prior to, to the release of chat GPT, the first version that was publicly released. And that was, first of all, a ways away from something that's actually like a functional business tool in a supply chain. And realistically, most of them existed long before that. And so you still have generations of technology that are behind, As much as you mentioned, like put the label of AI on them. And you don't have, like, there's still that effort of how do you build in the complexity that you have to have to operate a supply chain and these solutions effectively.

22:49Yeah. Now, if given that, and I think especially your point is interesting about the difficulty of having an immediate feedback loop, because there are so many physical constraints in even in interacting with supply chain technology and the materials that are being moved around. If somebody is still trying to build out their own organization's AI roadmap, how can they make sure that it's, let's say, optimistic enough to get people excited, but keep it pragmatic instead of allowing it to be so aspirational that everybody says, yeah, this is never going to work and the project ends up being sort of pushed aside or underinvested in?

23:30I like to think about roadmaps as things that

23:37is it about the destination or is it about the mile markers along the way and so when we have worked with organizations on building their supply chain road map we usually have a bigger emphasis on what are the near-term wins that we think we can achieve and if we can get those wins, that allows us to then think about how do we have the right to go find the next one. Yeah. I would encourage most organizations to think about this as like a living, breathing roadmap. Technology is evolving so fast. There was a big movement at some point around we want to have connected solutions across our supply chain.

24:16We've seen some organizations decide they want to swing back to more of a best-in-breed approach is they think about, hey, we don't know where technology is going to be in three years or five years. And of course, like the tools that are around for bringing agent to capabilities in organizations are also evolving very, very quickly. And then those legacy players continue to build their capabilities into them as well. And so it's really hard to have that crystal ball about where are we going to be in so many years? But we know the business problems are still going to be the business problems, right?

24:50If I'm selling potato chips, right? I'm still going to be figuring out how do I make sure I have the raw and packed, I have manufacturing efficiency, I have an efficient inventory, like distribution framework and footprint, I have the right products to compete on the store shelf, right? Those are somewhat intransient elements of that type of a business. Yeah. And we want to make sure that we're evolving a roadmap that's thinking about how to address those, but still is understanding it's about the mile markers on the way, not about where we're trying to get to at the end. Yeah. And I have to think that it's probably not one of the mile markers, but I'm sure it's a critical thing for getting there is the kind of user experience that any of these interfaces have.

25:35And it's interesting because, you know, in some cases we talked about, okay, what's the data or what's the LLM that might underpin something. But increasingly, I feel like these are becoming the actual interface that people deal with and other things are sitting underneath the LLM. The big concern with all of it, of course, being adoption, because you're never going to get those business outcomes. You're never going to get the ROI without that. What role does that user experience end up playing? I mean, are we very tolerant right now because it's so exciting and it's early days? Or are people starting to say, no, you know, I have a preference.

26:10Like this one's easier than that one. And I'd be interested in that sort of human touch point around getting to those outcomes. I have a friend that's at a, he's a head of transformation at a large retailer. And he was telling me at some point, he's like, I don't get it. Anytime there's a prompt in any type of solution, everybody's suddenly excited about buying it. He's like, the rest of the solution could not function very well. He didn't say it as politely as this. He's like, but if it's got the prompt, people are interested. I love the concept of AI for UI and how do we make people's lives easier.

26:47That's been one of the challenges with adopting new supply chain solutions for many, many years, right? Well before AI is to put something new. You got to make people's life. It's better if you can make people's life easier as well. And so if I can use a prompt to help me answer some of the questions that otherwise I'd have to dig through reports for, that could be a huge win for the user community. If I can use it to structure scenarios that I then want to run, if I want to evolve a procurement plan or a demand plan and be able to run those scenarios by typing on a prompt, that could be another place that's a huge win for an organization.

27:30But if you can make people's lives easier as well as make the outcomes better, that's one of the things I think drives user adoption. But it's easy to say on a podcast, right? Doing it, I think, involves a lot of skill and trial and error to make sure you get it to the point where you can get over that hurdle to drive enthusiasm about something new versus people feeling like they got to do two jobs as they operate the old solution and the new solution. And it was classic challenges when you're looking at implementing something new. Yeah. Yeah. And, and yet it's, so I'm not an overly technical person.

28:10Like I actually do pretty okay with some of the, the easier user interfaces that allow you to build something or train on a scale or that kind of thing. But I'm curious just to sort of bring this all full cycle. So you talked about starting your career in applied math, which I am no more qualified to do than run a restaurant. And I know you got away from that into some of this more operational work. I'm curious about which piece of your thinking. So there was something about that foundational skill set, that thought process, the way that your logic works that led you down this career path. Everything about AI from my perspective is very softened for the user.

28:51Which piece of your mathematical or logical thinking do you find that is still completely key or core to the work that you're doing today, even helping non-mathematical people like me get through some of the benefits of these solutions? Yeah, so much of supply chain is very quantifiable, discrete, like data-driven. And that's what drew me, and constraint, right? You can only do things that you can do in the real world. And even then you have processes and operations that, you know, two companies that might put this, you know, competing products next to each other on the store shelf may operate entirely differently.

29:38That's what drew me to it. And that's what it continues to draw me to it is there's this very data rich, quantifiable business problem. Often that is something we want to go chase. And there's this strategic element to that as well. How do we think about balancing customer service and cost to serve, right? Some of those things that also become some of those critical considerations beyond just the quantitative considerations around supply chain. But I don't think any of that's gone away. And we look at AI. AI, large language models, started with language. And that's one of the reasons I think they're great for writing emails.

30:28They're great for code. They've evolved them for looking at images and even video. They're great at writing the code to go look at data. They can even now do a good job of ingesting data and understanding oftentimes what it might mean, especially if you have the right context or ontology that's helping it. But when you get to complex, highly constrained decision-making processes with lots of trade-offs, and those are pervasive throughout supply chain, they tend to struggle. and they'll tell you very confidently they have an answer but it's not a answer that necessarily is based in you know framework that inspires confidence at scale yeah repetitively over a long period of time and that's where i think the challenge is and how do we think about leveraging some of these things and sometimes it's it's the old stuff and the new stuff together that drives great outcomes.

31:29Yeah. So this has been so interesting, Russell, looking at the way that you think about it and bringing it in, your advice about focusing on outcomes. If we want to leave people with a piece of advice, and given most of my audience is in this underpenetrated, lots of constraints, no physical feedback loop kind of world, I'm not going to ask you to look in a crystal ball, but do you have an actionable piece of advice for people today? What's a realistic thing that people should be doing, should be thinking about today to make sure that they're at least on that journey, right, to the larger roadmap and sort of moving in the right direction from an experimentation and development perspective?

32:17Building the roadmap is important. and going back to it's about the milestones in the roadmap, not about the destination. But what's the framework that you're using to evaluate what success looks like for each one of those milestones along the way is really critical. And too often we see organizations want to throw AI at something and they get started building something, but they don't have a way to evaluate up front. What are we trying to do? Is it going to drive the outcome we want to get to? And then that clouds, not just like, do we get the right thing out of it? What are the decisions that we're making along the way to get there?

32:54If we don't have that North Star, the entire journey becomes harder. And so building more clarity up front, even for the little wins on what actually is that win, will make the entire process of experimenting, of building enterprise capabilities, and everything in between

33:17more predictable for the organization and an easier journey. Yeah. So working our way towards that recipe that you talked about in the beginning, slowly figuring out the pieces, getting the ingredients together, learning as we go. Russell, it's been a really interesting conversation. I appreciate you sharing your honest thoughts and your experiences. I'm actually kind of excited, given all the discussion, that there's still such a huge opportunity in the supply chain operations space. But thank you so much for being here to take the audience on this journey and share what you know. Thank you for having me, Kelly.

33:55It's been a pleasure.

34:01Art of Supply is part of the Art of Procurement family of brands. To ensure you never miss new content, I invite you to sign up for our newsletter at artofsupply.com. If this episode of the Art of Supply podcast made you think, please spread the word. You can support the show by giving us a thumbs up, a star rating, or a review wherever you find your podcast audio. Join me next week for another deep dive into the art of supply. 3 5 You