Cracking the code of failed AI pilots

0:28Welcome to the practical AI Podcast drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

0:48Welcome to another episode of the Practical AI podcast. In this fully connected episode, It's just Chris and I, no guests, and we'll spend some time digging into some of the things that have been released and talked about in AI news and trends, and hopefully spend some time helping you level up your machine learning and AI game. I'm Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? Doing great today, Daniel. So much happening in the world of AI. Holy cow. Yes, yes.

1:31Lots to catch up on. There have been a number of interesting developments in the news that maybe people have heard about. And yeah, it might be good to just kind of distill down and synthesize a little bit in terms of what they mean, what they signal, how people can think about how certain things are advancing. So yeah, looking forward to this discussion. Any things that have been particularly standing out to you that you've been hearing about, Chris? Sure. I think the thing that has really been noticeable in recent weeks has been so many people that are both in the AI world and outside of it, but impacted like everybody is, are talking about jobs.

2:18And, you know, we've talked about the impact of AI on jobs many, many times on the show over time, but people are really, really feeling it at this point. The job market is pretty tight. I've talked to lots of people out there looking, you know, whether they're currently employed or whether they're out of school. And particularly, there's a lot of people in technology coming out of university that are really struggling right now. And I believe there was a report recently from MIT that highlighted that. Yeah, it's interesting that we spent a good number of years on this podcast, of course, talking about, you know, occasionally talking about some of the wider impacts of this technology, you know, within a certain company or industry.

3:05Now this is kind of like a global across all people thing, you know, all sectors, you see things being hit hard, especially in, you know, maybe it's sales and marketing or kind of junior developer type of cases. If I remember right, Chris, the MIT report that you're talking about, which we can link to some of the news articles about the MIT report. I don't think I've actually seen the actual MIT report yet. So I guess our listeners can keep that in mind. But one of the things that I've actually heard on multiple calls. And I was at a last week at a AI corporate type event where there were a bunch of corporate leaders, and they were certainly talking about this, that one of the things talked about in the MIT report was that 95 % of AI pilots fail.

4:03And that I think has generally spooked a lot of business leaders, investors, lots of different people across industry, just the level at which these kind of 95 % of AI pilots fail. What's your thought on that? I think it's a weird juxtaposition right now that we're in, and that that's accurate, that you're having a tremendous number of Gen AI in particular efforts fail, but at the same time, companies are holding back on hiring junior devs out of school. And so you have this weird mesh of people going hardcore on trying vibe coding and things like that, but with very limited success, struggling to get it adopted.

4:52And at the same time, they're making bets on the future by not going ahead and bringing in the junior level developers that they always have, which kind of leads to an interesting kind of what-if situation in the months ahead. You know, junior developers eventually, historically, have turned into senior developers. And right now, companies are betting on those senior developers with these new AI capabilities over the last couple of years to make up for that deficit, hoping to save money. But if you're failing 95 % of the time, it puts things into an interesting place. Yeah. One of the things that I'm just reading one of these articles about the AI pilots.

5:36And one of the things that it highlights is that this isn't from the report's perspective. It's not that the AI models were incapable of doing the things that people wanted to prove out in the AI pilots. But that there was a kind of major learning gap in terms of people understanding actually how an AI tool or workflow or process could be designed to operate well. So I find this very, very prevalent across all of the conversations that I have, especially in workshops and that sort of thing, is there's kind of this disconnect. And people are used to using these kind of general purpose models, maybe personally.

6:29And there's this concept that the way I implement my business process with an AI system is similar to the way I prompt an AI model or a chat GPT type interface to summarize an email for me. And that is always due to create some pain. Number one, because these AI models only sometimes follow instructions. But number two, your business processes are complicated, right? They're complicated. They rely on data that is only in your company and probably has never been seen by an AI model unless you accidentally leaked it. jargon, all of those sorts of things. And number two, these business processes that people are trying to automate or create a tool around, often the best thing for that is not a general chat interface.

7:32It's not like you want to create a chat interface for everything you want to do in your business. No, actually, in one particular case, it may be that you want to drop a document in this SharePoint folder. And when it's dropped in the SharePoint folder, it triggers this process that takes that document and extracts this information and compares it to information in your database and then creates an email and sends out an email to the proper people or add something. These sorts of processes are not general chat interfaces. So people are coming at it like, oh, I know how to kind of prompt these models to do some things.

8:12And so they try to kind of build or prompt these models in a certain way without the kind of, I don't know why there's such a disconnect, but without the understanding that really what they need is maybe a custom tool or automation. They need data integration, data augmentation to these models. They don't just need a model plus a prompt. And I think that that's a pitfall that I see, unfortunately, very, very often. So it's not super surprising for me to see this kind of highlighted in the MIT report. Yeah, I agree. It's I think, you know, to your point, these chat interfaces, you know, it's kind of becoming the universal hammer in most people's head and everything is starting to look like a nail for that hammer to hit.

9:03And they are neglecting a toolbox full of tools that give them the right software components for getting their workflows put together the way they want. And so, yeah, I think it's a certain amount of over-expectation that is then exacerbated by choosing the wrong software approach or an incomplete software approach to try to get the job done, that workflow, that business workflow done. So that's certainly my sense of it. I think a lot of these are driven top down, you know, by excited executives that haven't taken the time to really understand how to optimally use these tools. Yeah. And another thing that was kind of interesting in the study is this kind of just prompting of the models generally kind of failed.

10:02There's kind of a knowledge gap on how to integrate data and how to build custom solutions in a way that could succeed in kind of a POC sort of thing. But I actually don't kind of agree with the premise of the, you know, that this should spook investors, you know, from kind of AI, maybe some AI companies, but AI companies that especially are verticalized in general. And, you know, I'm part of an AI, I lead an AI startup, so I may be biased, but our AI startup isn't one of these kind of verticalized application layer things. So I feel like I can maybe speak objectively with respect to those. It's really these kind of AI companies that are in whatever it is, healthcare or, you know, the public sector or education or finance or that sort of thing.

10:54They are putting in the work, I think, at least many of them are, to understand business processes and build robust AI workflows and tools that are fitting certain business use cases and that sort of thing. And I think one of the stats in the report were that a lot of the kind of trials of these sorts of tools did actually succeed kind of a majority of the time. And so there's this kind of major gap between on one, I guess, in summary, what I'm trying to say is there's this major gap between on one side you have this idea that all you need is access to a model. And then on the other side, these kind of pre-built, purpose-built AI systems for particular verticals that understand the business processes.

11:46There's a whole kind of gap in the middle because many companies, especially in the enterprise setting, will have to customize something. I don't think in the end they will be able to always use a tool off the shelf that works completely for them. If you look at any enterprise software, it's always customized at some level, whether it's manufacturing software, ERP software, CRM, whatever, it's always customized. I think that's where maybe this is highlighting that gap of companies not understanding the gap between having a model and a verticalized solution for their company. And that actually does require, you know, significant, you know, understanding of how to architect and build these systems, which unfortunately there's a skills gap and a learning gap in terms of people that actually have that knowledge.

12:44Great. Yeah, I think I think you're drawing out a great point there. And I know we've talked a little bit about this in previous shows where, you know, the model constitutes a component in a larger software architecture. And we know, as you just pointed out, that the expertise of those business workflows being integrated into vertical software stacks, you know, where it is designed to solve the problems and not just a chat box is really important to getting to a good solution that works for your business. and I think that there is I think this is where one of those challenges that we're seeing in a lot of folks out there in the business world is kind of forgetting that core tenet and leaping straight for the model will run my business from this point forward without that supporting infrastructure so maybe there are some hard lessons to be learned in the days ahead for some companies but hopefully that will happen

13:48well friends when you're building and shipping ai products at scale there's one constant complexity yes you're wrangling models data pipelines deployment infrastructure and then someone says let's turn this into a business cue the chaos that's where shopify steps in whether you're spinning up a storefront for your ai powered app or launching a brand around the tools you built. Shopify is the commerce platform trusted by millions of businesses and 10 % of all U.S. e-commerce from names like Mattel, Gymshark to founders just like you. With literally hundreds of ready-to-use templates, powerful built-in marketing tools, and AI that writes product descriptions for you, headlines, even polishes your product photography.

14:33Shopify doesn't just get you selling, it makes you look good doing it. And we love it. We use it here at ChangeLog. Check us out, merch.changelove.com. That's our storefront. And it handles the heavy lifting too. Payments, inventory, returns, shipping, even global logistics. It's like having an ops team built into your stack to help you sell. So if you're ready to sell, you are ready for Shopify. Sign up now for your$1 per month trial and start selling today at shopify.com slash practical AI. Again, that is Shopify.com slash practical AI.

15:16Well, Chris, there's a couple of things that I've been following with respect to the model builders, but it does actually connect maybe to this MIT report as well. because one of the other kind of common things that I see that companies, or a way of thinking that companies have when they approach kind of AI transformation, is they come at the problem of AI adoption kind of with the question of what model are we going to use, which I think is the completely wrong question to be asking for a number of reasons. First of all, if you're coming to AI for the first time with your company and you want to transform your company with AI and build knowledge assistance and automations and adopt other tools and build verticalized solutions.

16:19The model actually will shift over time a lot. So that's, I think, number one. There's no one kind of on the market that, at least right now, there's certainly many providers of models. There's a lot of good models. No one knows who will have kind of the edge on models in the future. And I think what we're actually seeing is that the model side is fairly commoditized. You can get a model from anywhere. The second reason I think that that's the wrong question to be asking is that if you're trying to build an AI solution within your company, again, think about that SharePoint thing that I talked about.

17:01I'm going to process this document from SharePoint and extract these things and send it to an email. You actually don't need a model. You need a set of models and potentially other things in the periphery around those models. So you likely need a document structure processing model like a dockling. You need a language model to process pieces of that. You maybe need embedding models to embed some of that text or do retrieval. You need a re-rank model because you've got to re-rank your results after doing retrieval. You need safeguard models because you want to be responsible and check your inputs and your outputs of your model.

17:42So once you start adding these things up, even for that simple use case of processing this document, you know, through SharePoint and out the other end, if you're coming to one of these proof of concept, you know, projects like we've been talking about and you're thinking, what is the model I'm going to use for this? and you decide, okay, the model I'm going to use for this is a GPT model or a LAMA model or whatever, well, you're already setting yourself up for failure. Because what you don't need is a model. What you need is a set of models. You sort of need an AI platform. You need an AI system that gives you access to multiple kind of different types of functionality, right?

18:24And so I think that that kind of perspective plays into this kind of POCs failing thing. And I have more thoughts about that related to the model builders. But do you think I'm off in that? Oh, no. How would you correct me? No, I have no correction. We've talked about this as well a bunch of times. And I keep waiting for, you know, I think we've had a lot of really in-depth conversations with people on the show in the past about the need for multiple models to tackle different concerns within your larger business focus and the software architecture that supports that. And I think as you look forward and, you know, we're seeing agentic AI, we're seeing physical AI developing more and more, and all of those require a number of different models to do that, you know, kind of obviously, you know, very different, distinct things that you can see in those spaces.

19:21And so there seems to be this hang up in the in the public about the model, you know, and which model and then I pick the model. And as you pointed out, that's completely the wrong thing. It's it's what is your model architecture look like as related to your business workflows and the job that you need doing and how do you do that? And as you were describing some of those potential models that one might have in a workflow earlier, as I was listening to your examples, I was thinking, wow, sounds a lot like software architecture. You know, it's just each component is invested with one or more models now in that component.

20:02But there are still many components that make up a full business workflow. And so I guess maybe because we talk about it fairly regularly on the show here, it seems quite obvious to me that that's the case. But clearly it's not. If you look at the business decisions that are being made out there, some of which is there is clearly a need at this point. You may have a senior developer type by whatever title you're applying, working and kind of sort of knowing software dev at some level and sort of vibe coding and putting their knowledge of architecture together for a solution. But if you're not going to bring in junior devs, then you're making a gamble that you're not going to need that at some point.

20:50And yet what we're seeing is, you know, per that report, a 95 % failure rate on using these tools at the current point in time we're at now on that. So it's there's some it you know, we've we had a recent episode on risk management and from a risk management perspective, I think that there are a lot of very risky decisions being made by executives, largely in ignorance, I think, of kind of understanding how models and software architecture fit together to your point. So, no, I'm I'm in violent agreement with what you were saying a moment ago. Yeah, this idea also that you kind of bring in a model also produces a little bit of problematic behavior around adoption of models, particularly in private kind of secure environments.

21:46And I know this one from experience where you kind of think, well, which model am I going to use? And then you think, well, there's a couple of categories of models, right? There's closed models, there's open models, like closed models being the GPTs or Anthropic or etc. The open models being the, you know, Llama or DeepSeek or Quinn or whatever. And, you know, you have smart people in your company. and whoever, Frank over in infrastructure, he's like, yeah, I can spin up a Lama model in our infrastructure, right? And there's innumerable ways to do that at this point. So you can use VLLM or you can use Ollama or whatever it is, right?

22:33I can spin up one on my laptop. And so you spin up the model and then you're like, all right, well, let's build our POC against Frank's model that he's deployed internally because we now know how to do that. But again, it's not so much that Frank did a poor job and the deployment is bad or the tools are bad, like VLLM or something is very powerful, but it's not a proper comparison because what you've deployed is a single model, not a set of AI functionalities to build rich AI applications. You now have a private model, which again, only does what a private model does. That one particular model, it doesn't give you that rich set of AI functionalities.

23:21And so it's not really a knock against open models. What it is an indication of is that you maybe shouldn't roll your own AI platform, right? So there's a lot of things that go into that. And there's various ways to approach it. But I think that misunderstanding of what model do I need also impacts the perception of these open source models, because most of the time when you deploy that open source model, you're only getting a single model endpoint versus kind of a productized AI platform. Yeah, I guess, you know, I know that in your own business that, you know, you do help bridge some of that gap there and some of the things that you guys do.

24:06But in general, if you're talking in the broader market, how do people – they have service providers out there that can give them some of those services. But as they are going and deploying, and in a sense, we've always encouraged open source and open weight models out there. We've talked about that a lot. And we like to see that. And yet there is this skill gap or understanding gap that you've just defined in the business community of, yes, you have these capabilities, but you've got to connect all of your resources, you know, using the term in a very generic way, all of your resources together to give you the capabilities you need for your business to operate the way you envision.

24:54And, you know, that's definite falling down and understanding within that gap space. What are some of the different options how people can get through that gap? Well, I think one of the things that can be done is to approach this sort of problem from a software and infrastructure and architecture standpoint to what you were saying before. A lot of what we're talking about really kind of falls in that architecting side of things. And so I think from the beginning, the question is not, again, if you come to it with the question not being, what model are we going to use and who's going to deploy it internally, right?

25:37But you come to it from the standpoint of we will be using models. There will be many models. They will be connecting to many different software applications. OK, well, that changes the game a little bit in terms of managing that and making it robust over time. And there's very many capable engineering organizations that know how to scale, you know, bunches of services and keep them up and set up uptime monitoring around them and, you know, alerting and centralized logging and all of those sorts of things. But you never get to have those conversations if you kind of cut it out before you get there by just saying, we will have a model living somewhere.

26:20And so you really need to approach it from this kind of distributed systems standpoint. And once you start doing that, you start talking to the experts that are on your team. And there are very many tools, depending on the standpoint that the company wants to approach this from. everything from the company still managing a lot of what they want to deploy and using orchestration tools, whether that be like a rancher or something like that that's generic and not AI related, right? But they're used to using it. Maybe they're already using it in their organization and they can orchestrate deployments of various AI things.

27:01And then there's AI specific, you know, approaches to this as well. So I think it is really a perspective thing. And as soon as you kind of get into that zone of, well, we do need this software architecture, we need this kind of SRE and DevOps kind of approach to things, then you really have to ask some of the hard questions like, can we vibe code our way through this? What kind of software knowledge do we need to actually support this at scale? And I think what people will find is you do at the minimum, I think, still need that software engineering and infrastructure expertise to do it at scale well, or at least to kind of guide some of the vibe coding type of things that happen, right?

27:53So there needs to be an informed pilot to help guide some of these things and make sure the ship is going in the right direction. I think that's very, very well put. And I think, you know, kind of going back to the fact, you know, kind of combining this with the hiring decisions that we're seeing out there in the job market and kind of the collapse of the bottom end of the software dev industry, there is a lot of developed expertise over the course of a career. And while all of the discrete points of knowledge may be captured in various models out there, there's still the necessity of extracting what you need from those models in the right context and the right order.

28:44and at least for the time being in this kind of Gen AI dominated world, you have to have somebody who can provide that kind of architectural view, know how to provide the context to get the things you need from your 5 coding. I think people are finding small successes under that where they say, I want an app that does this thing. And they describe the app in great detail and the models that they're using will turn out kind of an app. It may or may not be architected the way for sustainability. And, you know, there's a whole bunch of issues that might make a very good prototype. But if you're not going to bring in junior level coders that in the future will be your senior level coders that have this knowledge, then you're kind of betting on today's talent, producing something, and you're hoping that your model gets the nuance of all of those components and is able to generate its own context without your expertise, which may happen, but it's a big gamble if you're a company right now.

29:47It seems to me a lot less risky to go ahead and continue to bring in some junior level developers for the purpose of growing them over time and being able to have that. And maybe at some point that does change in the future, but I think the companies that are doing that today are taking some fairly significant business risks that are largely invisible to their executives.

30:24Well, friends, you don't have to be an AI expert to build something great with it. The reality is AI is here. And for a lot of teams, that brings uncertainty. And our friends at Mirror recently surveyed over 8 ,000 knowledge workers. And while 76 % believe AI can improve their role, most, more than half, still aren't sure when to use it. That is the exact gap that Miro is filling. And I've been using Miro from mapping out episode ideas to building out an entire new thesis. It's become one of the things I use to build out a creative engine. And now with Miro AI built in, it's even faster. We've turned brainstorms into structured plans, screenshots into wireframes, and sticky notes, chaos, into clarity, all on the same canvas.

31:13Now, you don't have to master prompts or add one more AI tool to your stack. The work you're already doing is the prompt. You can help your teams get great done with Miro. Check out Miro.com and find out how. That is Miro.com, M-I-R-O.com.

31:33Well, Chris, I do think that there's some consistent news stories from other sources outside of the MIT report that kind of reinforce some of what we've been talking about. And also, I think are just generally interesting as individual data points. And I've seen a number of those as related to OpenAI specifically, in terms of if we just look at what kind of has happened with OpenAI in the previous number of weeks, some interesting things have happened. And I think that they signal some things that are, like I say, very consistent with what we've been talking about as prompted by this MIT report, just to highlight a few of those, and then we can dig into individual ones of them.