Hyperventilating over the Gartner AI Hype Cycle

0:28Welcome to Practical AI. near your users. Learn more at fly.io. What's up friends? Intel Innovation 2024 is right around the corner. Accelerate the future. Registration is now open and it takes place September 24th and 25th in San Jose, California. This event is all about you, the developer, the community, and the critical role you play in tackling the toughest challenges across the industry. Ignite your passion for AI and beyond. Grow your skills to maximize your impact and network with your peers as they unleash the next wave of advancements in technology. Here's what you can expect. Understand the emerging innovation and trends in dev tools, languages, frameworks, and technologies in AI and beyond to empower you and the solutions you're building.

1:18Get in-depth technical experience doing hands-on workshops, labs, meetups, and hackathons to collaborate and solve problems in real time. You can explore featured partner and Intel solutions. They have partners there, startups there, customers there. And Intel is showcasing the latest in products, services, and solutions across keynotes, tech sessions, and the show floor to help you meet your development needs. Collaborate with experts, learn and have fun. engage in interactive sessions to connect, get certified, gain unique ideas and perspectives, build long lasting networks, and of course, have fun and get inspired.

1:57Hear from leading industry experts, technologists, startup entrepreneurs, and fellow developers, along with Intel leadership, CEO Pat Gelsinger and CTO Greg Lavender, as they take you through the latest advancements in technology. Don't miss this chance to be at the forefront of innovation. Take advantage of early bird pricing right now until August 2nd. Register using the link in our show notes or to learn more, go to intel.com slash innovation. Once more, that's intel.com slash innovation or go to the show notes and click that link.

2:43Well, welcome to another episode of Practical AI. This is Daniel Whitenack. I am founder and CEO at Prediction Guard. I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? I'm doing fine. We got a fun one today, Daniel. This is going to be a good one. Yes, of course. It was wonderful not that long ago to be in the great city of San Francisco and run into our friend Demetrios from the ML Ops community. And I figured I'd just bring him along for another conversation. So Demetrios, how are you doing? I'm great, man.

3:24We're back. And I've got some bad news to break to you right now. I wanted to do it on air. Go for it. Just to get your reaction. Oh, boy. You can be vulnerable. This is how we build community. Yeah, I'm nervous. Yeah. So PredictionGuard, awesome. Congratulations on all the success that you've had. We're doing a data engineering for ML and AI virtual conference. And one of your colleagues, Daniel, filled out the CFP. I haven't gotten back to him yet, but I can't accept him. I just am way too full, way over my head. And as much as I want to, I'm going to have to divert him to doing his own special event.

4:06Basically, we're going to actually take what may have been a bad thing and turn it into a good thing. That sounds great. I'm looking forward to learning more. there we go you know i gotta i gotta make sure that you get all the love and shine you deserve because i'm super stoked at what you're doing yeah yeah well appreciate that it was great to see you and and you had your own event in sf how was that i do not recommend doing live events to even my greatest enemies if anyone out there is contemplating organizing an ai conference you can do it but i don't recommend it gonna hurt yeah yeah it's painful man but it was a big success it was just a lot of work leading up to it as you can imagine and we had fun and on the day of it was like i think over 750 people showed up a lot of great conversations a lot of fun like spontaneous sporadic meetings with people and that's the stuff you get at in-person conferences that it's really hard to replicate virtually.

5:12Yeah. You know what the secret is? The secret is it's AI and it needs a lot of hype. It really needs a lot of hype. There's one thing that we don't have enough of in AI. It's we don't have enough hype. If you had hyped it more, it would have worked. You know, I do a fair amount of hyping. And so for those out there that are sick of the hype, like myself i've only got myself to blame on this well uh chris you sent me um a very interesting looking hype filled chart the other day you want to go into what that was well i i will uh and i'm actually blaming it all on demetrius uh he was making fun of the gartner hype cycle and gosh i I hope they're not a sponsor because we're making fun of them today.

6:04And he was going through that. And it was funny. And I said, dude, we need to do an episode where we all analyze the Gartner hype cycle in 2024 for artificial intelligence. And we break it down. And we're going to assess it and decide what we think of those things. And we're not doing this in our normal, extremely serious manner. We are doing this in the fun way. And lest you don't know Demetrius out there, which I can't imagine because he's a regular guest on the show here, he is, in addition to being a brilliant guy in this field, he is also the funniest man in all of artificial intelligence.

6:42So this is going to be good. And we're going to dive into the Gartner Hyde cycle today and break it down for you. We're going to start with the real one, and then we're going to maybe make some adjustments to it. You know, Chris, you say making fun, but I mean, Gartner seems to have fulfilled their mission. I mean, we're talking about the hype cycle. We're going into it. So maybe their mission was fulfilled. We are their fulfillment. Yeah. Oh, my gosh. Yeah, we're hyping it up right now. We're hyping it up. Okay. And we're going to have fun doing it. I just have to say, yeah, please, if anyone knows how I can get a job doing this kind of stuff, just making up words and then putting them onto a wave graph, let me know because I would love this as a job.

7:31It just seems like it's too much fun. Well, let's see. I think surf's up. Let's hop on the wave and let's start talking our way through. Demetrius, do you want to lead off on some of your ideas there? I think the most surprising to me out of this whole graph, and for anybody that's not familiar with the hype cycle you've got the big like upward side and then it goes down and it kind of crashes and then it starts to climb back up and it's the traditional like and the two second version of that and i did in our in a previous episode i did a longer version uh when we were looking at some specific things on it but the two second version is new technology comes out everyone's super excited about it they think it's going to be the greatest thing since sliced bread.

8:21It doesn't live up to the hype. They get frustrated. They go, this thing sucks. And it falls down on the hype popularity side. And then cooler heads prevail and they kind of go, okay, well, maybe it can do something okay. And then it's into a reasonable sense of productivity. So that's Gartner in a nutshell. So the biggest surprise for me is at the bottom of the slope so after it's gone all the way up the hype cycle it's come down and crashed down and is at the absolute bottom that the trush of disillusionment exactly there is cloud ai services yes and for me that is the biggest misnomer because if anybody is making any money out of any of this and i guess maybe hype and actual money they're detached and they're very decoupled here.

9:13But for me, that was like, wait, what? There's no hype in cloud AI services. So bedrock out of there. Hype is killed. It's at the trough of disillusionment. Any type of SageMaker, if you're using that or Vertex, no, out of there. It's the lowest of the low. And so when I saw that, that was instantly like, dude, why are you even doing it? Yeah. I did not believe a thing that I read afterwards, but that was my thing. Any big surprises from you guys? I think your point on, if there's anyone making a killer amount of money on this, it's Microsoft, it's Amazon, it's Google. Part of my struggle here is some of these terms, like I could interpret them one way or another way, right?

10:02Like SageMaker, for example, which for those that don't know is uh it's kind of like a model deployment service within aws and there's various convenience around it and that sort of thing like that's been around for quite a while now like a very long time even before sort of the kind of piped gen ai stuff a long before but yeah so like is that a cloud ai service like that's been around for a huge amount of time or are we just talking about like hosted model APIs, right? They don't say. Which also, to be fair, have been around a long time. Like you look at something like OCR or translation or something like that.

10:42And in cloud services have been around for a really long time and are sort of ubiquitously used. It's funny that it's down there. I get your point. Maybe it's just like everyone knows that's where the cloud, that's where all the services are. We're all paying for them. Yeah. So does Does hype correspond to usage, I guess? Like in this chart, is it that people aren't hyping cloud AI services even if they're used? I think it's an emotional thing. You know, the hype side is, you know, how much people are talking. So maybe it's accurate in this context. There is nothing sexy about AI services in cloud providers.

11:19And maybe that's what they're getting at is like, yes, we're paying an arm and a leg. We're giving them all of our money, but there is nothing sexy. But productivity wise, it's definitely productive. I would think so. Yeah, it's very pragmatic too, especially for those people just starting. I don't know any easier way than to just grab an API from like Amazon Bedrock is just hosted model, hit that API like you would hit an open AI API, but now you have a suite of models, right? So that seems to me like a near miss. But then at the top of the peak is the other one that was a huge surprise to me because I've noticed this trend.

12:00I don't know if you guys have noticed it, but people who were formerly ML engineers, we've all converted into being AI engineers. And an AI engineer is so misleading because you don't know, is that somebody that is coming from like a front end development world? And now they do a little prompt engineering. They use a few frameworks and they can chain together some prompts to make a bit of a demo on Twitter. And now they're an AI engineer? Or is it somebody that was deep, deep in the ML platform weeds? And because AI is now the new rage, they call themselves an AI engineer. So I don't know about that, but it's at the top.

12:44I think it's the same. Yeah. I think people use AI, ML, and before it really fell out of vogue, deep learning interchangeably. Yeah, exactly. I don't know if it's also maybe connected to the fact, like Chris and I talked about this, I believe it was maybe last week, the fact that some of the disillusionment around AI is sort of the realization that turns out AI is integrated in software. and you still have to do engineering to build software. And it doesn't just sort of like having a model is a solution doesn't really play out in reality. You mean I can't just buy an AI model and stick it out there and magic things happen?

13:28Yeah, I mean one would think... I'm so disillusioned. Yeah. It's funny you guys mention that too because I've seen a few people talking about how LLMs are not a product. You have to build on top of LLMs your product or whatever it is, your service that needs to be there. So you can't look at an LLM as a product per se. And then I've also seen or I've been thinking deeply about something that is like the companies that are really getting a ton of value out of this AI movement. I'm thinking about one of my friends companies who does like a support software and now he's leveraging AI and LLMs for creating like multi-agents and helping answer feedback or answer questions and queries for support and he's using AI that's awesome he's able to sell that support product to companies really well.

14:30What I haven't seen is companies that say, hey, I am fraud detection as a service, and I'm going to sell you this, whatever, traditional ML product as a service. Whereas you can create regular business unit products as a service that leverage AI, but you can't quite, or at least I haven't seen anybody crack the nut, create some kind of a traditional ML service type of product. I don't know if you guys have seen that. And I also don't know if I'm making much sense right now because it's something that's relatively fresh in my mind. I'm going to turn that one over to Daniel. So no, I wasn't making much sense, I guess is what the nice way of saying it is.

15:18I mean, so you've got like, what I would say is the things that I have seen most are either what you were talking about. So utilizing generative AI embedded in the functionality of sort of domain specific applications like the customer service you're talking about or financial services or whatever, or access to models over some API infrastructure. right there's maybe less like general i guess maybe the biggest one i've seen is sort of just general like fine tuning as a service if you look at something like you know open pipe or something like that but that's still fairly general purpose it's not specific to any sort of use case that you might use maybe to some degree you know certain rag services would fit into that like we were talking to Pinecone about their recent, like they have more kind of prebuilt things to have you do kind of like load in all your documents and have RAG set up and all that stuff.

16:25So I don't know, that's maybe the closest that I've seen to that sort of scenario. Yeah. Well, also the big question is everybody wants to, and this kind of ties back into the hype cycle. Everybody wants to be doing rag and wants to have all these great use cases with their rag. And so like you were talking about with Pinecone, they make it really easy for you to do your rag. But then at the end of the day, is that a viable business or is that actually super useful as opposed to somebody's got this support software that they can come in and really cut down the burden for your customer success engineers or your customer success people.

17:12And that is fascinating to me because it's a booming business right now. The rag business, maybe, yeah, that's great. Maybe there's some interest there. Is it a booming business? I don't know. I haven't seen numbers. But I think the really fascinating part to me is if you try to juxtapose that with a fraud detection as a service type of product. I just haven't seen that anywhere because I think a, you're not able to really like give away everything as freely and be what works for one fraud detection use case doesn't necessarily, it's not like you can productize that and then go out and sell it as a service in my opinion.

17:59So, so this is a little bit of a tangent I know, But all that to say is we're at peak hype for AI engineers. Peak hype, yes. So I'm going to draw us back over to the hype cycle just for a moment. And I'm going to do something boring for a moment. I'm going to read off the things where they are for our listeners. Because the three of us have the benefit, obviously, of seeing the graph in front of us and for listeners who aren't. So I'm going to take a moment and then we can go back and start hitting them there. Very quickly, heading up the curve initially, the innovation trigger, we have autonomic systems, we have quantum AI, we have first principles AI, we have embodied AI, multi-agent systems, AI simulation, causal AI, AI-ready data, decision intelligence, neurosymbolic AI, composite AI, artificial general intelligence, otherwise known as AGI.

18:56and then we're hitting the peak of inflated expectations. At the top of that hype cycle, we have sovereign AI, AI trism, prompt engineering, responsible AI, and at the very peak, AI engineering. And then starting to slide down, we have edge AI, foundation models, synthetic data, model ops, and generative AI, and just going into the trough of disillusionment is neuromorphic computing, smart robots followed at the bottom by cloud AI services, and then we slide up the slope of enlightenment to autonomous vehicles, knowledge graphs, intelligent applications, and finally the singular one on the plateau of productivity, which is where you want to end up, is computer vision, which is basically, yeah, we can do that.

19:42It's boring and no one talks about it anymore, but hey, we're making money. So if the listeners out there are not confused. Oh, there's a whole bunch. I don't have any idea what they are. I was going to say, which ones do you actually know what they are? What the hell is embodied AI? Oh, I learned what that is after I put out the post. So someone said, oh yeah, embodied AI is when you use AI in robots. It is? So yeah. But there's also smart robots on the cycle. And I used it at a former employer. I was specifically doing AI systems in robots and I've never heard of it. You never called it embodied AI?

20:23Well, it's been a few years. I'll give you that. It was so. But no, we weren't calling it embodied. I mean, so I think I'm at like a 30 % hit rate on these. And I really would love to know what first principles AI is because that feels like buzzword bingo to the fullest. I don't know. Let's see. First. Yeah. Daniel's going. He's cheating. He's going to models to find out. the AI generated card in my Google search says when applied to AI first principles AI suggests developing AI systems and algorithms by understanding the foundational principles of machine learning neural networks and data science from the ground up don't we do that anyway when we're isn't that kind of inherent in training new models and stuff oh but no no we're really going back we're going back to the very first ones you're at the second or third principle or beating you yeah no because all you guys that are out there that aren't using first principles you know that's lower down on the hype cycle okay oh this is yeah so the other pieces i mean were there any other surprises for you guys because i have so many other pieces on here that i'm like what i think for me like some of these things are themselves correlated and yet in different places on the chart, right?

21:45So it's like, if you look at generative AI foundation models, edge AI, AI engineering, prompt engineering, probably some others on there, all of those like sort of fit into the same-ish bucket and yet are on different sides of the hump. So yeah, I don't know, like some of these, it's also a matter of where do you draw the boundaries? Where's the boundary between generative AI and foundation models or generative AI and prompt engineering? I'll give you one, you know, as we're at the very bottom on the innovation trigger is quantum AI. And I've, okay, so that's not going to happen anytime soon. And I will note that they have it on the greater than 10 years, but I would suggest it's probably greater than greater than 10 years.

22:36But isn't that, I mean, one of the things that's interesting about this whole cycle is there's that one, maybe you all can tell me or I can look it up. There's a one law. It's like a general law that people talk about where you underestimate short term innovation and overestimate long term innovation or something like that. Yeah, yeah. Sorry. I said that backwards. Yeah. So it seems like some like it's hard to, especially the time angle of this. It's hard to because things just pop up and you like really didn't see certain things coming and others that you thought would would come don't. So, yeah, it's extremely difficult.

23:19100 percent one thing that i am just to tag on what you're talking about daniel with the bucketing these please tell me what the difference is between an ai engineer and a prompt engineer what like a prompt engineer is someone that only does prompts i guess and that's all that matters so they're just so i can see how how it's like where's the line here when prompt engineering came out, Daniel, you might remember, I kind of made fun of that. I was like the whole that you talk about like, cause people were saying their new jobs are for prompt engineers and stuff. And I'm like, that is a passing fad.

23:58Like that will be just so ingrained in what everybody does all the time. That the notion of there being someone who that's their entire job all the time for years is not going to happen. Yeah. I also, um, didn't know. So like, I've never heard anyone use the word or if it's a word it's an acronym AI TRISM. Do people go around saying that? Yeah what is that? What is it? So it's I looked it up and you know what's what's funny because this is exactly the area that I'm working in every day. It's AI TRISM is tackling trust risk and security in AI models. Okay. You've never heard that used have you?

24:41And I've never heard that, but now I feel like I should put it on our website because it's hyped. Yeah, you definitely need there. That's right. The funny part is it's almost as hyped as prompt engineering, which you is basically all you hear about is prompt engineering, right? Yeah, they're right there together. AI Trism, you never hear about. Yeah, there you go. But the Trism, it's out there. it is we hear about you know the the components that make that up all the time sure but just never the and i've never heard them put together that way and i'm sure there are people that are out there that that you know their focus is in the that area and they're like of course it's trism honey but yet guess what most of us don't know that no not at all i don't even know if i go and I just look at this, I don't know what causal AI is.

25:33I don't know what the AI simulation is. The multi-agent, I do understand. But then even when you say quantum AI, I don't know what that is. The one that I would say is probably in the wrong spot is synthetic data. It feels like that should be still going up on the hype train because we're just discovering what we can do with synthetic data and every week i feel like we unlock new use cases and synthetic data is just uh it's the gift that keeps on giving in my eyes i think that's the difference in you who actually does it and somebody at Gardner who was tasked to go put the chart together and doesn't actually do the thing in real life.

26:28I've terribly offended somebody out there. Well, we're glad that it's out there. Let's just say that we are very happy that this exists so we can have a whole episode dedicated to breaking it down. Yes. It's a conversation starter. That's what I mean. Achievement made. Unlocked. Yeah. Unlock. So one thing that I noticed isn't there at all, which really surprises me, given how much it's bantered about, is ethical AI. It's not on the chart. And that doesn't go in the trism? Maybe it does. Maybe this is where I, you know, is ethical AI now transformed from a labeling standpoint into trism? Is that where we're going?

27:08I don't know. Or what is the overlap between responsible AI, trism, and ethical AI? okay well yeah and there isn't really anything on here about gpus or hardware so yeah i think that's because they made their own hype cycle for gpus that's right if i'm not mistaken i i feel like i've seen that somewhere on the internet you'd be cannibalizing your other chart exactly so you can't put any gpu hardware anything on the ai one you gotta refer people to the gpu hype cycle and maybe it's like that with ethical ai like they made a whole other ethical ai chart that is the hype cycle for ethical ai maybe so i'm not familiar with it how many charts can you make that's if you're gardener i guess they have i mean we have just the artificial intelligence hype cycle here but they probably have i think i've seen multiple you know subdivisions and stuff out there so that's why it's a great business to be in gardener selling all these different hype cycles well speaking of what to hype what uh what's not on the hype cycle but should be all right if i could have talked to somebody at gardener before they were making this i would have advised and so this is my basically this is my video job interview right now i'm busy typing an invoice up for you to send to them okay just exactly i would have advised ai gateway that is very popular that's climbing the hype cycle right now because people really like to have the option to hit an ai gateway and if it is not that complex of a query you don't need to hit gpt4 you don't need the most expensive model if you have some kind of open source model that is cheap then let the simple query go to that 7b model and so i've been hearing people call it an ai gateway others i think have called it like a LLM proxy router maybe or router yeah that's another one so we would have to agree on the actual name but that's yeah gaining hype for sure yeah agreed yeah it's uh I've definitely seen the router language whatever it is like the languages overlap with networking um which is basically like you're just routing API calls so I guess that makes sense yeah any any that you guys would have liked to have seen on here and where i had the ethical i'm still wondering what composite ai is did we ever get that answered or if i just am i having a senior moment or what is it yeah what is it um what uh the one that really stands out to me unless i'm just like there's a lot of words on this page so maybe i'm totally missing it somewhere but where is multimodal AI.

30:11Oh, good catch there. It's not on here, is it? No. Who cares about multimodal? That's so weird. That should be in the peak of inflated expectations. This is like the thing of 2024, like multimodal AI. That's so fun. Even multimodal rag should be on here, like climbing the innovation trigger. Multimodal models should be on the peak of inflated expectations. that that is such a good catch i know tons of people who who say multimodal and have no idea what it means well what what does it mean chris quiz time well it's having different mobilities of of input there so that you can combine different inputs to get a a rich output you know in a very general sense i have no idea yeah so voice i know when i see photos yeah video photos video yeah all the things all the things yeah exactly which is what we want i want to throw a bunch of stuff that i have and and have a fantastic just have it sorted out and give me the best answer uh and even with today's multimodal models that doesn't happen very well there's i i'm i'm often i'm often frustrated and disappointed with uh with those outputs so yeah it's i i'm expecting better yeah and along those lines i have two that i would like to have seen one is just transformers in general where's that where are they on this hype cycle because that also feels like are they climbing or are they going down i don't know it would be trough of disillusionment heading downward because that it's kind of we're we're past that and people are now talking about post transformer models you know quite often so it's kind of like yeah yesterday so there needs to be another dot for post transformer models that's definitely going up and that's right speaking of which it feels like okay we've got small language models where are they because that is all the rage it is that was like and maybe it's all the rage for every vendor who is not open AI because they can't compete on GPT-4.

32:34And so what do they do? They say, well, you can just host your own small language model and fine tune it and get better performance than GPT-4. And so I think small language models are probably, they should be in that innovation trigger, maybe the peak of inflated expectations because anyone who's ever used a 7B model might not want to use it if they have the choice. Well, maybe it's, are you sure that's going up or could it possibly be sliding into that disillusionment that you just reverted to? Potentially, that's true. Because maybe it is going into the trough of disillusionment, just hypothetically, because I do think that when it gets to the plateau of productivity, small models will be just the workhorse.

33:24You'll have them out on the edge everywhere, Every freaking device you've ever imagined or seen is going to have small models in it that are inferencing. We won't ever have anything that doesn't have them. It'll be just the O-Yon. Of course, we have our small models in our watch. Which leads me to the next one that I'm like, where is this? Why do they not have wearable AI? That is a perfect buzzword that should be on here. and if you look at like what meta is doing with the glasses or if you see any of those necklaces that you can wear and it records everything yeah that's wearable ai right there i just i may have just made that up or i may have seen that before but that one should be on here it should be there i agree

34:21hey friends outshift cisco's incubation engine merges innovation with the art of possible a launchpad for transformative emerging tech outshift blends startup agility with corporate strength to develop next-gen technologies from the ground up in AI, quantum technologies, cloud native, and more. Their newest AI innovation, Motific, addresses a critical challenge in the rapidly advancing world of Gen AI, bridging the gap between concept and deployment. This model and vendor-agnostic solution supports the entire Gen AI journey. From assessment and experimentation, Motific accelerates deployment from months to days, while safeguarding against Gen AI security, trust, compliance, and cost risks, all while empowering business function and IT teams to rapidly configure end-user assistance powered by organizational data.

35:17Motific provides advanced, customizable policy controls to prevent unauthorized access to sensitive data and helps ensure compliance throughout the entire process. With deep visibility into operational and business metrics, Motific enables you to track ROI, optimize costs, and make informed decisions. By offering a centralized view, Motific deters shadow AI usage and empowers teams to innovate responsibly. So move beyond the traditional constraints of AI implementation, utilizing AI deployment that is both responsible and is revolutionary, ensuring your projects are not just quickly launched, but built on a foundation of trust and efficiency.

36:02Visit Motific.ai. That is M-O-T-I-F-I-C dot A-I.