How Businesses Are Using AI Right Now with Greg Kamradt

0:00Today on the AI Breakdown, we're talking to Greg Kamrad about how businesses are actually using AI today. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord, and our newsletter.

0:24Hello, friends. Welcome back to the AI Breakdown. As you know, this week I am traveling for my wife's and my mother-in-law's birthday, and so I am doing the rare interview shows, and I'm really excited about this conversation. Hello, friends. Quick note before we get to the rest of the episode. You have probably heard me talk about the AI education beta over the past few months. We've had a ton of you participate, which has been amazing. And now we're almost ready to announce something big and something new. If you want to be one of the first to hear about our new approach to learning AI that is hyper practical, hands-on, immediately relevant, continuously upgrading, and anchored by community, go to besuper.ai and sign up to be notified when the project goes live.

1:07We're getting there in just a few weeks, and I want all of you along for the journey. Once again, that's besuper.ai. I first came across Greg because of his YouTube tutorials on AI tools, and I've been following him ever since. And recently, he's been interested in some very similar questions to those I'm interested in around how real people and real businesses are using AI in practice today. Greg recently went really deep on this question, interviewing a ton of companies about it and putting together a report called Conversational AI for Work. We talk about what he found and what it suggests about the state of AI.

1:39And I think this is a pretty practical conversation for those of you trying to figure out how to bring AI into your work. Without any further ado, let's dive in. All right, Greg, welcome to the AI Breakdown. How are you doing, sir? Yeah, we're doing all right. It's a beautiful day out. Happy Tuesday and excited to be here. Yeah, awesome. So, you know, you and I talked a bunch. Before I knew you, I was following along with tutorials and content you were making on YouTube. And what I want to talk about today is sort of an exploration that I feel like you've been undergoing for a while now, which is how people are actually using AI, right?

2:12How people are integrating into their companies, what tools they're actually using. You've done a lot of not just speculating on that, but going out and finding it. And I think there's a lot to be learned there. But I think first, just for people who aren't familiar with maybe your content, give us a little bit of background and sort of what you spend your time on. Yeah, absolutely. So again, my name is Greg Cameron. I started doing a YouTube channel a little while ago. And during COVID, I started doing technical tutorials for other data analysts. So I was teaching people how to use the Pandas library, which kind of seems small and niche with it.

2:41But that's where I got my first foray into content. So I'd make pandas notebooks and blog posts and things like that in a video. Those are definitely some started from the bottom videos. It does not look good. Anyway, long story short, fast forward to a fast forward to the AI realm. I was doing more tutorials on teaching developers how to build their first AI applications. But my background is in B2B operations and product and kind of finding product market fit and user research and all that. So what always fascinated me was how are we going to turn the cool parts of LLMs into actual customer value because that does, that's interesting to me for two reasons.

3:16One, customers get value and that's just an awesome thing. And I think it's so cool because it's so hard to do, right? You can't just flip a switch and go and do that. But the number two is because that's what businesses are run off of. And that's how you create a huge business as you go and create customer value. So really when chat GPT came out, what first caught my attention was I noticed a bunch of people on Twitter sharing how they use chat GPT for completely random purposes. So they'd say things like, oh, I use it to create bedtime stories for my child. I would use it for financial analysis.

3:47I'd use it for creating recipes or something like that. And my hypothesis was that, hey, using chat GPT for any one of these workflows is likely not the best way to do it. So there's likely some cool products that you could build using these as early signals to go after it, right? Anyway, long story short, keep that going. And I found myself asking more and more businesses, what do you actually use AI for? Because there's a lot of papers, there's a lot of stats, but not a lot of tangible use cases that come from that. So I do a lot of talking with businesses on their interviews, seeing how they actually do it internally.

4:20I just got done with this big report called the chat with your business data report, which is all about the tools that are chatting with your business data and what use cases employees are doing. So it's been a fun journey. And I'm happy to go into a few of those if it's interesting. Yeah, absolutely. You know, I think validating that sort of initial thesis, each of the things that you just described is some of those early use cases where people are experimenting. You know, I spent a lot of time watching new AI startups and things like that. And each of those that you just mentioned has like five or six companies that have now tried to, you know, productize that particular thing, right?

4:51The kids' bedtime story, just everything with kids is a huge area. And, you know, who knows which, if any of those things will work, but it's clearly enough people think that there's enough there that they're really interested in exploring. So let's talk about what you found when you started to dig in with companies. And maybe to try to take a step back from it, when you started asking around, when you started asking on Twitter and in other places, if you're a company, what are you actually doing? How are you increasing your productivity with AI or how are you using it? What did you expect the answers to be or did you have expectations?

5:24And how different or how quickly were those expectations either confirmed or denied? Yeah, you know, it's funny you ask that because I only came to this realization more quickly or recently I thought I would ask enough people and I would get a magical use case that would blow me away And it would blow my audience away and everyone would just say this is so cool We're all going to go rush and go do this But after talking with hundreds of businesses and hundreds of use cases I'm starting to come to the realization that there isn't a blow me away use case Is just going to revolutionize everything and that the real power of these language models and these AR workflows is a simple language model call to the right user at the right time in the right location.

6:05And so it's nothing revolutionary, but it's just about integrating these things into your workflows where you get one to 2 % improvements multiple times a day in multiple places of your workflows is what I've seen the best. And so my expectations, I thought there'd be something that was going to save me, you know, weeks and months of my time. But in reality, I find it's a lot of 1%, 2 % increases use cases here and there. So my first jump into this was when I asked businesses, like, just tell me, I put out a type form. I said, tell me what you're using this stuff for. And that's a really interesting use case as I did a whole video on this.

6:36And number one, super easy was around a support bot. So these are all companies who are building their own integrations as opposed to using a third party tool. And so basically they would get an inbound support request. They'd look at previous requests that they've already answered. They'd use that for retrieval. And then they generate a new response to the inbound support request based off of how they've answered things in the past. Super easy, really straightforward. forward. Other ones were around survey correlations. So if you have thousands of survey responses about your product, about pros, cons, bugs, features, product requests, all this other stuff, that'd take a long time for somebody to go through and go check those out.

7:09So getting an LLM to be able to parse those for you, I thought was really, really interesting. Another huge one, which is around AI generated templates for your product. So if you run a product that has a template like action, so say for example, Zapier, you know, they really want their users to have Zapier, Zapp's already set up for them. They don't want them to have to create them from scratch. Being able to generate those with AI based off of who the user is and what they're trying to do in the role. Really, really cool. And then the last one I'll mention, which I thought was so, so cool was around a company that built a chat bot that used Slack as their knowledge base.

7:45So the reason why I thought this was so cool is because if you think about it, a lot of Slack data is unstructured tribal knowledge, and it's been locked behind keyword search for a long time. If you wanted to go find something. You had to go know the exact conversation or keyword. However, language models, one of their expertise is being able to structure unstructured data. So this group, all of a sudden, they had three or four years of tribal knowledge sitting in their Slack. All it needed to, it just needed to get uncovered. So they actually had an intern that built a bot that looked into their Slack knowledge base and all of a sudden they were able to answer questions for them.

8:18And one of the reasons why this is so cool, and this is where I'll wrap it up, is that all of a sudden junior your employees were more confident to ask questions and they didn't have to go bother leadership anymore. They could just go ask an AI and then they get the answer that's there for them. So you free up leadership time. You've gained more confidence for your junior employees. Anyway, so that one blew me away as well. Yeah, I think it's interesting. So something that you said that I want to hang on for a moment is this idea that you kind of came in expecting there to be perhaps like some killer app, right?

8:47This is the way that we used to talk about it in the internet days. What's the killer app of this platform, blah, blah, blah, blah, blah. And instead, you found these things that are sort of just like obvious improvements, but in these smaller, subtle ways that you stack them on top of each other and all of a sudden, you know, making real change. I think that one of the things that people are struggling to sort through right now in their heads is how on the one hand, AI could be sort of not have those singular sort of like massive use cases that everyone can point to all of a sudden, but at the same time still feel every bit as big and inevitable as it did when you first tried ChatGPT.

9:24And the way that I see this manifest doing a daily podcast is you constantly see journalists or media or whatever, people on Twitter, trying to push AI, call it into the trough of disillusionment after the peak of inflated expectations. And it just doesn't resonate. It just feels like media narrative making. And I think that part of it is sort of what you identified, that on the one hand, the individual changes that any one application or program might be bringing or any particular context might be small, but they are, they're 100 % changes. They're completely, you're never going back to the way that you did it before, right?

10:01So if you are a YouTube creator who didn't use Midjourney or Dolly 3 or whatever on your thumbnails before, there's not a universe in which you're going back to not using those image generators, even if it's a small thing, right? It's not like saving you hours every day. It's potentially saving you minutes every day, but There's just, there's no universe in which you're going back to the pre-AI process. And I think that a lot of these things are like that, where the change is absolutely inevitable. Like you're, you're not going back, but it's still taken alone is small, right? It's, it's, it's discreet.

10:32And that's a hard thing to sort of reconcile, like the discreteness and specificity and smallness of a lot of these individual changes. But the fact that they are total and complete and, and, you know, and totally, uh, you know, there's no universe in which you'd go back to a previous system. Yeah, totally. Well, I like to think about, can I pull anything out from previous technologies that as a framework that we could use to evaluate the AI side? And so the one that I like to quote or reference is, you know, when cell phones came around, okay, cool, we have mobile data. Okay, I mean, it's not super revolutionary.

11:02I can still go look at my blog posts on my mobile phone. Okay, cool. Well, we also have GPS, so mobile GPS. Interesting. So yeah, I can see where I am on a map. That's kind of interesting. It doesn't change my life. It's not too revolutionary. It is cool though. Now those are the primitives of what happened in the mobile world. And those primitives on their own are not that exciting. But then all of a sudden you combine those two and you get an Uber that comes out the other end. Right. And so then I think back to the AI side, it's like, okay, well, what are the primitives of AI that don't look too revolutionary in and of themselves?

11:34But then when you combine them, you get something cool out of it. So the huge ones that I, that I know are just unbelievable is obviously the reasoning power. So ability to make decisions. Okay, cool. So you have this thing that can make a decision between the two. And then also data transformations. So you can take any sort of input and really get any sort of output and like kind of a format between the same there. Now, what's the Uber that's going to come out of that? Well, Uber also, you know, needed a few years after those two things are combined to be able to surface themselves, but you're seeing things like Devin and I'm not saying Devin is going to be, I'm not commenting on the performance of Devin, but those are the types of things I would expect to start to trickle out that combine these primitives in really unique ways that start to do some really, really cool things.

12:15So I also think it's a matter of time. And, you know, just the way markets evolve, we're going to need to let it mature just a little bit more. Yeah, I think that that's absolutely right. And I think the primitive thinking is actually really useful because you do start to notice patterns, right? So you identify data transformation of sort of a subset of this is call it content into content, right? As an individual content creator, I'm sure this is one that you think about, right? Like the ability to take whatever your base content is, be it an essay, be it a blog post, be it a video, and turn it into every other type of content, you know, radically faster, huge transformation that maybe at this point, only independent content creators are using because we're the only set that has such huge constraints on our time versus sort of what we need that content to do.

12:55But I think that you see that, you know, that's going to find its way into, you know, sort of everywhere that content is produced, whether it's internal or external or whatever. I think another primitive that you kind of mentioned before when you were giving some of these examples is just using LLMs to unlock big masses of data that were previously locked up, right? So knowledge that gets actually usable or repeat usable rather than sort of just, you know, unknown. That obviously becomes a huge opportunity. And I think to your point, the first set of businesses that are being built on top of that are really just how do we build the tooling to unlock it?

13:32It's less like, what are we actually going to do with that? How are we going to point people to do things with that? We're still sort of at a stage where we're relying on clever businesses to figure out how to use that data. But that's not going to be for long because what's going to happen is basically they're going to have, you know, more and more people are going to read more and more reports like the one you just did. It's going to spark a million ideas that they have, which is going to then prompt them to go do those same things that they saw other people doing in your report, but then a hundred other things.

13:57But then you're going to write version two of that report, which now has a hundred use cases instead of 20 or whatever it is. And all of a sudden, that knowledge spreads very quickly. Are there other patterns that you've observed, I guess, when you were starting to have these conversations, things that would fit in this category of primitives? Sure. The category of primitives. So we talked about reasoning. So yeah, I kind of have three big buckets is the way I think about it. So number one is reasoning or decision-making. So you have this thing, the quote that I like to say is, if I tell an LLM, I'm thirsty, and to my left is water, and to my right is the desert, which route should I go?

14:29That's a simple decision. but the fact that it can make it for us is pretty cool. Number two is going to be around a primitive of the semantic search. So sure that that word is almost trite at this point, but the fact that you can search for other things based off of intent and meeting is a huge unlock over keyword that unlocks just a whole different set of workflows, which is very cool. And then the last one that I said, like before is the transformation piece. So ability to turn one piece of data into another, whether it's a long thing into a summary, whether it's a piece of JSON into a SQL query, whether it's HTML into extraction, extract some cool things that come from it.

15:05So that's another huge unlock that comes from that as well. Yeah, it's fascinating. You know, there's a lot of things that are there's, you know, you start to walk down this path. And I think you can, the mind starts worrying with with with all the possibilities. So I guess when you were talking to these companies, how frequently were you finding that they had a thing, a process, a workflow that they had figured out, but they hadn't necessarily come to the same conclusions that all the other companies were working on versus there was sort of, you know, one maybe leading thing, but then they were experimenting with this stuff, you know, everywhere else.

15:37I guess like, it's like, how much were you seeing, how much were you observing convergence towards there being this sort of common set of use cases versus each of these sort of independent companies is really just playing around in one of these sandboxes at the moment? Yeah. What I observed is that companies believe that they're a special snowflake. And so when you're internally, you think that our use cases are super unique, our data is super unique, et cetera, et cetera. Now, if you take a look at a thousand companies, that may not actually be the case, but the attitude nonetheless is the same when you're inside an individual company.

16:08So most of the use cases that I saw and the cool parts were individual devs or operations people, somebody really good at maybe stitching together Zapier workflows or something, and able to have the domain expertise of their company, know exactly what the right workflow would be that would almost like a scalpel be able to just cut right to the core of what they need and then go ahead and implement it. So I would say it was more internal exploration with regards to their own value props than it was sharing what's gonna work everywhere else, right? So yeah, that's what I've seen most often. What were blockers that you observed that you came across?

16:44Were these companies reporting lack of tooling, too hard to experiment? What was anything like that? Sure. Two big blockers. One is just the product work to integrate it where it needs to go. And so it's super easy to put something behind a Slack bot. Okay, cool. You can spin up a Zapier and go do that. But to get a new Salesforce button or to get something in your database or a new derived column or something like that, that just takes straight up product work to go and do, which was, it was just kind of annoying. So I would say it's nothing AI specific, but it's just the resources to be able to integrate that yourselves is a big pain in the butt.

17:21And then number two is there is a Delta and a gap between the tasks that people want to do and what AI can actually do. And let me explain that. So the majority of the world is non-technical, right? And so they just have a goal that they just want to do. That goal, however, is grand and it requires a lot of steps to complete, right? Whereas technical people, they're really good at breaking down their goal and what they want to do into individual parts and then processing those individual parts. That's really good for what a language model can do. However, you know, you have your other employees who are non-technical and they may say, please write me this marketing plan.

17:57And it's like, well, you got to do a lot of work to write a substantial marketing plan for what your business needs. Not only that, but it also needs to be context dependent for your business. And so that gap between the amount of tasks that you actually need a language model to do and what it can do right now, that's causing problems for people. And that's also why planning is such a big emphasis with regards to open AI and Qstar and all these other things. And so I would say ability to integrate a language models into your products. And then also the gap between tasks people want them to do and the planning required to complete them.

18:31Yeah. It's super interesting because these are such different categories. One is sort of the inertia of the existing world in terms of implementation. I think this is like, one thing that is pretty observable is that that sort of inertia feels like it's going to slow down AI adoption more than anything about AI in a lot of ways. It's just that these, you know, processes take time and, you know, organizations have their own challenges. Even just the data hygiene required to make some of this, you know, like LLMs are very good at this stuff, but it's still like making, you know, information useful for, you know, for different applications is not a trivial task in many cases.

19:05It's interesting though that on the other side, the sort of gap between what people want and what AI can do. I think we've observed this a lot in, I'm sure you remember, it was exactly a year ago at this time, March and April of last year, when the auto GPT excitement got to the peak, right? Baby AGI, auto GPT, all these things. It was like this explosive excitement because it reflected, I think exactly to your point, people went immediately to that version that was sort of able to actually do multiple tasks and figure out, you know, if I could give it an end state, how to, you know, how does it produce that end state?

19:43And I don't have to sort of walk it along, along the path. And it's, it's, it's notable that on the one hand that hasn't come to fruition in terms of actual practical application, uh, in the, in that year, but that it seems to be pretty much the biggest focus of every lab is, is sort of getting closer to that reality. Yeah. I mean, that's exactly the first comment, man. There is no time like being on Twitter in March of April of last year and just seeing the fervor. It was crazy. It was absolutely nuts. People could hardly contain themselves. Yeah, that's fun. I know we'll get back to that, of course, but we're not having been there for a little bit.

20:17But yeah, I mean, you're right. You know, on the recent interview with Lex and Jan, they were talking about, he likes to give the example. I'm sitting in my hotel room in New York and I want to get to Paris. Go do it, right? Humans are really good at hierarchical planning. and figuring that out. Okay. I need to go to the airport. Okay. I need to leave this hotel. Okay. I need to get an Uber. Okay. I need to, and then you can go all the way down. Okay. I need to get up from my bed. Okay. I need to put my muscles up to get up from the bed. And you can go all the way down to firing individual neurons for what you need to go do.

20:46So planning is a huge, huge piece of this. If you want to do any sort of a complex goal, right? It really is. But then the other huge piece of this is quantifying what that goal actually is, is quite difficult actually. And so being able to break it down. And so I'm doing a lot of reading of Francois Cholet around definition of intelligence. And just like how intelligence is really hard to measure or define, defining the subcategories of what is a skill and what is a goal and what is a task and being able to translate that into something that a computer can do is also quite hard. So I think there's a lot of really cool areas that we're looking forward to, but we're definitely not quite there yet.

21:22I also think kind of bringing back a point that you were making much earlier in terms of how there's this inevitable period after technological primitives become available where people are experimenting with what's going to be actually interesting use cases. And we almost always start with things that seem obvious, regardless of whether they're actually useful or not. So I have a, I have a, I don't know how contrarian it is or if people agree or not, but I have a very strong disinclination towards, I don't think that personal assistants, like everyone walking around with a personal assistant to schedule their travel is going to be an actual use case of AI in the long run.

21:58I just think that even if it got great at scheduling, I just don't think that that's a sufficiently valuable time saver relative to just looking for your own flights. That's the type of thing. People love, I mean, but it's where all the demos are. All the demos that these agents speak are, can it order me food faster? Can it order me flights? And I don't think that that's because that's the extent of the imagination of the agent designers. I think it's because they have to clear those hurdles on the path to making it more broadly useful. But it does create a situation where I think these things can underwhelm with sort of expectations because you're like, okay, so all of this energy and what we can do is just order faster, you know, or order Dordash faster or whatever.