Humans Using AI Outperform Humans Without AI by a LOT (Case Study)

0:00Did you know, Neil, that humans using AI outperform humans using without AI by a lot? I didn't know that because AI can help you automate quite a bit. I do think there is one issue though, which is companies using AI without humans. And sometimes it works, but in majority of the cases, we're seeing a lot of errors and issues and then humans have to go in and fix it. But I think when you combine both of them, you can get more done in less time in high quality. So, yeah. And the reason we're talking about this right now is because I found case study and I actually read it this morning. And so basically here's the story.

0:37So this guy, Ethan, he put a team of social scientists together working with BCGs or Boston Consulting Group, which is one of the elite kind of consulting firms out there. And basically they turned the offices into the largest pre-registered experiment on the future of professional work in the AI age, right? And there's a working paper that's out. I'm not going to read all 58 pages to you. I'm going to read segments of this post over here with some highlights. And those of you that can see us on YouTube, if you're still not subscribed to us on YouTube, you can see the images as well. So you can now see my screen here and where my cursor is, you can see that the headline here is consultants using chat GPT for outperformed those who did not buy a lot on every dimension, every way we measured performance.

1:18Okay. And you see a graph over here where it shows, you know, kind of the before is like this graph and it's hard to explain, but it's basically, let's just say on a scale of one to 10, right? In terms of quality, the before, so the people that don't use AI, maybe their quality is a four. And then the people that are using AI, the quality is more like a 6.5. So that's a significant increase in terms of quality. And that's how I'll kind of easily explain this, right? And then over here, Neil, you can read this line over here. Can you see this? Kind of. Consultants using AI finished 12.2 % more tasks on average, completed tasks 25.1 % more quickly and produce 40 % higher quality results than those without.

2:01Those are some very big impacts. Now let's add in the nuance. I can't read the last part. There's a lot here. I mean, I'm not going to read all this stuff to you, but basically, you know, at the end of the day, there's the fact that this study was done with BCG. And this is like, ChatGPT hasn't been out for a year yet. And we're already seeing a 40 % an increase in quality. Or in some cases, when you look at the improvement, that's a 62 % improvement on the four to 6.5 or so. That's just the very beginning of all this stuff. And so do we think this stuff is overhyped? I think, Neil, you made a really good quote when we spoke on stage at HubSpot.

2:37Do you want to repeat what you said in terms of the long-term and the short-term? Yeah. So the way I look at AI is people overestimate what they can get done with AI, and they're overestimating it in the short run. In the long run, they're underestimating it. See, in the short run right now, everyone believes that, oh, I can use AI and just get it going in my marketing or my business. I'm just going to change everything and we're going to grow faster and make more money and save a lot of costs. But when they try doing it, one, AI technology isn't there where it can just automate everything with a snap of a finger.

3:07It requires a lot of human intervention. And two, it's inaccurate in many cases. You know, I was reading something about Chad GPT and it was saying roughly 20 % of the time it's inaccurate. it. Now, I don't know if it's 20 % off now or a little bit less or a little bit more, but either way, it's not perfect. And the way you can get it to work really well in the short run is you integrate humans throughout the process when you're leveraging AI. In the long run though, people are underestimating what it can do for them. The capabilities are endless. Just imagine having a robot like from the movie iRobot that's walking around with you and is helping you with your task or knows your life and knows what kind of music you like and just helps you throughout your life.

3:50In the long run, I think AI is going to really change the world in a much bigger way than most people think. But in the short run, it's not going to have as big of an impact on people's business and marketing as most entrepreneurs or marketers think. You know, I think this is a Bill Gates quote, actually, where he says that we overestimate what we can do in a year and we underestimate what we can do in a decade, right? And it's the same idea here. And so this is a Harvard paper. So if you want to read the post itself, just search for again, centaurs and cyborgs on the jagged frontier. That's not an easy title.

4:22I'm going to repeat it again. Centaurs. So the half human, half horse, right? So centaurs and cyborgs on the jagged frontier. Okay. And then you'll be able to click into the Harvard study. I just want to read this piece off as well. So they did this study with 758 consultants, which comprises about 7 % of the individual contributor level consultants at BCG. So I guess they have what, almost 11 ,000 consultants. That's a lot of people. Okay. So consultants across the skills distribution benefited significantly from having AI augmentation. That's a keyword right there. With those below the average performance threshold increasing by 43 % and those above increasing by 17 % compared to their own scores.

5:02So we've talked about this on the podcast before. There's three forms of AI right now. You have actual artificial intelligence, which most stuff isn't. You have augmented intelligence, and then you have automated intelligence, right? This would be a sense of a lot of stuff we're doing is augmented or automation. And I do want to give credit where it's due. Again, this comes from Saul Rashidi, who's a chief data officer formerly at L 'Oreal. And Neil and I, we actually did a session, which we'll release in the next couple of weeks, where it's about 30 minutes or so. We talked about a lot of practical use cases for AI.

5:30But for this episode, just so we don't leave you hanging, we'll just give you one practical use case each. Neil, what's one practical use case right now that is maybe not as common. Yeah. So use AI to analyze your analytics data and figure out where you're wasting money and where you're not. Don't have it make changes to your ad campaigns instantly, but more so just flag it to someone internally on your team and they can review it each day. And that's great because it'll just help you save a little bit more money on paid ads and then reallocate that to places that are performing better. Simple.

6:00A lot of businesses aren't doing that. But if you look at how much money is being spent on ad spend roughly 160 billion a year just on Google alone. Facebook's over 100 billion a year. There's more savings to doing that than it is to saying, hey, let's have AI help me write a blog post. Whoop-dee-doo. That would have cost you 15, 20 bucks anyways. And on my end, I've been on this programmatic SEO binge right now. So it's actually starting to take effect where a handful of our, I think we're trying three or four different initiatives with programmatic SEO, where we basically will do the topical mapping, the keyword research, and then we'll design templates for those pages and we'll just create these templates at scale.

6:35And then we'll use chat GPT for, and then we'll use a human in the loop just to kind of spot check to make sure that things look good. And then we're able to ramp things up there. Now, am I saying that works for everyone? Not necessarily. You're going to need a site with good domain authority and the type of business, like a, let's say you're a trip advisor where best places to eat or best hotels, like there's tons of permutations you can make. So it doesn't work for every single business and there are nuances to it, but it does appear to be working. So that is it for today. Please don't forget to rate, resubscribe.

7:03It helps us grow more than you know, and we'll see you tomorrow.