How to Balance AI Magic with Human Expertise

0:00I was like, this, I found it. I found it. This is what I've been looking for. I can honestly say has genuinely changed the way I run my business. It's changed the results that I'm seeing. It's changed my engagement with clients. It's changed my engagement with the team. I couldn't be happier. Honestly, it's the best investment I ever made. What you just heard was a testimonial from a recent graduate of the Duct Tape Marketing Certification Intensive Program for fractional CMOs, marketing agencies, and consultants. Just like them, you can choose our system to move from vendor to trusted advisor, attract only ideal clients, and confidently present your strategies to build monthly recurring revenue.

0:39Visit dtm.world.scale to book your free advisory call and learn more. It's time to transform your approach. Book your call today, dtm.world.scale.

1:02Hello, and welcome to another episode of the Duct Tape Marketing Podcast. This is John Jantz. My guest today is Jeff Coyle. He is the co-founder and chief strategy officer for MarketMuse. He's a data-driven search engine marketing executive with 20-plus years of experience in the search industry. He's focused on helping content marketers, search engine marketers, agencies, and e-commerce managers build topical authority, improve content quality, and turn semantic research into actionable insights. So Jeff, welcome back to the show. Hey, thanks, John. It's good to be here. So when you proposed this topic of AI and authenticity, I jumped on it immediately because certainly that's one of the biggest complaints.

1:48I mean, you hear people out there saying, oh, well, never use AI because that's inauthentic. So why don't you just globally set the table for how you're approaching that topic? I think the easiest way to think about it is the AI that you may have access to isn't equal to all of AI as a concept. Right. And so the way I think about it is setting the table is just because the thing you have access to on your phone doesn't do it doesn't mean it can't be done. And doesn't mean that you're using that type of technology to, you know, to inject all of the things that would represent you and your authenticity or your brand's authenticity or your company's authenticity.

2:32So not to say it's user error, because there's a lot of problems with technology that exists and it is accelerating so fast. However, what I've heard, there are large waves of folks who are almost like discrediting an entire science based on their experience. And that is always something to take notice of. And I build content strategy, content plans, content briefs, content, state content with authentic examples, authentic items. And I think that the last piece of this I think worth talking about is content execution, content marketing. Building content doesn't only reference the generation of text or the draft.

3:23It's all the steps that lead up to it, and it's everything that happens after it. And those things can be built and, you know, customized, made to be authentic throughout that process, as long as you have, you know, an understanding of what it is and what it isn't. Okay, you said about 10 things there. I know. Let's back up a little. I tried to set a table. It's a buffet. So let's back up a little bit there. What are the differences between some of the more robust technologies, if we want to call that, and maybe the more pedestrian sort of let's start with chat GPT, you know, just because so many people know that.

4:01And I'm guessing when you said that thing you had on your phone, you were talking about that. It could be that. It could be Siri. It could be, I mean, Alexa. You know, the first time you had an Amazon Echo, you asked it a question. It probably didn't get it right. It probably got better over time. Yeah, I listen to obscure bands, apparently, because it can't ever get that right. I know, it never does. So what are the differences between the more robust technologies, if we're going to call it that, and then something that you referenced that you might have access to? Sure, sure, sure. So when you're thinking about, you know, a chat GPT or a Claude or, you know, Bing chat or the chat you have on a search engine, those are just your access account to query a, you know, you're hitting a large language model with prompt and you're getting a response.

4:54So the most basic prompt and response large language model, but there are complex AI to use for predicting, algorithmically predicting stock picks or algorithmically predicting churn risk. You can build models that will do things that are very relevant to your business that have nothing to do with large language models. Another example is, you know, you hear quite commonly, you'll hear that. So contrast that if I ask my phone, right, what stock should I pick tomorrow? And it tells me, hey, go pick Microsoft and Google. And I'm like, hey, that doesn't pick stocks right. Hey, I can't do that. It can't.

5:39It's just not this. Another example would be complex math problems, right? So you ask ChatGBT to do some sort of complex college math. They won't do it. However, there's an entire Google team, the Minerva team, who is working on college math level mathematics and doing a very fantastic job of it. So it is the simple access to an attuned for the masses large language model will get you the most predictive response to whatever, however it interpreted what you sent. So it's trying to, and then there's some error built in. So it's basically saying that the capital of North Dakota is, well, 99.99999 % of the time, the right answer to that question is Bismarck.

6:29It's going to follow up with that. So if it's giving more of a holistic predictive step through, it's trying to pick words, maybe create some, put some creativity in it. Even with ChatGPT's technology, the GPT-4 now that everybody's using, you can go in the back end and you can tweak what they call temperature, make it more or less creative. Right. But in your experience on your phone, you don't have that. You get one input, one output with not a lot of configuration. So you're getting a front end on a very complex technology that even its own, even the thing you're using can be configured in dozens and dozens of different ways to be more creative, less creative.

7:15You hear people make fun of its use of the word delves on everything, and its use of the word crucial. It's because there are particular settings in the backend that are yielding that outcome. You can tweak all of that. I mean, it can be as customized as you might want just from the base model. So they're giving an attribution to the user interface and the default settings of the software, not the science. And that's really differentiator. So you mentioned a couple of times the language model. I mean, essentially what you're saying is that's what it's been taught, right? That's what it's been fed.

7:51And so that's that model. A lot of people are finding that they can actually teach. like they can feed everything, you know, everything that John Jance has written in the last 15 years and build a model around that. Are you going to get closer to like, if I want to be in my authentic voice, is that going to give it a much closer model by feeding it or teaching it that? Oh, absolutely. It does. And it's already, I mean, that's one way that you can start to get. So what you can do is you can take something that's been trained or an ensemble of language models, and then also stack on your stuff, right?

8:30Everything that you want. You can do just your stuff, but that could have challenges, right? So you can work with collections of those items. You can even do complex comparisons. So there's a thing called an AI agent, right? So you might have heard they're doing custom GPTs with a chat GPT and that software. However, you've been able to build agents that do a lot of different things. So you can actually chain together various steps in a process to take and reference each of your works, right? Let's just say you found out that it would be more precise to reference them individually or to step through each one in a particular pattern versus just considering you as the training model for the LLM.

9:19Yeah. So that you You can do different types of approaches to solve specific goals and instruction sets. But I mean, it's not limited to that. You can get into a situation and say, only include a reference to this book by John. You know, that is that this is the source of truth. Everything you do must reference this book. And you should consider this to be the tome of all knowledge, right? So there can be specific instruction sets, and these are not tremendously complex. These are things that a business user can learn how to do, not just an AI developer or somebody in product management, engineering, and data science for 20 years like me or anything.

10:06So one of the things that I think a chat GPT was so, it's both plus and minus. The plus was it's so easy to use. You just go type something, right? The minus was you got a lot better result if you knew what you were doing, if you knew how to query deeper, if you needed how to ask for tone and all those kinds of things. I'll use MarketMuse as an example. What are some of the use cases that you are building that actual kind of steps or templates, if you will? I don't know if we're going to call them that. Talk a little bit about how tools like MarketMuse are actually taking the power of AI, but actually building it into use cases.

10:44Yeah, so some of the most exciting, I mean, I'm building right now, the engineering team and the data science team at Mark Muz is building, by far, its most amazing things we've ever built. I mean, we have the content briefs that we're building now are by far the best content briefs that exist in the market, which is exciting. They're not out yet. I'm pushing really hard, right? But what we're able to do is really special. And it's when you already have your own data and you already have your own innovations and insights, you can use artificial intelligence to work with those things, you know, very magically.

11:23There is a technology called, you might hear RAG, the RAG Retrieval Augmented Generation Vector Databases, which if you imagine a concept in 3D space and like these, you know, imagine like clusters of information in 3D space, you can get, you can work with those types of technologies. technologies what used to cost I mean millions of dollars worth of research and innovation for thousands of dollars tens of thousands of dollars to get like usable and usable things that you can work with and so what we're working with now is we've always had this beautiful content inventorying product where we can look at your entire site we can look at your competitor site and understand and prioritize we've always been able to build out a great you know basic topic model how to cover a topic comprehensively to sound like an expert, but we're able to use all of our pre-existing innovations in novel ways to get amazing solutions like, hey, what article should I write today?

12:22And have that actually come from a place of knowledge, not just what you would get if you wrote, hey, what should I write today to ChatDVT? Your website, your domain, I mean, that's real estate that you want to own. If you're an influencer, online creator, blogger, or really anyone who cares about their personal brand, then you need a unique domain. And now you can get your name dot bio, right? John Jantz dot bio, right? Create a bio page to house all your various interests. It's short, simple, easy to remember. Put all your links in one place instead of a laundry list of locations you want to send people in a profile.

13:04You can reserve your own dot bio link for around$3 right now at porkbun.com slash duct tape marketing 24. That's right, around$3 right now, porkbun.com slash duct tape marketing 24. So if you think about that, it starts you on the path of having a content strategist, artificial intelligence product, which we are in effect building. We also have done the same with thousands and thousands of examples of developmental editing, which is a process of giving feedback on pages after you've consumed them. And we can do that from a place of expertise versus if you just ask, hey, how should I improve this piece?

13:52You're going to get random stuff. But when you get it from a place of an understanding of what's going to work for organic search, what's going to work from editorial processes, pretty cool stuff happens. I was just working on a page example and I was trying to mess with my own system and gave it a page, gave it something to optimize the page for that it wasn't really about. And it coached me through how to actually make the page about something that it's not. So I was like, oh, wow, this is pretty wild and very much through the lens of expertise. And so those are the things we're working on is how to prioritize what to do, you know, analysis, clustering, quality analysis in a very unique way.

14:30what should I prioritize and then executing with amazing briefs. So what did I not say? Actually generating content. I, you know, I firmly believe that's the thing that these technologies do the worst. And if you can coach yourself almost there and then take it all over the finish line with your expertise built in magic, things happen. Very magic things. So you mentioned expertise at least four times in that answer. And it was right to my next question, you know, search engines, you know, are theoretically now, you know, looking for expertise, looking for actual experience. And when you're writing about a topic that you can demonstrate, no, we do this or I've done this.

15:11How, I mean, that's certainly one of the things that, you know, can be a potential shortcoming in AI. So how do you overcome that? Because again, you are clearly thinking about it because you've mentioned it numerous times. Well, I've been doing it for a very long time. I've been saying, you know, authority, you know, topical authority for, you know, now, sadly, more than a decade, trying to get people on the horse to be thinking about quality for a long time. But now it's true. Like what you said, certain queries, certain research paths require expertise. They should have always required expertise, right?

15:47But in reality, could it be assessed? And so what I'm being a push for is ensuring that you have a human in the loop at various stages. And you understand the types of things that if you were to inject them in the process would exhibit expertise. So it's things like brand style, tone and voice. It's things like actually the personal anecdote that needs to be in there. Yeah. It's things like when I'm, you know, I own a microbrewery. When I'm writing an article that is featuring these types of beers, make sure the one I make is in that listicle that I make, right? Because that's going to make it unique and illustrate expertise as well.

16:29If there's a technique that I want to feature, if there's an area of expertise, if there's a visit to a particular country, and I need to include that to represent that I've actually been there, those are things you can inject early in the process and keep the flow logical so it doesn't just seem tacked on in the end kind of like a dreaded you know post-published edit which doesn't you know and those are the ones that are getting killed right now is these post-published edit i call them postage stamp content they have 90 percent generated and then they'll put a personal anecdote at the bottom you don't even know whether it's real or not and you know i The price of not being authentic, people are feeling it for the first time right now.

17:12Yeah. And that's, you know, they didn't go there early enough. So a lot of their pages exhibit actually authenticity or actually inauthentic information. And that's where you get into a big. Yeah. Because before ChatGPT, let's face it, for five, eight years, you know, there were a whole bunch of content forums out there producing ChatGPT-like content, weren't they? I'll call it that. However, the sharps were winning. The sharps were winning. And those sharps, many of them, the really sharp folks, are still winning. But yes, there was a lot of content that was, I'll call it powder room makeup on the pig.

17:52And you're seeing that for many reasons, those sites are tremendously viable right now. Where you had great infrastructure built, those still are, whether they were using generative AI or not. In the end, if the content is producing information gain or when someone reads it, they're getting value. And then it has to be part of an infrastructure that makes sense. So, like, for example, if I put out the most amazing article ever written, it's medically reviewed by 20 veterinarians. And it tells the story that, you know, the guide to owning a Boston Terrier. Right. If my site doesn't support the fact that I'm the source of content for that particular dog breed, it doesn't make sense.

18:36If my site's about ginger ale, it doesn't make sense that I publish that piece. So there's pieces of the puzzle that can't only be looked at the page level or the paragraph level. And it all tells the story of, hey, you will pay the price if any part of your site is inauthentic. Wait, there are sites about ginger ale? That's awesome. I'm sure there is. Because they're the two things I'm looking at is my dog and my ginger ale right now. So I'm sure you get asked, imagine talking to a group of agencies or agency owners of content folks at agencies. What's the future for that aspect of that business?

19:13And all the time, this is the question, right? It's what can I bring? And I think what happened was, I mean, you saw a huge fallout of low quality content providers. have all they're in the process of all going away or they're they're pivoting to focus on where could they go right they could go with oh we have to get really good at ai right using it basically on your behalf or we have to be get really good at what differentiates us so it is we will extract great information out of your expert's head or we will actually go find experts and tell that story what's the key there right the ones that are still i wouldn't call them thriving but they're a surviving asterisk right now from a content delivery is are ones that have experts in the loop at some level, or they have really great processes around building out extraction of information.

20:07Cause that's, I mean, what I, you know, had a presentation that was, you know, this is, this was over a year ago called the rise of the subject matter expert predicting. this conversation, right? I changed that this year to be the subject matter expert has risen because they are the most important person in the room. And that is where the fuck's going. It's going to be finding passionate people who can tell a unique story that can be woven into content that's assisted by AI has always been the magic. Now it's going to be the only magic that works with very few exceptions, especially when topics require experience.

20:48And you're also going to see maybe in cases where it's really hard to differentiate. So there's a lot of duplicative content. People are going to have to get creative and differentiate. And that's going to be a race to the top, frankly. So if you've gotten away with lots of generic content and it's still working now, I wouldn't feel comfortable. I would still want to go back and say, what are we bringing to this that's special? is it unique imagery? Is it personal experience? You know, I'm not saying write a 2 ,500 word log on top of all your recipes, but other industries have already gone through this and that's one of them.

21:28They've already gone through this folks. Cause there's only so many simple chocolate chip cookie recipes, right? They already had to go through this and figure out how to differentiate. And then do they differentiate with their brand? Do they differentiate with their clout? Do they differentiate with exciting imagery video? those are going to be the challenges that folks that haven't historically had to deal with that are going to have to deal with if they're still prospering with generic content yeah it's kind of like in the old days you know black hat seo you know stuff would work and that's why people kept doing it but then one day it didn't you know one day it's it's the pixar story spine but they don't like to finish it right it's like and then one day and then we said oh no the it's true it's It's all repeat, spin cycle repeat.

22:15And these are enterprise SEO brands, large brands, large publishers, away with Black Hat techniques. Oh, yeah. I mean, give me three beers and I'll tell you each one and exactly walk you through the processes that they will, that they are doing. Let's count them down. But you aren't able to do that. And if you are doing it, it is still a risk. It is still a risk, right, to do some of these things. And we see that, you know, with the gains and losses. Every step you take has to be, you want to get more predictive and less speculative. And if you're relying on tactics that are troublesome, you know, they have a shelf life.

22:56You just have to do it with those eyes open. Yeah, yeah. Well, Jennifer, I appreciate you coming and spending a few minutes on the Duct Tape Marketing Podcast to talk about this interesting and evolving topic. Is there some place where you'd invite people to connect with you, find out more about Market Muse? Oh, yeah, sure. So you can email me, Jeff, at MarketMuse.com. MarketMuse.com online, we have, you know, reverse trials so you can get free access to our paid solution, lifetime, you know, free access to some of these other solutions. Give me a call if you want to talk about more of the site level strategy, our recommendations, engines, those types of things that we're working on building.

23:34I've got tons of webinars and everything. I love this stuff. LinkedIn on the Twitter X, Jeffrey underscore coil. And yeah, what I would just recommend if you're playing with this stuff by yourself, there's a lot of communities trying to figure out best ways to do it. Just make sure you're bringing your expertise with you at every step of the process and your outcomes are going to get better. and better every day, whether it's your brand style, tone, format, or you're actually bringing in your personal anecdotes and ensuring that they make it into that final product. You can get away with just some basics like that and really making this work for you and always have a human in the loop.

24:14Don't publish right out of the machine yet. There aren't any examples of that working that I think don't have a shelf life. I'll leave it there. I know that I've written at least 4 ,000 blog posts in my career and I've never wrapped one up by saying in conclusion, which every single chat GPT article seems to do. Yeah, there are a lot of, you can, you can, they also, some of them say, you know, continue or, or I'm out of soap. Yeah, exactly right. But the reality is being able to look at one of those thousands of blog posts through the lens of a developmental editor. guess what I can guarantee you it'll be 4 ,000 great experiences and that's the kind of stuff that I'm doing because you are the expert and seeing getting through those blind spots and seeing what's magical and what you could do next you know there isn't anything out there that's doing that out of the box and so but it's not to say that it can't be done I think that's for everybody listening if you have a great use case get it in the community you know with your people you trust and they may have path to somebody who can do this stuff with AI because I know some smart folks who get this stuff done on the basis.

25:30Awesome. Again, appreciate you taking a moment to stop by. Hopefully we'll run into you one of these days out there on the road.