AI vs Google Search....behind the scenes

0:00Mark Williams-Cook:If you stand in the way of Google making money, they will flatten you. We're told make good content. That's not what Google's trying to rank. That's what they require, one of the ingredients for their system goal. And their system goal is to make ungodly amounts of money.

0:22Welcome, humans, to the Neuron Podcast. I'm Corey Knowles, joined as always by my trusty co-host and good friend Grant Arvey. What's up? Today, we're joined by Mark Williams Cook, who is a, frankly, a prolific influence in the direction of modern SEO. He recently discussed how AI is creating what he calls a leaky bucket problem for search. And today, we're going to learn more about that and a variety of other things. Mark serves as director at Candor, a Norwich-based SEO agency, and he's also the founder of Also Asked and the co-owner of Top Dog Harnesses. Mark, welcome to The Neuron. It's great to have you.

1:01Mark Williams-Cook:Thank you so much for having me. Really excited to be here. Awesome. So opening question here, and this is probably the trillion dollar question. According to who you follow on social media, AI has either destroyed or resurrected SEO. What are your thoughts? I mean, this is the running joke in the industry, right? so i've been working in seo 22 years and i think i've lived through this is maybe my eighth death of seo now um someone actually made a really lovely graph of the value of the seo industry over the last 20 years kind of going up and then marked every time it had died so uh for me there's there is a debate we maybe get into later of a lot of people renaming what we do seo as geo generative engine optimization uh for me if we're if we have to call it geo which i hate um it would be a subset of seo yeah so i i'm still doing pretty much the same things i was doing two years ago five years ago um just how i do them has changed slightly and and that's that's always happened throughout the years so no i'm still very much in the people are asking me for SEO.

2:23Mark Williams-Cook:We're doing stuff. People are getting results. You can't get rid of us just yet, despite what public opinion may want to. Despite every core algo update trying. Yeah. That's hilarious. Well, Corey mentioned the leaky bucket at the top, and this was from a podcast that you did recently that we put you on our radar, and we got really excited to talk to you about this. So can you explain the leaky bucket for our listeners and what you meant by that? And do you see the bucket getting less leaky over time or more leaky? Yeah, I think it's really, you can apply that leaky bucket analogy, which I'm probably going to regret saying off the cuff now on that podcast.

3:12Mark Williams-Cook:Now I've been cornered about it. But it's really just about how a lot of systems that are becoming reliant on large language model based technology and i say that specifically rather than just the general term ai is because the output of a lot of these systems is non-deterministic you know most of the time you'll get this answer but sometimes you'll something a bit spicy and wild it creates systemic issues for chains of things that need to be correct over and over and that leads in my opinion to all kinds of trouble with search engines because firstly the scale i think google said recently they did eight trillion searches last year like i mean that's how many searches they're doing right so even if you had a very small percentage of the results they were giving as wrong that's still an awful lot of people getting incorrect or faulty in some way information and tied to this and it's tied to how the user interface i think of a lot of the uh what whether you want to call them like chat assistants or ai search surfaces work is there's a strong user feedback loop even with traditional search they watch very closely how people interact with their search result pages what they're scrolling what they're hovering on what they click on when they come back and that works quite well when you have a kind of static page of results there's a lot of parity between searches and the user input is still messy and noisy but you've got some control over it i think the issue with providing things like ai overviews in google search is that users for a variety of reasons maybe won't realize the information is incorrect but it looks like it's correct and it looks enticing and they're satisfied with it and we start getting this um yeah this impact of we're reinforcing incorrect information or bad information and that starts to grow and then someone else starts um picking up on how that works i mean we already see it in the industry in terms of there was a lot written about generative engine optimization um which people basically were just guessing right because as soon as anything changes in SEO there's an ultimate guide to it like two days later before anyone understands it but then of course all of the AI search services you know drink this up as the answer and then a new wave of people like how does this work and then they just get a definitive answer read to them with 100 % confidence and then they learn that and go and reciprocate that and before we know it you know I'm like told now this is how this works how do you know that well chat GPT told me okay or how did it know?

6:22Mark Williams-Cook:Oh, it cited this website. This website looks like it's written by AI to me. You know, you can follow it back. So that's one aspect of that problem that exists. And I don't think there's a good solution for it at the moment. You know, those meetings that could have been an email, but somehow always managed to turn into even more meetings? Well, according to Atlassian, the average professional loses 31 hours a month to inefficient meetings. That's part of 66 billion hours wasted every year in the U.S. alone. That's why I'm excited to tell you about Simbly AI, an AI productivity platform powered by agentic tech and trusted by more than a thousand companies today.

7:07It fits right into your team's rhythm and just amplifies it. So here's how it works. Simbly AI automatically joins your calls. There's no downloads, no installs. It captures the discussion, then turns it into hyper-personalized deliverables in Word or PDF. We're talking action items, briefs, recaps, all of that, the post-meeting stuff that gets in the way of your afternoon. Recognized by Gartner as an emerging visionary in AI general productivity, simply helps you clear the busy work so you can get back to focusing on strategy. And it's built for the enterprise. GDPR and SOC 2 plus HIPAA, so IT can breathe easy, Simbly estimates companies can save$2.7 million a year for every 100 users.

7:51Founded by Gil McCliff and Artem Koren, they share 40 years of combined Fortune 500 transformation. Simbly boosts how you already work without disrupting it. So, ready to turn meetings into momentum? Visit Simbly.ai, that's Simbly, S-E-M-B-L-Y dot A-I, and try it on your next call. We had that problem at our parent company where we went and we looked up a search result and it was conflating two different websites. So it seemed like it was taking reviews about a completely different website that was like a SaaS tool and they were conflating it with our website, which was like a media publication.

8:34And it was taking the bad reviews from the SaaS tool and it was applying it to our media. And that was just right there in the AI overview. So that's like a huge problem, right? Like, because now people could then read that review and then they could think that it's talking about, you know, a different company just because I had similar names.

8:53Mark Williams-Cook:Absolutely. I mean, I highlighted, so agencies, you said it's called Candor. And there is in the UK another company called Candor Talent, which is a recruiting company. and I was googling just checking out perplexity results for kind of SCI agencies and they had a kind of rich company listing result of agencies and it had pros and cons and then it listed our cons as something like high staff turnover or something like that and I was like that's weird because we haven't lost anyone for ages and then I dug into it a bit and it was pulling in reviews from the candor talent agency and stitching them together that is such a problem and and this is you know this is because as well the i think when people talk about ai llms and the training data that goes in and this is related as well to a lot of people saying oh you need you should use structured data you should use schema so this structured markup that traditionally we put on web pages for search engines to explicitly label connections between things so you would say this is this website this website is part of this company this is the author he or she works for this company and you you map out those connections right the idea is it's explicit and it removes ambiguity now the way the large language models are obviously working with their training is you know they get given all this data or they get given they take whatever they like it seems um but when that goes through the the you know the process of tokenization and such the actual training data is not like saved within the model right that you know the text isn't there all that happens is they you know it's broken down into these tokens these components and the model is the relationship between those you know all the different combinations of tokens how it produces text so the schema side of thing the structured data can't survive that process it can kind of generate structured data because it sees those patterns together but the fact that it once saw that the organization was candor for this website because that's statistically such a drop not even drop in the bucket that's right forever gone so it doesn't that's not encapsulated in i guess what you could call like the language graph which is the the way they're they're storing knowledge so that's one of the reasons why you get those issues and i've had this with clients as well where it thought that two different websites that were named similar were the same entity if you want to call it that and you know that's another pervasive problem google is in a much better position to guard itself from this because obviously they have two decades of infrastructure of web crawling metrics they've got a knowledge graph so they can kind of ground their ai gemini against their knowledge graph and it's still not perfect but they have that data i don't believe as far as i know for instance like open ai has a knowledge graph of these are all the things and how they're connected to to ground against so what do they do they yolo it till they get more money pretty much i mean do you think this gets better with reasoning models like better reasoning models or or like well will reasoning models be able to like parse that problem or or reinforce it yeah i think so for me i think reasoning is a very interesting term to pick um and you know as far as i understand it's not reasoning in any way you or i would consider that term in the human context of reasoning it's essentially a loop of kind of trying to desperately fact check whether we've hallucinated stuff trying to recheck what averages are and find verifying information do things like query expansion um which for me still you know at the moment feels quite basic and i'm never going to be one to say oh well you know this technology doesn't work because it does this because technology is forever moving right and And, you know, I've always been, because I always come off very negative at AI.

13:34Mark Williams-Cook:I have been an AI maxi for years in terms that I've been telling people AI is going to be amazing. And I still think it's probably going to, well, you know, it's one of our last hopes of solving all of the general mess we've got ourselves into, right? What a mess we have. Yeah. Yeah. So, you know, I'm really hopeful in terms of that, you know, and there were people in the 50s being like, oh, AI will never be able to do that. defeat a human at chess because it's a strategic game and it's like i've got more than enough power in my watch to defeat 99.9 of players on the planet earth at chess right it's like a super simple thing to do now and then you know we've come on and built very specialized models so i wouldn't i'm never going to be that oh reasoning doesn't work it's not like human but i i think that before things get significantly better we need a new um type of technology i i don't think from what i've seen llms are going to get us to that stage we need to get um chat so gbz5 i think was the response was kind of a bit me you know compared to the especially early on yeah the progress we've had and the interesting thing that i've observed from gp5 is it seems more keen because i'm very interested in when these models are doing stuff like grounding when they're going off and doing web searches or they're using other tools um you know because the early models that came out you know didn't have internet connection tiny um like context window were pretty dumb gpt5 seems to be positioned more as a ask me a thing okay let me find the right tool to sort of go and use to to get the answer or generate the answer or quietly write python scripts in the background so i can get my answer yeah which i think is the right way to go um yeah and that's i guess veering more towards the kind of more agentic side of things which is i understand the task you want me to do and then i just need to learn how to use the tools to do that and we've got like all the mcp stuff coming now the model context protocol so you so that's allowing um these agents to interact with various sets of data and tooling um just by chatting to them that to me is all exciting i think we'll get to a to a better place because they've realized that you know knowledge cutoffs massive problem hallucinations huge huge problem you know even the the um the technical card for gpt5 they have the various stats on the model release for the hallucination rates you know and even under pretty good conditions we're still in the we're above one two percent in a lot of cases which again apply that to if it became google size trillions of searches yeah yeah that's still that's a lot when you're looking at it as catastrophe levels yeah yep if i went into a business and i was like look let me run all your stuff for you there's a one percent error rate they'd laugh me out the room right so yeah i'm you know i'm super excited about all of this um all this technology but i just i'm i'm cautious through experience about how much of it as well by the by the ai tech industry is is posturing for um funding for various stakeholders that are invested.

17:04Mark Williams-Cook:It's why they call it hallucinations, right? Because it sounds nice in a border and nobody wants to say, oh yeah, it just makes up shit once in a while. You don't want to hear that when you've invested$5 billion in it. There was a great paper on that about a year and a half ago on whether or whatever, that we should quit calling it hallucinations and start calling it bullshitting because that's what it's doing. It's doing the same thing we do when we don't know what we're talking about. Like, you know, you'll be in a conversation and you're there and you know you're on your toes and you make these little guesses.

17:39You make these little guesses. And I've always thought that that's one of my favorite scientific papers ever, just because it's so funny. But it's also, it makes a lot of good points. And there was the other one recently too, right, Corey? There was the one where OpenAI basically went in and said, look, we figured out why hallucinations happen. It's systemic based on the reward function. Like just for people who haven't read it, essentially they're like, we basically incentivize it to BS, like what Corey said. Because we don't give it points for saying, I don't know. Or we don't give it partial points for like, you know, trying, but then like realizing it's wrong.

18:20Yeah, it's scored just like we are on a test where like, oh, well, you know, if I can leave it blank, but if I put anything there,

18:26Mark Williams-Cook:i'm gonna get one of those five points yeah i mean that's how i passed a level biology on that exact principle exactly i need to write something yeah i was actually thinking about that paper ground when when when you said it because i saw that i think it was this morning or actually just yesterday where they're like oh we figured it out but it's like and as you rightly say it's like the the fitness function whatever you want to call it of the what you're rewarding for but how on else do you fix that if you know because you need to test it it needs to have this and and people have um you know when i had conversations with them about oh they reward the correct you know correct answers just correct answers and then the question comes up with how do you know it's correct right because you need pretty much a bank of human people to to verify that because this is really correctness is for me a really interesting topic from an seo point of view right because google has said before previously like we don't know what is correct or not and they rely very heavily on consensus which is which i'm assuming is how a lot of the models decide whether um they need to to kind of ground answers or not so if you ask what do red blood cells do?

19:44Mark Williams-Cook:When they see that string of tokens, the kind of bell curve of answers that they've seen is sort of very steep. It's like almost everything says it's high consensus. Whereas if you say, what happened in the news today? Every time they're going to come across that in training, it's going to be different. So it's going to be, I guess, flat. And then, OK, we need to go look for an answer. Yeah. But there's all the really interesting things right of um arguably things that don't have an objectively correct answer um which is a lot of questions oh it's a lot it's a lot of questions so then how do you train an ai to deal with that without making it the most vapid experience ever where it just replies like oh there's many different viewpoints to this you know those type of answers you get every time Yeah.

20:32Mark Williams-Cook:And it's just like, oh, like, I know. Just tell me which one's best. And it's like, there's lots of ways to define best. Like, just tell me what the best one is. Well, that actually reminds me, Corey, if you don't mind, like there was the article that I just read recently that was from 12 grams of carbon talking about the problem, which is the very scientific term of inshidification, which is basically how over time our platforms get worse and worse and worse. And he basically broke it down where it's like there's competitive reasons for that, you know, like and both sides, both the platforms and the users force that mechanism.

21:08But then there's the alternate take to that, which was Max Levchin, who basically said, hey, we believe that over time, AI will make people make better decisions because it basically looks at all the options and it's able to parse out and pick, you know, like, let's say our credit card or our they do what's it called? Buy now, pay later. So basically, like it's like point of sale loans. Like we think point of sale loans are actually better for the consumer than credit cards. So over time, the AIs will point you to the thing that's actually better for you. So there's this dichotomy, which is basically, is AI going to make everything worse with slop onslaught?

21:47Or is it going to make us actually choose, like you said, Mark, the best option? I don't know if you guys can both share your thoughts on that, but I wanted to point that out.

21:56Mark Williams-Cook:well i did a i did a talk recently and i asked the audience who regularly shops on amazon and pretty much everyone put their hand up and then i said you know keep your hand up if you think amazon is kind of a good ethical company you should support and maybe surprised or not lots of people put their hands down right and then we went into well why are you shopping from them and it came down to essentially convenience and and speed right you have to make an account i know i get next day or same day delivery um i know the price isn't going to be wildly more expensive than elsewhere because of the internal competition and the talk i was doing then was actually just generally about ai and about agents and search and the point i was making was that if you look at any technology essentially throughout history when you can provide someone with a similar reward a similar output for less effort that is what is going to to win because you know people want to spend their time doing doing other stuff so the and the talk was was looking at okay the future is most likely going to be personalized agents who understand you your preferences your budget your parameters you prefer buying from companies like this you like the color yellow you've got size 11 feet you know you're a man okay you want trainers this is how much like cash you have these are the brands you like and it will do the legwork for you which is arguably good in terms of i get the the product that i want and it's very efficient but it does maybe cut out some of that discovery process and i think it does also open us up to um more i guess monopolies just in terms of when things become convenient they you know the the natural kind of way of things is they become centralized and bigger companies eat smaller ones and then before you know it you can't compete and then we end up in the alien ads scenario of five companies running everything, right?

24:14Mark Williams-Cook:Yep. It's an interesting question. And something I think may help this over time. I don't know if you saw this, Grant, and you may actually have, it may have even been in the newsletter. I apologize if it was. Miramirati's company yesterday, Thinking Machines, has done this. They're publishing some information and kind of finally disclosed what they're working on. And what they're doing is trying to solve determinism in models. The idea that, you know, if we're going to ask the same question the same way 50 times, we're going to get 50 at least slightly different answers. And, you know, you don't want that for all occasions.

24:57There are areas where that would be really bad. But there are some of these things I think it will help with. And, you know, at its core, we're still very early and we're at a time where companies are just absolutely dogpiling on where am I in AI? What is my role? What am I going to do? Let's tack it on to every product we have and everything we touch and rebrand as an AI company.

25:23I don't know. I'm not sure where we go from there. I'm just kind of remaining confident in the idea that, you know, the researchers that we talked to and the people we've been chatting with along the way who are who are building this stuff. They seem to be really confident that we've still got a lot of road ahead and that it's going to improve.

25:45Mark Williams-Cook:from a from a search point of view the thing that always concerns me is the if you are a publisher online and you know you guys will know this right you have a certain responsibility in the information you publish like we work with clients where in regulated industries where if they're giving financial advice like they have to be super careful now somehow i'm guessing because of expensive lawyers and such the people that are generating answers seem to not be responsible for that you can get financial advice generated for you and because of the way it's made it's not classed as they you know they have published it that irks me from a point of view of google especially i feel has a responsibility in that they have 20 plus years of trust in their brand you know big company everyone knows google you know it's the verb right still pretty much google it yeah and i've seen my parents do this where they will search for something they will get an ai overview and they will confidently tell me this is the answer and i'm like no actually that's that's wrong like let's look at the site okay it's wrong and they have the smallest disclaimer about ai generating answers maybe you know wrong to use the like phrase yeah kind of use and i think that's so problematic because if you if you googled something and you went to a website and you saw yeah mark williams kirk or whoever wrote this you are a human and you intrinsically understand that humans can be biased that they can be wrong and you have some kind of defenses up against this oh you know mark has this political assuasion so i know what he's writing is tinged to this i can see that in his writing yeah the difficulty is when you say ai made this answer if you went out onto the street i'd be really interested to know how many people could tell you even roughly how ai views are generated and about predictive text and stuff like this it i don't think many people would and i think lots of people therefore just trust that wow it's ai must be smart it told me it was reasoning you know and it's it's it's google so that there needs to be more education to people of like you need to you check this you know and part of the the talk i did on ai was you know all the kind of basic stuff you've seen where we get these world-class models that because of how tokenization works can't count how many r's are strawberry was like the famous one we had um the academic papers you know and academic papers are meant to be like the source of our knowledge and and truth right did you see the um the term vegetative electron microscopy coming up in these papers yes you remember that one cory no basically it it so there was like two paragraphs that were next to each other and it took it as like one word even though they're part of two completely separate paragraphs and then it like spread everywhere right mark yeah and you go on to google scholar and this has been you know this is now in academic papers because they've been ai generated um and it just inspired this is kind of in this inspiring problem that and then we've you know we heard google recently say oh we're trying to just train our models on what we identify as human content because that's another huge problem right again if you start training models on hallucinated data incorrect data and we just where it becomes like ink in the water you can't ever then separate what's correct and what's not so that's always irked me from a safety point of view from search and i feel there needs to be more responsibility for them labeling look this is high you need to check this and again i don't you may have covered this on previous episodes about the studies that have come out about how developers have actually been slower in a lot of cases using AI.

29:54We definitely talked about that in the newsletter. I don't think we covered it in the episode. But yeah, it's like 20 % slower or something like that.

30:00Mark Williams-Cook:And I've seen that, right? Because you write the code and because you haven't thought it out yourself, you don't really know quite how it works. Then you've got to maintain it then something breaks and then you know my experience of coding with ai if you use it to debug is it gets progressively worse essentially like it will do a great job on that first script you kind of add to it and then you get a problem and then you know it's the classic programmer thing of you know there's 99 bugs in the code fix one there's 100 bugs in the code kind of thing but really fast with ai right um so yeah this is this is the thing that makes me nervous like how long we're going to stay in this stage before we get to what Corey said, like, you know, this golden road, this promised land, which is just a bit further down the road, just another billion dollars down the road.

30:50Yeah. And pretty quickly, we as humans have proven that we will absolutely trust anything pretty quick and easy. We do that in every aspect of human life. And I don't know why I thought humans would be more discerning about AI copy as far as when they're working with it and using it and making sure to give it a good overview before they're running out and publishing things or whatever the case is. But boy, we really don't as people. We've kind of just accepted it. Unless you're in your specific field where you have the knowledge, where you are aware of those nuances, it can be really convincingly wrong if you have a cursory knowledge of something as opposed to being an expert in your field.

31:41And like, I find that people who are, with that said, people who are really good at a thing tend to be better at it when they understand how to use AI in a smart capacity, like where it can be strong and understanding where it is not. But, you know, and it's one of the reasons, like, I love to bounce between models and companies. But the truth is, when I'm doing my work, I'm doing most of that in ChatGPT, just largely because I know where it's going to fall short. I, you know, it's what I've used for, if it's an important task, I'm going to use the tool that I know what I need to watch in, as opposed to one where I'm maybe less sure.

32:24Like, I mean, it's probably just as good. But, you know, the truth is, oh, these models are good now by comparison to, when you look back at what we were working with two and a half, three years ago, it's a, it's. One year ago.

32:36Mark Williams-Cook:One year ago. Pre-reasoning. I mean, with all its faults, it's still pretty bad. Yeah, we didn't have reasoning models. Yeah. Yeah. With all its faults, they're still quite good. Yeah, there's still a major leap. And fingers crossed we'll have a bunch of major leaps ahead of us. I have another SEO question I'd like to bounce off you and get your take on. So you mentioned that as people move to AI search, that traditional link graph that Google has relied on for all of these years is going to probably rot. And I was wondering if you could walk us through why that matters. What are the impacts of that?

33:15Sure.

33:15Mark Williams-Cook:So I think from an AI search, AI surface point of view, one of the challenges they have is understanding the source of where bits of information come from. Right. Yeah. Because I could give you some instructions on how to fix your car engine. I just stopped working and I have no idea about that, you know, but I could tell you some stuff that probably sounded right. Like, you know, you would much rather listen to someone who is like an experienced qualified mechanic. Right. And. The beautiful thing about the link graph, and this is like the foundation of why Google set itself apart as a search engine was the this page rank algorithm, which is essentially trying to judge initially the popularity of different documents on the web based on how often they're linked to.

34:13Mark Williams-Cook:because it's a very good proxy for popularity when people find a resource helpful they will link to it and those votes if we want to call them that accumulate so if not all votes become equal so if everyone's got their vote and everyone say votes for me and i get 10 votes my vote then is worth even more because i've already got 10 votes so it's really it's really interesting how this lines up and then there's other layers to this such as topicality so we know for instance if we're looking for information about cars there's this interesting network of sites and these three sites are always linked to when people talk about cars and they must be really good so if i'm if i'm looking for a document about cars while there might be 10 million on my very small internet and you know the the 10 000 that are in this little crust that i'm going to look at first because everyone references him and that's that in a very very um i guess uh not doing it justice but that's the basis for how i think google really became what it is today in terms of building on top of that now ai search doesn't have that they have the the text from everywhere but working out which is the best text apart from the most common which is the issue we came to earlier because the um the consensus answer isn't necessarily the best answer when you need expert information right now it's not widely spoken about but the large language model companies are actually buying leaf graph data from the uh companies the sas companies that spend a lot of time money making their own independent kind of crawls of the web and they have their own kind of scores um which are similar to google's in some ways and they are selling that data which i assume is being used during pre-training to turn the volume up kind of on those sites because they we know they are for want of a better word good non-good yeah yeah so um and this this has worked really really quite well for a long time and notwithstanding as i said before google uses a lot of interaction kind of click data to to refine what it shows but as a core we can trust these documents that's still the most impactful thing you can do i think in terms of seo once you've done all the other basics is is you know earn this coverage and links from people now why i said it's going to rot and what kind of worries me is a lot of the ai search tools will do grounding i.e they will do a search they will do a web search and this is happening almost exclusively through like google and bink they're doing they're saying okay you've asked me this question i know i need to check the answer i'm going to quickly go and do three searches on Google, look at the first 10, 20, 30 documents, check what my model output and make sure everything's kind of hunky-dory and then give some very scant citations to who I just digs all the content from.

37:31Mark Williams-Cook:Now those results are powered by that link graph. That link graph survives because people are out there journeying the open web, clicking on links, looking at websites but if we change to a paradigm where well i don't need to go to reddit anymore to get this advice i don't need to uh read this you know very specific guide on how to optimize my dwarf fortress building because i can just put it into chat gpt right therefore i'm not going to link to it i'm not going to even visit it i'm not going to link to it i'm just going to read read the guide right so the actual living expression which it kind of is you know of what is popular on the web just starts to fade away which will have an impact on the search results and actually the the user data which is hugely important also dries up because if the agent is doing the search instead of the user you've lost that vital feedback loop of oh well everyone's skipping the first result and actually going to the second because the ai is just going it's digesting 30 pages a second you know i'll just take it all um so then how how is a search engine do you firstly determine which documents on the web are reliable that was very challenging and then secondly which ones are relevant which ones do people like you know because that amount of data is dying and then that if that happened is going to have the upstream effect on the quality of the results in the ai search is also going to deteriorate and then i don't know where we go from there the only thing i can think of is we've seen oh you know and they didn't get forced to do it obviously google there was possibly they might have to sell chrome uh perplexity has launched comet its own browser um is that they could harness information or how people are interacting through the through the browser which i think again talking about earlier the the the simplicity of google results made i think the crunching of that user data possible i think the the variability of people asking a million different questions a million different ways and then trying to figure out what's good and what's not based on the browser interaction and you have to account for the fact that people might give positive signals to incorrect answers because they think it's correct as you said cory you know you're only really asking the ai basically questions that you know the answer to and it's helping you and inspiring you but then it's like what's the point in a system you can only ask questions you know the answer to it's almost like you need to know the identity of the person who's asking the question and their level of expertise to be able to weight their like interactions appropriately and then that's a whole can of worms yeah right and and that that's contrary to like all the changes we're seeing in privacy right yeah privacy is moving the opposite direction it's like no cookies like browsers now stopping um they're like stopping the letterboxing the fingerprinting with browsers so they're adamantly trying to push you know the uk everyone's on vpns now because we've got these sweeping online safety laws which means i can't even send direct messages on blue sky anymore unless i have government idea that proves i'm 18 wow of course i'm just going to use a vpn like yeah yeah that further obfuscates who i am so um i wouldn't say i'm i'm worried because it's not my problem but i'm really interested in how it's going to pan out because there's the big cheese right all the money is at the end of that tunnel and it's like who's going to blink first because you know there's a lot people you know way smarter than me working on this right so i really hope they've thought about it and i say hope because i've i spoke recently to um uh so this april to uh someone working at google and i was talking to him about search console and the data we get through there and what would be helpful and they were they were saying like well why don't we just give them like this score you know like from from google about this and he's like no no no you don't understand what happens if we tried it with like toolbar page rank like it will create chaos if you give them any data like that it becomes you know there was a whole market setup for link selling based on page rank of individual pages so it's like it's a bad idea to give seos specific information give them vague guidelines and you're an seo person saying that so you know that like it's a it's like uh you know both sides problem yeah absolutely i mean we found them we found a google exploit last year that gave us like thousands of parameters they use in their in their search ranking how they classify queries it gave us uh quality scores for websites um and in the end after having a good poke around that we did report it to google and get their bug bounty because i was just like if we released this it's just it's not going to be any good you know it's it's not going to make the weather a better place um you know and it's not going to help people that are actually trying to you know because although i you know seo has a depending you ask a varied reputation i am you know in candor we are generally trying to help people make the best business decisions in the search environment we're not actually interested on in how i can trick the search engine or you know how we can manipulate it because they're short-term things generally the the overriding goals of these systems will be in line with making a good business and there's lots of ways to go about that so even if we get insight into how google's working specifically while that's super interesting and answers you know fills in a bunch of blags giving away or publishing that data is only going to aid people who are trying to take shortcuts and then you know not the types of businesses to be honest i want rewarded you know that leads really well into another discussion we've had lately um i don't want to name any names and jump on any companies about it but we've seen a variety of ai tools that are are claiming to have all of this great data about seo and about not not about seo specifically but yeah geo uh you know to do with you know firsthand prompt information and these type of things and that they're really able to get to the nitty-gritty of what they're seeing and it i struggle to even understand how it's possible because like i know we're in a situation where most of these labs are pretty competitive and an ugly privacy situation would not be a win for them.

44:55We saw that kind of with some of the situations where like the chats that people were sharing was showing up in search results, right? Yeah, that too. I guess where I was going with that is, do you have any idea how that works? what what kind of data is being used to assess these and and you know i know there are a lot of companies spending money now trying to find out what should they be doing and that's that's

45:24Mark Williams-Cook:kind of an interesting direction so you're talking about the tools that are kind of trying to look at ai visibility yeah yeah that kind of stuff yeah so i've uh i've been grilling these companies so when they email me to say oh we've got this brilliant tool and i'm like great come and explain to me how it works and then i ask them all the questions that they don't want people to ask and i'm like okay but what data and where does that come from and how are you getting that um and and it's it's a real mix so i think you've got the uh traditional sas tool companies that have worked in the seo space a while they have existing huge databases of keywords and search volumes and stuff and some of them are just pumping traditional search terms like straight into um into ai tools to and then seeing who gets cited which according to them um it gives very similar results to if you ask like a natural kind of question yeah um but again i'm i'm skeptical just because they have a financial incentive to tell me that um and like a lot of the work we've been doing is so without going off off from your question here when we are taking traditional kind of search terms and trying to do research for ai surfaces if you tell chat gbt you're a vegan one day and ask for a vegan recipe and then you ask it for running shoes two days later it's gonna go oh i know you're a vegan here's cruelty free brands and shoes without leather which is a completely different result to if you fresh out of the box was were like give me running shoes right so like you can't prepare for that yeah well so what what we've been doing is we will write one two three four five depending how many different like personas line up to that company take the traditional search terms use the chat gbt api and say okay you are a middle-aged male beginning running health conscious vegan guy you're after you're looking for this information and then you give it the traditional search terms and you say what would you what would you ask chat gbt now i would never recommend anyone do keyword research on an llm for a variety of reasons but this is not that this is actually the perfect question um that's encoded in their language graph because the language graph has been hoovering up hundreds of millions of conversations from very specific websites where people say hey i am this old and i'm interested in this i'm looking for this whether it's a running forum or it's the vegan subreddit it's the perfect thing to ask then we get those prompts and then we can there's other tools we can work further down the conversation um with so i think that's missing from that classic sass approach which is dumping the keywords cold into it um some of the the other tools have been taking very interesting uh i think this is more the funded route where it's they've generated queries like that they call them synthetic queries synthetic um they're like synthetic cdo's they didn't cause a problem but um so they take those uh queries and basically they just hammer them into the into the uh into these such um these ai search platforms to try and get the uh most common answer because that was my question to them of well how do you know they're being cited because i might get a different result and they're like oh we just keep like banging our head against the wall and look it's like isn't that like really expensive they're like yeah um and that's why those platforms are expensive yeah and there's some very like hand wavy clickstream data um which gets a little bit murky um whenever you know you talk about clickstream data that's the usual stuff of people sacrificing privacy for functionality like the whole free you know vpn game is basically if you're using a vpn for free it's probably because you're an exit point for people that are paying somewhere for a residential proxy that you didn't read the t's and c's about yeah yeah freebie free bpn bad idea yeah yeah absolutely so um my view which isn't massively popular in the SEO world is at the moment i think the cost in terms of time literally energy and and money for the value you get from some of this tracking is too high for me at the moment for some of our clients we're just setting them up with their own tracking because it's actually not too difficult um as long as you're not doing you know like 20 000 prompts daily yeah you get a kind of toe dip into are we even showing there's other really cool tools like um what ai knows about you yk which allows you to explore some of the base models as well and what that will do it's really clever is you just put in your website and it will basically start asking bunches of questions based on your website to the model to find out what it thinks about you which i think is actually one of the most valuable things because that's cool if in like the 1930s say right brand was basically what you told people what you would stick on billboards and what you ran on television and then you know as we got the web brand became more what people say about you what's alive on you know if you did something awful it would get around the web and people would know yeah and i think we're approaching an age where brand will become essentially what's imprinted in ai models about your company and that's how people last because finding reviews of companies and products is hard work you know i I chose my house conveyancer by using AI this time around.

51:21Mark Williams-Cook:I was like, go and fetch me a bunch of reviews. I want summaries of pros and cons of what they've said, you know, price ranges. And I did verify, you know, I got the top couple and I went and sort of did some fact checking myself, but it did a good job. You told it to make you a buyer's guide. Yeah, right. Essentially, it made you a custom buyer's guide. Yeah. So that's awesome. I think the tools like that for visibility are more useful. So I want to know what does the model kind of think about me? What shape has it got? Does it know that this thing is important to us? And they're the differentiators, right?

51:58Mark Williams-Cook:So rather than the, yeah, we are this company and we sell this product, the AI needs to understand the why. Why should people buy from you? What do you offer? Who do you appeal to? that's more interesting to me than just hammering the ai about or what does it return for this query or this one because as well there's been studies that show they don't even seem to stick that long the answers they change quite regularly um so i just feel like it's a lot of time money and effort chasing something down the road when and it's a classic seo problem we could be spending all that money in making a better company and making our product better and making our customers happier and working on why they should buy from us.

52:41Mark Williams-Cook:And that will surface itself in the model, which is what I'm saying. If we understand the goal of the model, let's aim for that rather than try and aim at where the model is now and how we can kind of exploit that. Manipulate it, yeah. It's a good long-term strap. I feel like what we're kind of talking about here is that the idea is there's a lot of noise right now and maybe the best course of action is focusing on doing your best darn work you can and adhering to your SEO fundamentals, kind of? Is that? Yeah, so there's definitely things that need to be done differently. So as I said, we've got a whole new keyword research process we go through to understand how those queries might look.

53:29Mark Williams-Cook:We map out. We've got data that maps out where the conversations go so if someone's buying running shoes right after they investigate the price it's really common for people to ask uh something like how much faster will expensive running shoes make me run so they're trying to justify the the purchase right they're like what's what's what will this shave off my 5k time so understanding all of that is way more important than understanding kind of the the keywords in terms of what we do day by day some of the technical aspects are more important so we know a lot of large language models don't do the expensive rendering of websites with javascript so even more important than it has been previously to make sure all your main content links load just uh you know as it's served in the html i think digital pr which has always been you know part really of seo or at least very adjacent is way more important because the visibility side of things has moved on from becoming this is our website and we must get it to rank to we want to be seen in all of as many of the results as possible so if ai is doing that search and we've got our running shoes the first one is a review website the second one is someone's blog whatever like let's send them samples let's get on there let's have them talk about us link to us let's have good sentiment about us um so it's like red reddit comes up a lot at the moment in discussions about seo right i like my reddit account's old enough to drink now it's like 19 years old right in the uk at least um so i've been hanging around reddit a long time and it's a very interesting place on the web it sure is i love internet elders on there and it it's it's always awful for me now all these marketers being like oh yeah we need to be on reddit and it's like no stay away you'll come ruin my favorite website yeah yeah and so i've been talking to clients about because they're like oh we see you know reddit's using the train reddit comes up a lot and it's like yeah so what you need to do is become the story that the redditors talk about provide the data that they find interesting and then go and join in the conversation i was like the strategy a lot of people seem to be taking is like oh there's a party going on there i'm going to climb in through the window and then just talk loudly to everyone that will listen it's like start dropping links to blog posts and uh yeah like it's and you can see them straight away right on their brand new fresh account and you look at you know people look at post histories on reddit right it's one of the first things they do it's like you seem a bit astroturfy let's see okay yeah even the reddit bots will eat them alive yeah so yeah it's it that to me is like all part of what SEO is becoming the education we've got to help um clients with and again it's that strategic thing of you're aiming for what the system goal is not what it's doing right now so just because yeah reddit's coming up quite a lot doesn't mean you have to like be there like yeah but there's ways to become that that that thing people are talking about yeah so what what is then in your view technically the the system goal it's to surface the best information for the the user's query like is that or is that wishful thinking so i had this as a as like a slide when um we're going through the presentation about the google exploit which was understanding you know um google's um like system goals which there are many and we're told as seos as webmasters as business owners you know make good content or make quality content whatever they want to call it this year and people do that and then they don't rank right and it's because that's not what google's trying to rank that's what they require one of the ingredients for their system goal and their system goal is to make ungodly amounts of money like that's the nowadays the overriding system goal right and the reason that organic results exist is because people prefer clicking on them to adverts.