229: Does llms.txt Work? What 137,000 Domains' Server Logs Show

0:00We need to talk about hacks. Most of the stuff that's sold as quote-unquote AI search optimization has never really been tested by most of those people selling it.

0:15You know, the tracks don't survive contact with the reality, contact with server logs. So today, let's talk about them. Let's talk about hacks. Let's talk about AI hacks and let's talk about why the market, why digital marketing keeps producing short term tricks. And let's talk about how we can all avoid that. This is no hacks, authentic web, no hype, it's not easy to do but I keep trying.

0:51So here's where we are. Over the past year or so, there's a whole economy that's grown around one promise. We will get your ass into AI answers. There are special files you're supposed to add. There are visibility scores. There are citation services to get you mentioned in the places where AI systems are supposedly getting the information from. There are checklists. There's Reddit bombing. Some of it costs a few euros a month. Some of it is a lot more expensive. But you know what? Almost none of it has evidence. And I'm not saying the evidence is weak. I'm saying there's zero evidence for things that are being sold as AI search optimization service.

1:35The typical pitch I keep getting is either a screenshot of ChatGPT or sometimes they're not even a screenshot. It's just a, we can make this work for you. And I get why it sells. Because AI Answers and LLMs and GEO, AEO, it's the blackest of black boxes. You cannot see inside the system that decides whether your website exists, whether it should be mentioned. And when people cannot see, they pay someone who sounds certain. Think about the people that in the, not the early days, but like the early days of SEO growing, there are people, I'm not going to name names, but there are people that come to mind that just sounded confident.

2:22They were not good at their job. They just sounded confident. That's the market right now. The fear of a black box, big budget because spending money on traditional SEO is not as wise as it used to be. and a lot of people pretending they're confident. That's the entire market. So let's do what nobody selling this stuff does. Let's actually check one of those. Let's actually check one of the greatest things in the world of GEO. Ask any LLM how to optimize a website for LLMs and this will be number one or number two or both. I'm talking about LLMs.dxd. if you are lucky enough to never have heard of llms.txt, it's a text file you put on your website, kind of like robots.txt, but for LLMs.

3:20Not really, but kind of. And the idea is that the AI system will read it to learn who you are, where your important pages live, all that kind of stuff. It's a very plausible idea. On paper, this sounds good. Now, there's a spec. there's a proposal, there are numerous conference talks, and there are thousands, if not tens, or hundreds of thousands of websites adopting this. SEO tools are flagging websites that don't have one. Agencies just put this in a deliverable. This is a task that an agency doing AI SEO, GEO, AEO for a client must do because the clients expect it. There's one problem. There's one problem with LLMs.txt and it doesn't make a lot of sense today.

4:16It might in the future, it could be LLMs.txt, it could be something similar, but today it does not make sense. In June, so two months ago, Ahrefs published a study. They looked at server logs across 137 ,000 domains. Server logs, not analytics, not Google Analytics or anything like that, server logs. What do they need they found?

4:43So 28 % of the domains that was observed in this study had a valid LLM.txt file. Sure, and Ahrefs customers are more technical, they're more advanced. So 28 % is probably as high as this number could ever be. Still, it's enormous adoption for a file that appeared out of nowhere. Second number, and I want you to pay attention to this. Listening, 97 % of those llm.txt files across 137 ,000 domains got, you want to guess how much traffic they got in May? Zero. Zero. So not occasionally visited. Zero for a month. Zero requests. And that's exactly what I'm seeing at nohacks.co. I'm looking at Cloudflare logs.

5:41Nobody cares about these files. So all the crawlers, GPT bot, CloudBot, Perplexity bot, everything else. the systems that this file was invented for, they are happy reading HTML. Huh. Now, there's a fun detail on the 3 % of domains that had something fetched the file. Huh. Of those 3 % of domains that had anyone look at their llms.dxt, 22 % of those readers of LLMs.txt files were, you're not ready for this, SEO audit tools, the tools that tell you that you have a problem if you don't have an LLMs.txt file. That's the main thing on the internet that reads LLMs.txt.

6:37So if someone tells you that this will improve your website in any way, they're selling you a hack now does this file have a good potential use? yes it does absolutely and even Google has it in Lighthouse for the agentic check agentic readiness I don't know what they call it they will check if you have an lms.txt file if it's formatted well all that stuff if you are a developer you can just point your coding agent assistant whatever you use at an LLAMPS.txt file on a documentation website. And for that, it kind of works. It's a niche tool. This has nothing to do with AI visibility, with AI search.

7:26And look, every client I've had this year has at least asked about LLAMPS.txt. I don't know how they learn about it. But just like the rest of the AI industry, this thing is so loud and so useless. not the rest of the AI, most of the AI industry. So useless that I just don't get it. It's a markdown formatted text file. There's nothing fancy about this. And according to this study, well, nobody's reading the file. So that's one thing. LLMS.txt, it's been bothering me for a long time because, you know, you know those Jordan Petersons and Hubermans and all those grifters and scammers, they will tell you something that sounds like it makes sense.

8:19And I'm not suggesting that the entire AI search industry, I don't know if that's the name of the industry, but that entire category is trying to be Jordan Peterson. I hope not. But if someone opens with LLMs.TX, and tells you that they will put you in chat GPT, run, run, run away, don't look back and never, just block their email. So the question really is, why does this keep happening? There's always some stupid hack that if you think about it for a second, you realize that there's no way this makes sense. Like stuff your page with keywords years, decades ago. This is so stupid. If it works, it works because the system that's supposed to use this and monitor this is not advanced enough.

9:14It will stop working and then you'll get punished. Same thing with Reddit bombing. I don't see a lot of that anymore. But the geo strategy for a while, maybe still is, for a while was just post all the things about your brand, just listicles or whatever on Reddit, comparisons, and then ChatGPT will cite that. Look, that's nice and sweet, but you know it's a hack when you start doing it and you know it's going to stop working when you start doing it. So the conditions for this are really, really good because this AI thing, LLMs, it's a black box for people who are building these things. Nobody outside a few labs can even observe what's happening accurately.

10:00I don't think they can, but they got a higher level of accuracy than us. So everybody has real fears because the traffic reality is shifting. We're going away from SEO and Google sending clicks. And now those budgets need to go somewhere and you need to show something for them. And it just needs to be spent. And that combination produces products and situations that just don't make a lot of sense. Look at visibility scores. A vendor will sell you a number that just says how visible you are to AI. Not all vendors, but it's a selling point for a lot. I get those pitches daily. There's a new AI visibility tool every day that wants to be featured on the blog or podcast.

10:55And what does it even mean, how visible you are to AI? What's the prompt? if I prompt all the LLMs about something that's specifically about me, I'll be very visible. Fun fact, if you want to go on an LLM deep dive rabbit hole, go to free chat EPT. You're not logged in. You're logged in, but it's free. You're not paying. It doesn't use web search by default. and just type in what is and then your first name, last name, known for. And just send me screenshots. Hi at Nohack CEO. I want to see what you're getting because I'm not happy with what I'm getting. It's disturbing. So visibility scores are just a grading exam for the test that you're writing the way most people are selling because they'll just pick some prompts that are easy to be mentioned for.

11:59I'm not saying it's all like visibility, AI visibility, share of voice, all that stuff. That's not fully a scam, but the way it's being sold will get you into ChatGPT. I mean, you ask ChatGPT something, if it doesn't have you in the knowledge base before the model cutoff, training data cutoff, it will just search. it will use Serp API to search Google or get Google data and it will get whatever's ranking on Google or wherever else. Let's talk about citation farming next, the prompt stuffing, all those things that make very little sense to me. They're all, just like anything else with AI, they're a sexy demo where you can show that something is happening or suggest that something is happening.

12:53But it's not new. Every generation of search produced the hacks. I mentioned keyword stuff, link farms, doorway pages. And every generation of hacks just died the same painful, boring death. And the ones selling that, they're not there to explain what failed. They just move on. I want to feel bad for people getting burned because there are some people that are just getting scammed into this. But for people doing the scam, that's just, no, no, no, no. So my position about all this, and like I've been talking about this on the podcast, I've been writing about this on the blog. My take is that I really believe that your next million visitors to your website will not be human.

13:41I do believe that the machines that are now reading and citing websites, are moving into using the websites, buying from websites. That's what the podcast is about. GEO stops at the door. It just focuses on read and cite. I think there's more. I think there's a lot more. So there's one part that there's a lot of nonsense being sold. And there's also my belief. I really think that this is going somewhere. I really think that the Internet is changing and it will never go back to how it was. And both of those things can be true at the same time. I'm not anti-urgency to change things. I'm not anti-AI in general.

14:29I'm just anti-unevidenced, for lack of a better word. But the shift is real. The shift is measurable. But a lot of stuff, the stuff that's being sold, simply is not. By the way, a reminder, I have a weekly newsletter. If you're not subscribed to that, this is where I try to share the most important links from the previous week. I usually send them on Wednesdays. I might change the day. I will see about that. But if you're not subscribed, please go to nohacks.co to slash subscribe. It's also linked in the episode description. Back to the episode, back to hacks. So as I said, just being purely a skeptic, it doesn't really do much.

15:20So if I tell you that all those tricks don't work, and I really believe they don't, I'm not seeing any of that working. I need to tell you what I think does work. And I'll bore you to death because what works is boring. There's a reason this podcast is called No Hacks. I named it in 2021, 2020 before launch, really. Before any of this existed, we didn't have LLs back then. It was, well, it was the pandemic, so it wasn't better times. But it was easier without the LLs. The name of the podcast. I don't hate the HBO show Hacks. I love it. It's not that. But the name of the podcast is really Fundamentals Over Tricks.

16:05So what survives a Google update? What survives model updates? It's simple. It's boring as hell, but it's simple. It's a structure that a machine can read without getting confused. It's facts that are stated plainly enough to avoid being left on the page and survive being pulled out of the page and use somewhere. It's real capability that an agent can call on the page, an actual endpoint defining whatever way that answers an actual question for that agent. It's architecture over short-term tactics. And if you want my definition, a hack in this space will try to influence what the machine says about you.

16:57Not what you are and what you do, but try to have that image distorted. Architectural changes change what you do and how you do it and how you say it. And then based on that, the way a machine can read and what it can do on your website. The first one is purely rented. It's short term. It might work. Then the model gets updated and doesn't work anymore. The second one is fully owned. And I want you to think about that. Just think about all those Google updates over the years. It was painful for a lot of websites. And there was a lot of complaint. And now when there's a Google update, when there's a YouTube punishing channels and demonetizing channels, yeah, yeah, dude, you knew you were doing something wrong.

17:46And now the time is up and doesn't work anymore. I'm sorry, but I'm really not sorry about that. If something is a hack, it's designed to stop working. If it's an architectural, structural change, it's not going to stop working. And that shift to the non-human visitors, I know it's early. I know agents are barely buying, barely doing stuff. The foundation for that is being set this year in particular. But that shift to non-human visitors is very, very, very real. And that causes a lot of panic. That causes a lot of confusion. But the way to make sure you're ready when you don't know what the way to prepare is, just focus on doing all the fundamental stuff the best you can.

18:39So I'm not saying all those products that help you track air visibility are necessarily bad. I'm saying if your primary focus is being best X for Y in chat GPT today or perplexity or Claude or Gemini, maybe ask yourself if you really are the best X at Y, if your product is, if the world and the internet think you are, if they agree on that. and maybe that's the problem you need to fix because LLMs are really nothing but a mirror of everything happening online by design they stole the internet and that's what we get let me know what you think about this hi at nohacks.co I really want to hear from you thank you for listening talk to you next week