Vibe Coding - yay or nay?

0:00In this episode of Search of the Record, I'm talking to John about Vibe Coding. We talk a little bit about what it is and what tools we have tried, what our experiences were, and where we think these tools help us already and where they might not be as helpful as they want to be. And we also want to hear from your experiences. So let's dive in.

0:33Hello and welcome to another episode of Search Off the Record, your podcast coming from Google Search Relations, right to you wherever you're listening to podcasts. We want to take you behind the scenes of search a little bit and show and tell you a little bit of what's inside of our heads. With me today is John Müller. Hi, John. Hi, Martin. Same team, same place, same fun. John, I want to talk to you about Vibe Coding websites. Okay, let's do it. Okay, excellent. I think there's a lot of talk about AI becoming a tool that kind of like fire and forget, and you can do anything. You can create videos.

1:13You can make music. You can do whatever, including making programs and websites and apps and stuff. And I tried it. I finally tried it. I tried AI Studio this time. Did it work? It was great, actually. I wanted it to build me a website. A static website didn't need, well, not really a static website. I wanted it to build a client-side tool with some JavaScript. And it did that really well. And the nice thing is I was surprised that I could actually read the code. And it looked more or less like a standard Next.js application. I wasn't too thrown away with how the weird setup sometimes looks like when you have other people writing code for you.

1:57So that was nice. But then at some point, I got into a loop where I asked it to not do something. And it vibe-coded. It literally kind of vibe-coded because I was like, and use this library. And it was like, no, no, I built this myself. And I'm like, no, no, no, no, don't do that. Use this where I know it works. And it's like, OK. But it lied to me and kind of got stuck. And then I asked it for a half an hour. I tried to make it not do what it wanted to do and want to do what I wanted to do. And that was weird. Okay. Did you try something and did you have other results or did the same happen to you?

2:35I've been making a lot of websites mostly for myself and to test things out using Vibe coding. I don't know, Martin, just an aside. Was the UI in purple? No, I heard that from many people. It wasn't, interestingly enough, not in purple. Okay, nice. Okay. Just checking. So I think maybe taking a step back, Vibe coding is this fancy new term that someone came up with from the AI world where basically you have an AI system write the code for you and you basically talk to the AI system using English. Or I'm pretty sure you can also do all of this in German or any of the other languages where you basically just say like, hey, I want a personal website.

3:23And it should look like this. You kind of describe what you want it to look like. And it should have these pages on it. And then the system goes off and basically creates those pages for you, those files. And basically, from a technical point of view, you have to do an additional step after that, which is basically take these files and put them on some server so that they actually run as like a normal website that you would do. And I think the allure a little bit is, well, you speak to the system in English. You don't have to know JavaScript or any other programming language, basically. And it goes off and does all of this technical stuff for you.

4:10And then you take the output and you put it somewhere, and then you have a normal website, which practically speaking, nobody can really recognize as being like, this is a vibe-coded website. Of course, if you use some of the frameworks or platforms that are used commonly by vibe-coding, people can recognize those patterns. But otherwise, it's a normal website. And I think one of the things that is perhaps tempting is to take it to the next step and say, like, oh, you should also create my content for me. It's like, ah, now you made my website look nice. But why don't you write an about me page that lists all of the things that I've done?

4:57Or it's like, why don't you write a marketing page that makes sure that everyone clicks by, which is like, sure, you can do that. But it's not something where I'd say it's like, where you're getting the most value out of it. Interesting. Yeah. I mean, with content, it becomes tricky because then, like, why would I visit a website where the content has been AI written when I can just, like, talk to the AI directly, right? But with the structure, I mean, if you're using frameworks, then most of the things look roughly the same. Interesting is when AI apparently prefers purple. So if I see a purple website, then I can ask myself, is this really the preference of the editor or the author of this website?

5:41or is this something that just happened because it was AI? Yeah, it's kind of weird. I think also one of the things that is, from my point of view, kind of interesting is, like one of the selling points is that you don't have to know how to code. I was about to ask you that, yeah. Is it this fire and forget kind of thing where you just describe it in English, in plain English, like I want this with that and this button and this thing and this stuff, and then it just generates the right thing because it worked for me this one time, kind of, 90%. I think it depends. It depends on the complexity that you're trying to do and kind of the long-term setup that you want and also almost like the cleanliness that you want because it can definitely create a generic website based on the information that you give it.

6:46But it will make a variety of assumptions along the way. If you go and say, I want a personal website and these pages, it could go off and say, well, I'm going to use a static site generator because those exist, those are easy to use, and I will create these pages for you. Or it could go out and say, well, I have all of these JavaScript frameworks that have a lot of open source documentation and tooling. I will make a JavaScript-based website for you. Or it could go out and say, well, actually, you probably want a CMS, and therefore I will create this complicated system that has a database backend that lets you edit your homepage and all of this stuff in a browser kind of thing.

7:32All of these are reasonable assumptions where if you talk to a developer, they will also make these assumptions. But if you just tell the AI system, like, I want a website, then it will pick one. In ideal case, maybe it asks you some questions like, what do you actually want to do with this? Are you going to make changes or not? Do you mind working in GitHub? Do you want to do it in a UI? Those would be reasonable questions to ask. But often the system will just go off and say, sure. It's like, let me take care of this thing for you. And then it goes off and does stuff. So that's the kind of thing where I think if you know what you're doing with regards to how to make a website and you have a concrete idea where you're like, well, actually, I want a static site generator because I just want to copy and paste the files to, I don't know, Firebase hosting or Vercel or anywhere.

8:30then you can tell it that ahead of time, like when you're setting this up. It's like, hey, I want you to create a static site using maybe my preferred framework. And like, here's like the setup, the structure that it's supposed to have. But for that, you need to have the technical understanding. Yeah, exactly. That's where I think the technical understanding comes in. and especially when you take a little bit past that and you're thinking like, well, what about SEO? And of course, in the end, you can always tell the AI system, now add some SEO to it. But you know how that works out. It's like if you go to a developer and add some SEO, it's like, what do you mean?

9:17Like sprinkle some meta tags and add some structured data? It's like, what do you mean add some SEO? So those are the kind of things where I think if you have the technical understanding, you can guide it from the beginning. And you can say, like, actually, when you create my site, I want you to do this, this, and this. Make sure that the pages have, I don't know, canonicals. Here's the domain name I want to use. Make sure to set up a sitemap file or whatever that you're trying to do. So on the one hand, like knowing what technically you kind of want helps to get there quite a bit. What also helps a lot is if you understand things like how a deploy script should look or how a pre-submit should work, how a linter works, those kind of things.

10:13then you can also include that in the configuration that you're setting up. So that you can essentially say, like, this is the structure you should create. And you should make sure to, as like before publishing, make sure that like these URLs all return content. Make sure that, I don't know, none of the JavaScript files are blocked by robots.txt. And I was like, create a robots.txt file and make it useful kind of thing. Like all of these small details that help a lot. And I think especially the pre-submit stuff is something where for a lot of the mainstream frameworks, there's a lot of tooling out there that helps to make sure that, oh, like I built this site and I published it and it's actually I'm publishing the right thing.

10:58I'm not publishing my source code and all of my API keys or whatever you're doing. That keeps happening to people. I keep seeing that accidental API key publishing. Okay. But all right. assuming that it can build these things. So it's not like a one-click more or less kind of thing where you just say some play in English and then get whatever you wanted back. You need to still have some idea of where you want to go and some of the technical lingo because that helps steer the output. But when it's created, I also have to test it. And depending on what I have asked it to do, testing manually might be a bit tricky.

11:37And also when it fails in subtle ways, Like, I mean, if it fails with an error message, you can just give it the error message. You can say like, oh, I clicked this button and then I got this error. Can you fix that, please? I think please makes it more expensive, right? I don't know. So one of the weird things I heard or I read somewhere in one of the random leaks that happened from the AI labs is that they have dashboards specifically for the coding tools about how often people curse at the AI system. Oh, okay. Because it's like a sign. is like, oh, it's like our system did something wrong. And like, it's sometimes just enough to count like how many times people are cursing at it.

12:19So I'm not saying that you should, or I'm also not judging, like if you're screaming at the machines, like whatever, like if it's a toaster and it burns your toast, like maybe it does good to scream at it, but it's like, you're using the toaster to make toast. It's like not the toaster's problem that the toast burnt. Oh, wow. OK. But is there a way for me to also use the tool to kind of have it test itself? Or does that fall short because it doesn't actually know what they expected? Because I remember back in the days, we used, I think it started with Selenium, and then at some point used Puppeteer to kind of remote control a browser from code.

13:03But you would have to write that code so that it would actually go and fill in the registration form, put some products in the cart, and so on and so forth. Can AI help you with that as well? Sure. Like, can you write code this as well? Can it say, like, also test this thing in the browser? I mean, at that point, you really have to have a strong technical understanding of what you want to do, I think. Like, maybe there are ways to kind of go at it naively, but basically for pretty much all of the mainstream AI systems, they have this mechanism for plugins, which is, I think, like MCP and their other tools and skills that you can set up.

13:46I don't really understand all of them. I don't have that much time to spend on not spending time, which, I don't know, kind of weird. But they have this notion of plugins, basically. And there are tons of plugins out there, including things at test pages from a static point of view to understand like, oh, is your HTML actually valid? Is it reasonable HTML? Is it semantically structured? And also including things like browser use agents, where basically it will spin up in the backend something like a Chromium browser and then basically use that as a way to, I don't know, access your pages. So it could do that if you're hosting it locally.

14:35Like if you have something like a, when you set up your site and you have it hosted on a local port kind of thing. Ah, like local server, basically. Local server. Yeah. Yeah. So it can do that. It can also access the live web page. And a lot of this, of course, depends on how you set things up. So I basically just have a Linux computer at home that I can access where I have my stuff on it. And on there, I have kind of like this Chromium agent set up so that it can use Chromium to access the pages. And basically, because it uses the AI system to access your pages, is it doesn't have to know the exact IDs of the elements to click on.

15:22It doesn't have to know the exact coordinates to click. Those kind of things, it can basically be controlled by text. So one of the things, for example, that I did is we, it was like the apartment where we live, they set up a weird rental portal website kind of thing with, I don't know, this fancy React app, which apparently is kind of like, this is how things are done now. You don't get mail anymore. You have to find your PDFs on a website somewhere. And it's like a really annoying thing. And I don't know who coded it. Maybe it's also Vibe coded. That would be weird. But I basically used an AI system to access that website using Chromium and to click through.

16:09And it's like, if there is a cookie banner, just click accept. except when it has a list of files, you should check the files for new files and download any which are new. And you don't have to go in and be like, oh, cookie banner has this ID and you have to click on this button with this ID and you have to send it a click event or like a post or whatever. You basically just tell it like, if there's a cookie banner, just like click OK and get back to work kind of thing. And that, I think, is really fascinating, where you can do kind of these complex things that, in the past, you could also do.

16:50But now you don't have to spend as much time focusing on all the details. Like, for me, figuring out how to run Chromium and click on a specific element, like, that would probably take a day. I don't know. It's like, I haven't used it in a long time. I don't have that much experience with, what is it, JavaScript, TypeScript. I don't even know the language. That's so awkward. But I can go and tell it roughly what I want. And it can create some code that does that. Obviously, taking this code and creating a version to sell to people is a completely different story. But for me, for my personal use, it works fine.

17:34Interesting. But you would argue that it's not like you don't have to know any technology. You still have to know roughly what you want, and you have to know enough to be inspecting what's going on. I don't know. I think maybe that's also just a bias that I have, which is hard for me to evaluate. Because I basically understand the command line tools. I understand the basic infrastructure. And if something goes wrong, I'm like, oh, you stupid machine. You clicked instead of swiped kind of thing. And you did this. And it's like, you did this in Chromium. Can't you try it in Firefox instead kind of thing?

18:16Where I basically tell it specifically what I want. But how much of that would just work if you told it vaguely and it's like, hey, this didn't work. Try again. I don't know. I think if you're selling a service and you have things like user data in a database, then you probably want someone who understands what they're doing. Yeah. But if you're just making something for yourself, if you're making a static site for trying things out or for a friend that just wants a website, like, that seems like the low risk kind of situation where, like, make something a little bit more fancy. but it's not like, I don't know, something will break or burn down if you do it wrong.

18:59Yeah, for me, I wonder how much bias is in there as well because, of course, I'm a software engineer. I kind of like to write my own stuff. But I tried specifically to use AI for situations where I was bored enough to not, like it's a thing I need to do, but I really don't want to have to spend too much thought and coding on it. so I tried to kind of counteract this bias by doing projects where I was like I need this thing but I don't want to spend time on it and then kind of want to like offload the time and at first I was very impressed how far I got very very quickly what was a bit tricky for me was that I noticed at some point I got stuck and I'm not sure if it's because I used only one tool I might have actually tried other tools probably And the other thing is, and to be fair, that kind of tells me that my bias wasn't too big here.

19:53I didn't really look at the code much, so I don't know how maintainable it is. Because that's one thing that I was wondering. Yeah, I mean, AI doesn't have a notion of cost of work, right? It can just add code and code and patch and put layers upon layers upon layers. whereas you as someone with experience might go, ah, no, if I do this, if I patch it now this quickly and by adding like this additional layer, then that'll come back later and bite me because I then have to deal with this additional layer. I don't think AI has this notion. I think that's something that definitely happens and you definitely see it if you understand code because you ask it to do something and it's like, well, like I have to do this common thing, but I will just write my own code to do it rather than using your shared library that you set up somewhere or the existing shared function that you already wrote last time.

20:49It's like, oh, you need to sort these elements. Let me write a custom sort function for you. And it's like, well, you can, but this is not what I would recommend if you wanted to work in a group together with other people and you have to maintain your code for the long run. But I find those are also the kind of things where AI can help you to refactor a little bit if you understand what you're doing. Where I notice it, for example, with some home automation things that I do. I use Home Assistant, which uses primarily YAML files for configuration. Like it also has this whole web UI kind of thing, but it uses YAML files.

21:31And because of that, there are a whole bunch of GitHub repositories with all of these YAML files. It understands, like the AI system has a lot of training material to work on. And it basically can generate these YAML files for you. And I took a bunch of the things that I created over the years and basically told it to like, hey, does this still make sense? Is this kind of like modern use of the functions kind of thing? and for some of them it really came back with some good ideas on how to do things differently so i found that kind of useful that is indeed nice so you set up your site or i guess your page or your tool with ai studio tool yeah web app and did you end up publishing it or Or no, that's when I hit the, so I don't know why, but I was basically using, I was like, okay, it's working now-ish.

22:29I want to click on publish. And then it was like, okay, so now let's integrate with Google Cloud. And Google Cloud is a great platform, except that in this case, I was like, ah, spinning up a project and then like dealing with the billing and all that stuff. So Google Cloud is great for bigger projects, but this was basically just HTML, JavaScript and CSS. so I would have loved to see a range of options where I could publish. What I didn't think about at the time was I could have exported all this thing as a zip file and then just deploy it separately. But then I have a lot of manual work again and then kind of...

23:03Yeah, no manual work. Yeah. So the things I've started creating, I guess, like mostly test websites for me to try things out, to see how requests are made from Googlebot with regards to all of the back and forth stuff there. Most of them I created using Hugo as a static site generator. Hugo, I think it's Hugo. Hugo, oh God. Now I don't know how to pronounce trivial names. Basically, I had it set up the whole setup for me and I published it to Firebase Hosting. which has this free tier kind of setup thing it also connects to cloud functions and has some basic database backend kind of things that you can use and basically I had it create all of this website and tried it out locally and then published it to Firebase Hosting and it basically told me what I need to do is get the key here put it in this file kind of thing and trust that it doesn't steal your key and hallucinate it for other people.

24:21I don't know. It seemed pretty safe. And I had it set up to use GitHub as kind of like the storage place. I think something like GitHub makes a lot of sense for all of this stuff because you see all of the previous versions, which is something that I'm super paranoid about with all of the Vibe coding. it's like I tell it to do something and it's like hey I deleted everything and started over and then if you don't have a backup of what the previous state was it's like well it sucks to be you yeah maybe it works maybe it doesn't so that basically worked out I think fairly well for me and then it also set up the github actions which basically mean that whenever I make a change in in the GitHub repository where it stores all the files, it automatically publishes it to Firebase Hosting.

25:19And this whole process kind of worked really smoothly. With regards to SEO, the main things I looked at there where it's like, are these pages kind of reasonable HTML? And they were. And do they link out properly? And they do. So all of that, I think, mostly worked out. Oh, that's nice. Yeah. What tools did you try so far? I initially started with something in VS Code. I think it's Copilot. Like they have some plugins for different AI systems. But then recently I switched to the command line tools like Cloud Code and Gemini CLI. Oh, okay. What I find interesting with those command line tools is that you basically don't look at your code anymore.

26:11You look at it on GitHub, kind of see what happened there. And if there are variables that you need to edit, setting files, things like that, you do that manually. But otherwise, you interact with it primarily by writing things out. And I do that by writing things out in a Google Doc and copy and paste them into the CLI system. And then it does that. And I kind of find it almost like a more natural way of interacting. And what I find interesting is both of these systems have a notion of agents that go off and do things, that do things in parallel a little bit, that run things like tools and other functionality in the back end.

26:58So you can tell it like, hey, this is not working. Can you check the logs to see what went wrong? and in the backend, it'll know, oh, it's like you're using Firebase hosting, the logs are hosted here, the project ID, you told me that already. And then it goes off and uses the command line tools to look at the server logs and figures out what it needs to do. So all of this kind of back and forth functionality is available or is kind of happens for you automatically. It's a lot less of the kind of the old style use of AI when I use it for coding, it's like you go to ChatGPT or Gemini, it's like, hey, write me a JavaScript snippet that does this.

27:39And then you copy and paste it in your thing. You basically tell the full thing. And it kind of takes care of the whole process. Well, that's quite nice. I think with regards to maybe future-looking things, it feels like people are still deploying Drupal websites. So it's not like these old school. Oh, my God. It's like web is so fresh. But it's like these old school jobs are going away. People will continue to use Drupal and PHP and WordPress and whatever. but it feels like a lot of the simpler things are easily possible with this kind of vibe-coded setup. And especially if you know what you're doing or if you work with someone who knows what they're doing, it's easier to make sure that it's not like this hallucinated framework kind of thing that looks like a website, but it's actually just a giant GIF.

28:37Like, could also happen. Or a PDF kind of thing. like with all of the tooling and testing stuff around it i think it's possible to get reasonably far with this setup god i feel like i could talk about this forever good i'm so sorry for all of the listeners keep going i learned things here what i found interesting is also because uh gemini MyCLI and Cloud Code, they basically use a terminal window for interactions. You can just SSH into your website from anywhere and basically tell your system to update the website. And that means you can run an SSH terminal on your phone, and you basically can use the voice to text mode to enter stuff.

29:30And you tell it what to do, and it'll update your website like that. I used this recently on a flight to the US. Like there was Wi-Fi and it was like, Wi-Fi is kind of spotty and doing something with like a remote desktop kind of environment was like super annoying and slow and flaky. But in a terminal window, it was like, doesn't use a lot of bandwidth. You can do reasonable things and try things out. That's true. So I don't know. Like, I also saw some photos of this developer office where people just have microphones and are whispering to your computer kind of thing, which I don't know. That feels kind of weird.

30:12But it's definitely an interesting way of coding when you don't have to think so much about, like, what is the exact parameter order for this function in JavaScript that I think exists? Or maybe it's in Python. I don't know. where you basically tell it to add functionality. And then you think about it a little bit more as what do I want my website to do rather than like, oh, it's like, which functions do I need to write? True. And the other thing that I'm seeing is, especially from the experiences that I've seen and what you just said, whenever you are starting a project and agencies tend to do that a lot of, like a new customer comes or a customer comes to the new project, you have this setup work even if you just set up drupal or whatever and you might have some templates but then each project is oh yeah that template fits but then we need to make these seven different changes and that just takes time and is not creative interesting work it's tedious most of the time and if ai can do that and walk you through that process quite quickly then that would be really really helpful i think you could probably do this in minutes instead of hours and then when And it becomes interesting where the more subtle things happen.

31:28I think that's where human time becomes valuable again. I think that's, yeah. Yeah. I think for the Search Central audience, being able to spin up websites quickly to try things out is super helpful. Making your own web-based tools is super easy to do. If you want to see if your website is crawlable, you just have it create a crawler and you try it out. these things are super easy to do with regards to agentic access where like I don't know maybe it goes off and accesses search console using the API kind of thing like I don't know how that will evolve or any of the it's like oh look at Google Trends and then write me some articles about the top trends that just feels like spam at that point yeah but there's a lot of work to even just generate like a mock of a website or a mock of certain pages.