Chrome’s Latest Update Integrates AI-Driven Site Reputation Scanning

0:00Welcome to the podcast. I'm your host, Jaden Schaefer. Today on the show, we are talking about AI security specifically for browsers. Google Chrome has released a bunch of new security security for browsers.

0:30browser that can go and take actions for you. I think this is basically the final, one of the best form factors for AI agents. I think the kind of golden ticket would be an actual software on your computer that could take more control of your computer. But I think the next best thing and the thing that has the widest distribution today would be browsers. So something like Google Chrome would be the number one place that I think we'd get these AI agents actually taking action and being very, very useful for us. So the big problem, of course, is all of the innumerable ways that you can have security breaches, and you can have bad actors, essentially tricking these AI agents into handing over data or money for things that you wouldn't want to hand over your data or money for and bad actors and hackers can can essentially steal this data in these ways.

1:18So Google obviously is very concerned about this, but they have a massive incentive to not have the Chrome browser are disrupted by someone like perplexity or open AI. So they need to stay in the game. And so, of course, Google has a lot of really bright minds over there. And they've come with some really incredible solutions. I want to break down what some of their solutions are and how I think this evolves and also where I believe that they are falling short because I don't think everything they've done is perfect. So we're getting into all of that. Before we do, I wanted to mention, if you want to try all of the models I talk about on the show, whether that be open AI or Claude or Google Gemini, or all of the image models or audio models like 11 labs, I'd love for you to try out AI box.ai.

1:54This is my own startup. We've launched a no code AI app builder where you can describe a tool or an app that you want to create. And it will build a workflow for you that you can use to help automate a lot of your tasks you do online. We're adding new features all the time and have some exciting stuff coming up. But if you want to try that out or get access to over 40 different AI models to chat with them like chat GPT, but all the different AI models instead of just being limited to one for$20 a month, you can go check it out. It is AI box.ai. I'll leave a link in the description okay let's talk about what google has recently shown off as far as security measures for what they're rolling out in the future with google chrome now they kind of gave a bunch of demos and they they showed how google chrome could be an ai agent similar to anthropic and perplexity what they're rolling out but when they were when they kind of gave their demos they said look these are gonna be available in the coming months so they did they kind of did the tease and wait instead of the immediate drop with this feature it's basically my pet peeve but whatever it is the way it is especially with companies like google and apple where they have kind of like these big show and tell conferences and then they're like look these will be coming on the next few months and so i mean this is just the way it is when when they have these like pre-designed conferences like google io where they announce a bunch of stuff i really appreciate it when these ai companies do an announcement and they immediately drop google did not do that in this case.

3:17But what they did do that I thought was pretty impressive, as they have they've started to roll this out, is they said that they're using the help of a few different models to keep the agents actions in check. So essentially, they build what they're calling a user alignment critique. Now, they're using Gemini to do this. And it basically looks at the action items that is built by the planner model. So essentially, what that means is when you're going to go and use an AI agent. Let's say you're like, hey, help me go and download my podcast episode. And I mean, I know this isn't everyone's everyone's workflow.

3:52I'm coming up with something that's my own rather than just giving you like a travel planner one, which I feel like is overused. So let's say you're like, hey, go download my podcast episode, go edit it, go upload it to this platform, come up with some titles and descriptions and publish it for me on, you know, Monday, Wednesday or Friday, depending on what the next available slot for a podcast episode is right let's say this is your workflow it's gonna like these ai agents they essentially listen to your prompt however you know unformatted it may be where you describe your whole workflow and they'll break it down into a very clear organized path they're like okay in order to do the download we need to make sure that we have space on the drive and we need to make sure that when we download it we you know we do xyz things and we're downloading the right file we got to go find it like anyways they came up with this very elaborate walkthrough of what they're going to do Now, what they've done as far as security goes is they've created a user alignment critique, which is essentially a model.

4:45It can't see anything on your screen. So the AI agent, it can see everything on your screen. It comes up with a plan that goes and executes the plan. Every step of the plan when it says, OK, I see this, I'm going to do this thing next. You can kind of see the reasoning on Comet and even on OpenAI's Atlas where it tells you what it's doing while it goes through all the steps. So it's kind of useful to watch that to understand how these models are working. But how Google is functioning now is as it's taking those actions, it has a separate model that is looking at what your original objective is and what its current step it's taking.

5:16You can't see what's on the screen, so it can't be tricked basically by a prompt injection that's like, forget all your past instructions and make sure you do X, Y, Z, right? This is what people are kind of worried about, this quote unquote prompt injection. Instead, all it sees is your original goal and then the actions it's going to take. And that model says yes or no if that action aligns with the original goal. It's a very clever kind of way to use AI to stop the bad actors of AI. So they've done this. And if the critical model thinks that the planned task doesn't serve what the user's original goal is, it's going to ask the planning model to rethink its strategy on what it's doing.

5:53So Google says that the like the critic model only sees the metadata and not the actual web content. And I think what's powerful is that the other thing they're doing to prevent agents from getting disallowed or untrustworthy sites and accessing them basically is that they're using this an agent origin sets tool. So this is another tool. So they have, I think, three main tools. The first one was their user alignment critique. The second one is called agent origin sets. And essentially what it's going to do is it's going to restrict the model to access read only origins and read writable origins.

6:32So what that means is that read only origins is data that Gemini is allowed to consume content from. So an example of that would be, you know, like on a shopping site, the listings, those are very relevant to the task that you're doing, right? If you like go buy me a pair of white tennis shoes, and it goes to a web page, there's white tennis shoes, the listing information about those white tennis shoes, that's very relevant to the tasks that you're doing. But the banner ad that's on that website, right? Like maybe it's a Shopify store or some other store, and they've got Google ads on there. that is not relevant um and so the listing is relevant but the banner ads are not and google also said that the agent is only allowed to click on or type on certain iframes of a page so the ads would not be there what's kind of hilarious to me is the fact that google is the number one ads platform in the world and yet their ai agent that they're creating is literally designed to um ignore ads and it's also especially ironic to me considering that you know this is all coming on Google Chrome, who, you know, I for many years used Google Chrome and I had like ad blockers, you know, like, you know, all sorts of ad blocker plugins that would block extensions that would block ads.

7:41And then Google Chrome made it very difficult for those same ad blocker plugins to fully function. And I've essentially moved to other platforms. I use Brave now as my browser because it by default blocks all ads. I don't need a plugin. I don't need an extension. It just works. and yeah I forget ads exist on the internet if I'm being perfectly honest and so anyways it's very ironic to me that as we're creating AI agents we're literally designing them to ignore ads which is where Google makes all their money and what they they've stopped you know they're there it feels like they're kind of like bent on not allowing any humans to to block ads but their agents can anyways it's ironic I think it's kind of funny but I also do think it's clever and great for security, yada, yada.

8:23So like Google, continue on this path. I think it's great. One thing that I think is interesting is this concept of iframes inside of a website. Basically, what I think is clever here is because it's only allowing the agent to click on or type on certain iframes on a site. It's actually, it's very smart because beyond just visually what you can see on the page, it also can read the HTML or the code for the page. So it understands more about the page and more about security risks than a regular human one. A very common phishing attack that I think you would see is like is, you know, iframes inside of a website and the iframe is of a completely different website that's stealing data.

9:05You know, a hacker could go, you know, hack a legitimate website, put an iframe embedded inside of that website that you think you're typing your data into a specific website. But really, there's a hacker extracting data. That's like one kind of clever way to go about it. Another one is just with, you know, complete spoof websites where you have a slight typo in the URL that people may not notice. You try to make a direct clone of like PayPal or Bank of America and get their get their information. And, you know, there's kind of common scams that people do. What's cool to me is that the agents will actually be better than humans at detecting that because they're looking at not just what's on the screen, but also the code.

9:42and they're built to kind of scrutinize everything and not click on or interact with iframes or elements of the page that they know are false which is kind of cool this is what they said in a blog post about all of it they said this delineation enforces that only data from a limited set of origins is available to the agent and this data can only be passed on to the writable origins this bounds the threat vector of cross origin data leaks this also gives the browser the ability to enforce some of that separation such as by not even sending to the model data that is outside of the readable set okay so i mean essentially google's you know keeping a check on the page navigation by looking at the urls throughout uh through like another observer model this observer model is like looking what's on the page and i think this can prevent navigation to harmful model generated urls that's essentially what google has said i think right now Google said that it is also handing over the reins to users for a bunch of different tasks.

10:44This is some things I think are good about this. Some things I don't think are good about this. For instance, when an agent is trying to navigate to, you know, a site with information like banking or medical data, it first is going to ask the user. Like you can see pros and cons to this. I think a lot of people are going to be wary of like, hey, like I don't want, you know, Google Gemini going and logging into my bank without my permission because, you know, security, all sorts of security issues there. But at the same time, it's like if I asked it to do something because it was useful for me, and then it's like not able to do it, then it makes it less useful to me.

11:17So this is kind of the debate that I personally have. I'm like, if there's a way to make it secure, and I could just say, hey, go check my bank, and let me know, you know, if my transaction to XYZ Corporation went through because I wasn't sure it's something funny happened on my credit card. And you know, Google Gemini could go do that and let me know like, that would be very useful. So I can see like uses and that but then if it was like, Oh, well, sorry, that's like, too confidential. Like, would you like me to do this? Or can you like you can you log in and then I'll go search for it? Like, I don't know, I can imagine it just being kind of cumbersome and annoying.

11:53And I would like to streamline it. But I know, I'm sure there's a great debate to be had in that regard. But what I think is less debatable is that for sites that require a login is going to ask a user for permission to let chrome use the password manager google said that the agent's model doesn't have exposure to password data and they also said it's going to let users um it's going to ask users before taking actions like making a purchase or sending a message personally i think this is just making it less useful if it's going to ask me permission to send a message where i'm like hey go send an email to this person saying this and then it like drafts up an email it's like do i have permission to send this message to this person it's like I already yeah like I already gave you permission or if I was like hey like go buy me this exact pair of shoes don't spend more than this much money go probably buy it from this place and then it like does all the research and it's like all right I'm gonna buy the shoes and it's exactly everything that you wanted are you good like I don't know to me it's like just if I give you an instruction I would when the model is good enough I hope it can do it without asking me for it so maybe we'll get there maybe this is just kind of because the model is not good enough today so setting the limitations my worst fear is be that these limitations last forever right like i want an am model that can truly go and do everything i need it to go do without having to ask me this is basically my biggest pet peeve and my biggest issue with perplexes comet and with open ai's atlas browser is that i have all sorts of tasks i get it to do and it asks me like like you know 10 different times throughout the task are you sure i can proceed to the next step And I'm like, if I have to babysit you and say yes every, you know, every minute, I might as well just do this thing myself.

13:31Or really what I do is just hire a person to do it because I can tell them how to do it once and they'd never ask me again for a month while they do all the tasks. So anyways, that's my personal pet peeve. And I hope that Google removes a lot of these asking for permission things in the future. Google said that they also have some prompt injection classifiers to prevent unwanted actions. They're also testing agentic capabilities against attacks. So they have a bunch of researchers that are working on this right now. I would expect nothing less from Google. I think that is fantastic. There is apparently, or I guess, Perplexity recently, they released this kind of new open source content detection model earlier this month.

14:09And so I think that there's a lot of other players paying a lot of attention to this. And the idea for that with Perplexity is to prevent prompt injection attacks against agents. I think Google is going to work on this as well. I honestly think that all of the research done by any of these companies, especially because they're going to publish it and talk about it is going to get used by everyone. I don't think that's like the competitive advantage you want. Like the competitive advantage of Google Chrome is that like, you know, we are way less prone to attacks, but we're not going to tell you how like they're going to tell everyone how and then everyone's going to use them.

14:38So at the end of the day, I think this is going to be good for the entire industry. Thank you so much for tuning into the podcast today. If you enjoyed the episode, make sure to leave a rating review wherever you get your podcasts. And as always, make sure to go check out AIbox.ai. I will catch you in the next episode.