When millions of AI agents meet

0:00Welcome back to Google DeepMind, the podcast. us. Now, not very long ago, an AI assistant essentially meant a large language model. You ask it a question, it gave you an answer, but it couldn't go off and perform tasks on your behalf. All of that is changing with the advent of AI agents. While Google DeepMind has this long history of developing agents stretching back to reinforcement learning in games, for most of us, they hadn't really arrived. And then we saw open source tools like OpenClaw released into the wild. And at Google, a new generation of agentic tools is here, including Gemini Spark and Antigravity.

0:40But what happens when millions of AI agents are not just working for us, but transacting, negotiating, delegating to each other? Do we end up with a new kind of economy, a new route to AGI? And how on earth do we keep all of that safe? Well, one of the people trying to answer these questions is Nenad Tomashev, Senior Staff Research Scientist at Google DeepMind. Nenad, thank you so much for joining me. Very happy to be here. I think we should probably start at the beginning here, because for people who have only played around with large language models, could you describe to us the difference between that experience and acting with an agent?

1:18Yeah, no, definitely. I think this is becoming one of the main trends we're seeing this year. And it's interesting because agents are not a new concept. It's something that we've been looking at in the context of AI for a long time, Even before large language models, we had agents operating in simulated 3D environments, going on, collecting items, completing some tasks. This was back in the days we were really prioritizing actioning in the world as a way of manifesting intelligence. Now, similarly nowadays, I guess you could say that the main conceptual difference between just a language model and an agent is that an agent observes a state of the world and performs an action, makes an action in the world, in the environment that it's given, whereas a language model just gives you a continuation reply to prompt to query.

2:05Now, obviously, agents that we use nowadays, they use large language models under the hood, so the two concepts are not completely disambiguated. It still is the large language models formulating the actions. It's just that there is a harness around it made to enact the changes once they have been proposed. But it has a lot more autonomy to chain decisions together, I guess. Correct. And I guess this is ultimately the motivation, right? Because you could do everything, most things that an agent can do manually, painstakingly, by interacting with the language model very many times and you guiding the whole process, whereas an agent instantiates this harness that automates some of that away and gives you less work and gives the language model or the agent more autonomy to complete tasks.

2:53So if you want something done that takes multiple steps, that agent can make a plan and take actions on all of those steps, obviously requiring approval or human input for those actions that are, you know, let's say more sensitive or more likely to go wrong. How is it different though? I mean, if you're used to interacting with a large language model by now, what would it be like interacting with an agent? In many ways similar. Your interaction interface is somewhat similar. You're still talking to the agent in a way in which you'll be talking to a language model. There is a language model there.

3:27But because the agent is doing more things for you, you're more in a position of a decision maker to review and approve. And then once you've approved, the agent is going to do various things, purchase tickets, message your friends if you're organizing a party. and meanwhile you can put something on on Netflix hopefully and relax a little bit. The example I was thinking of was if you were, I don't know, planning a wedding, for instance, you'd go into a large language model and it would like tell you a list of caterers, give you a suggested list of venues, but actually you would have to do all the emailing yourself.

4:00But an agent, I mean, would be much more useful really in that kind of scenario. 100%, especially because agents are given access to all of these tools. So you could, You don't have to. You could give an agent access to your Gmail and give it permissions to send out an email. Of course, there is a chance of it sending something wrong, so you need to verify what it has composed. But in principle, by giving access to tools to agents, you just empower your large language model to do these things for you. And then the whole job is done. The organization has happened without you having to lift a finger.

4:34Ideally, presuming no mistakes have been made, but yes. Yeah, ideally is quite an important point there. So, okay, where we are right now, what tasks are agents actually good at? I think that where we are focusing a lot of our energy on, and we, I don't mean we as Google, we as the entire field, is on coding capabilities of agents. And this is just because so many formal processes and tasks can be formulated as software or as code. In terms of where they're currently at in the real world, Speaking of coding, we see lots of coding tools get used. We use them here internally, people use them externally, and it's really accelerating the development of software, which is bringing the human focus onto ideas and the design rather than the painstaking implementation of boilerplate around them, which used to take a lot of time and a lot of skill and very bespoke knowledge.

5:25And now that can just be done by the language models easily. But then at the same time, we are still at a stage where you have to keep a human in the loop throughout this. I mean, why? What can't these things do at the moment that means that it requires human oversight? I wouldn't even make a distinction between whether they can or cannot. It's more that every single thing that they can do, they don't do with 100 % accuracy. So every action, like with humans at the end of the day, has a certain failure rate. And the more complex the action, the higher the expected failure rate. Again, like with any form of intelligence, human one included.

6:06So while you may expect that an agent will execute the task correctly, it may still make a mistake. And this mistake may be obvious or it may be very subtle, which is actually an important point because there is this thing that has existed in other domains as well for a long time where different machine learning models have been deployed and that is automation bias where in this context, if you're using an agent, it does well, it builds one thing well, it builds a second thing well, eventually you switch off. You start trusting it too much and you fail to verify and you fail to find some important issue underneath.

6:47Then mistakes slip through. Exactly. So for humans, it's important not only to be in the loop because we are obviously designing these harnesses to keep humans in the loop, but to really be engaged and be switched on because as soon as you switch off, you're rolling the dice. So okay in the long term then I mean it sort of sounds like we're in this transition period where these things are sort of becoming more capable but in the long term I mean how much of a difference do you think that this is going to make? I mean will this completely transform the way that we use artificial intelligence? 100 % I think it's impossible to envision a world where there isn't some kind of a deep disruption and what we're all trying to figure out is exactly what that is going to look like.

7:27Obviously we have agency in that we are building the technology, we can design our solutions in a particular way, obviously to empower human developers and human experts across different fields as much as possible. But AI is definitely entering various fields where it just wasn't present before. Scientists are using AI on a regular basis. Up until very recently, mathematicians couldn't envision AI doing something in mathematics. Now it is becoming commonplace in a very short span of time, which is not to say that all of the problems have been solved. Obviously, there is still a big role for humans, but it's a very rapid transition.

8:02And that is the only unsettling part, I guess, because for most, even industrial revolutions and so on, we're used to taking some period of time, giving us more time to change our approach and settle into it, as you say. And it doesn't feel like the window of time is as long this time, so we need to be very mindful of how we approach everything. Why do we want these things? I mean, why build them? What's the benefit? What are they giving us that we don't currently have? I mean, for all of us who have been working on AI for a long time, we've had some version of the answer to that question, I guess, internalised.

8:39And for me personally, the answer is to advance science, improve health and human welfare. Now, these are very high-level answers, so it's maybe not as obvious as to how they map onto the specifics of the question as to why build agents and have agents. And there are people in the field that say specifically that we shouldn't be granting the system's autonomy, which is what agents have. But in my mind, if we can develop these harnesses and make them safe and have agents perform complex tasks autonomously, then we actually accelerate progress. Because then more things can happen with the same amount of human input.

9:24Just draw the line for me to science here, because I guess the examples that we've been talking about have been like, you know, building software, buying stuff for a wedding. They all sort of feel quite trivial. But just explain to me how this fits into the story of improving science. Yeah, so this is my main dream, main objective here. When it comes to science, it's not merely about having some good ideas and reasoning about them for some short period of time, like in a context window of a model. Lots of people are obviously using language models in science as co-ideators or to help with some formal derivations.

10:03All of this is already useful and actually amazing that it is possible. But when it comes to automating science, to some larger extent, there are other threads that are currently progressing at some pace. There are investments in the development of some autonomous research laboratories, for example. And under those scenarios, you would want to see agents be able to schedule experiments to run. Needless to say, lots of safeguards need to exist when such an interface with the real world is happening. Whether we're talking about material design or biotech, because even with, let's say, you're designing batteries, maybe you come up with a setup that overheats, leads to some sort of an experimental breakdown that would damage the hardware, have some consequences.

10:50So we need to have safeguards in place, and we need to have good, reliable protocols in place for these agents to close the loop. Because in the software, closing the loop is, as mentioned, easy. You write tests, and you verify through tests, and then you can proceed. In science, you need to run physical experiments in most areas of science to give you this feedback, whether your idea was good or not, observe that, analyze that, and so on and so forth. Because I guess this is the point, right? If the algorithm, if the agent has autonomy to go and test out different mathematical problems, for instance, rather than just waiting to be prompted by a human, I mean, that does then raise the question of where is the role for the human in all of this?

11:30Indeed, I mean, in the long run, we need to figure that out. I would say that in the short term, with the technology that we have, there is still obviously a major role for humans and our systems, they're not yet AGI. There are many things they still can't do. And I think for the current generation of systems, one thing that can be said with some confidence is that they tend to be good at, how to put it best, let's say, a kind of a combinatorial closure of what we already know how to do. They are, at the end of the day, mostly trained on human data. Therefore, they can replicate the skills we have and repeat them and combine them and find ways of bridging some smaller gaps.

12:08But we've not yet seen these models be truly deeply transformative, let's say in terms of science, making a discovery that no human would have ever thought of. Therefore, there is still plenty of role to play for all of us in this transformation. You mentioned a moment ago that people have been talking about agents for a really long time. Why has it taken so long for them to come into fruition? I mean, really, it's only very, very, very recently that people have actually been able to get their hands on them and play around with them. Yeah, I would say, obviously, some things that we would refer to as agents historically have been deployed, for example, in optimizing operations in data centers and so on and so forth.

12:47They have obviously been very limited because they didn't include language. So there was no way for humans to interface with them, to communicate. It would be a very narrow agent to train on a specific task, and it would be good at doing that task. But because there is no interactivity, there is nothing for us to do with just software in a classical sense. Maybe you can call some of the trading algorithms and investment algorithms also agents in that context, but they just operate on their own. The difference now is because these agents are based on language models, is we can talk to them, we can learn from them, we can influence them, we can steer them, and this is why all of us as people are interacting with agents much more.

13:31But then why are we still waiting? I mean, this sort of vision that you're describing of like an assistant that can just go off and do everything for me, it's still not here. What's stopping it being deployed more broadly? We need to take a step away from just designing the underlying model. A lot of energy has gone into that and there are still improvements needing to be made. But now that we have capable agents, capable models, we need to find better ways of coordinating them, orchestrating them, managing them. Once you have these admittedly quite powerful assistants that can do many things for us, we need to see ourselves as managers of teams and institutions in some way and to develop personal management skills to handle these workflows.

14:14Managing a team of agents is different compared to managing a team of humans, but they're obviously commonalities, right? Different in a sense that agents will make very non-human mistakes. They're not a human intelligence. But at the same time, an agent doesn't know you that deeply to be able to just go on and accurately guess everything you'd want it to do, still need to be involved. And therefore, we need to get better at orchestration, I think. Thing is, we're still in a world where large language models occasionally hallucinate. So it Is in some ways quite a big leap for humans to then trust agents to carry out tasks on their behalf when any hallucination might actually result in something catastrophic happening?

15:00Trust is given, but it's also earned. I think this is maybe an important distinction. So in our frameworks, we mentioned the need for establishing, let's say, tracking of reputation over time, where if an agent is repeatedly unreliable, it should obviously not be trusted. Even if it's mostly reliable, it shouldn't be blindly trusted. We should still verify its actions. But language models will always hallucinate to some extent. So we just need to integrate them in our workflows in a way which recognizes that and where we make sure that those hallucinations, and they're becoming more and more rare and hopefully will continue to do so, don't compromise the workflows that are being undertaken.

15:46I know one of the things you've written a lot about is the idea of delegation, that you might have a particular task and an agent might then go on to delegate it to a specialist. Just explain to me how that might work. Yeah, so this is the idea that one of the bottlenecks, one that we haven't mentioned yet, is that where we would really like to get help from agents are very complex tasks. So what language models and simple agents that many of us have access to can easily do is if you give a very direct instruction. Go book something for me. I want to eat at this restaurant tomorrow. Find a slot and do the booking and the agent can maybe do this via tools.

16:25If, however, you have a very complex plan that needs to be broken down into pieces, executed separately, you may be in a situation where even no individual agent can do each and every piece. So maybe an agent may need to, over this established agent-to-agent protocol that exists, hand off a part of that work to another agent. But then there can be failures along the way. So an agent that delegates or a human that delegates needs to manage and handle those failures and also preempt them as much as possible. Preempting them may involve figuring out which agents are first and foremost reliable to even delegate to in the first place.

17:06What are their capabilities? Is that something that we can certify? And also to safeguard the users and the agents from any kind of a malicious interaction. You mentioned, I think, was it a wedding or a party or something at first as an example, right? So when managing a big event, some of the bookings fall through, some accidents happen, some things don't arrive on time. So whenever you have a big coordination challenge, there are lots of things that go wrong. And in the process of managing that as a human, you need to deal with all of those delays and problems. And similarly, an agent that delegates to a group of agents needs to manage all of the problems that may arise.

17:46So one thing that's currently the case in many of the multi-agent systems that we see is that they act more as parallelization than delegation, where you may have many agents working on things, but rather than there being an intelligent framework around how the work is split up. It has to just be chunked into sort of random sub parts that are handed off. They get done in parallel. So you get a speed up presuming that all this is reliable and each agent can complete its task independently. But this is not the intelligent delegation framework that we talk about. So if the tasks are split up in a sort of essentially random way, you could have one agent that is buying the wine and another one that is buying glasses and doesn't realize that it's wine glasses that are required, there's sort of no communication between them.

18:37Is that the kind of potential problem that could arise? Potentially, but you're also, I guess, hitting on another point, which is that many of the uses we see are uses in, again, software engineering, for example, with agents at the moment. And that is a part of the reason, because in software, when you're building software, you can write tests, unit tests, as we say, right? And run them and verify that the code that has been written, at least in isolation, performs the function. But when it comes to many of these real-world tasks, verification is not necessarily as straightforward. Maybe there is a subjective element involved.

19:11How do you define nice-tasting wine, for instance? Maybe, yes. A bit of a subjectivity in that. But this is actually quite important when it comes to AI and language models because there is a notion of reward hacking that has existed in various contexts in the field for a while. So there could be situations where it does something that meets the request but isn't in the spirit of the request, technically. And for that reason, you really want to emphasize verifiability and to be very formal about a contract that's made between the delegator and the delegatee in that setup. At the same time, for tasks, we need to recognize that some are completely reversible.

19:48So if something goes wrong, there's no harm. You just rerun the task, retry, redelegate. Some may have consequences in the real world, whether it's spending your money to buy something or taking some other action that you can't easily revoke after the fact. So for those tasks, you want to put more care in what you do there. We've also seen with some of the early agents that are out there, agents delegating tasks to humans, right? Sure. Just talk me through some of that. I mean, that's an interesting, let's say, reversal of the more usual vision that we all have. So humans delegating tasks to AI, that's quite standard.

20:31But this other direction has been explored across a number of studies, I say. And my background for context is that I've done lots of prior work in and around medical AI. In medicine, we've had narrow systems that were at basically superhuman performance for very specific things that they have been trained to do. In medical imaging, in radiology, this had to do with a machine learning model, seeing a scan, identifying where there is a pathology, putting a box around it and handing that off to, let's say, a human radiologist to review. And these systems have been operating at a very high level for quite a number of years.

21:11They still have some failures, though, so they need to be reviewed by human experts. So people have experimented with AI human teams there where the idea is that a human would correct a mistake made by a system. And people have trialed with this flowing in both directions. Either having a human expert only consult an AI when a human expert is, let's say, uncertain, or a human expert look at AI's suggestions and outputs all the time, or maybe having an AI system do its thing, make the predictions, and then a flag when something is uncertain, when maybe there's something blurry, fuzzy in the image that can be interpreted in many ways and the machine learning system isn't sure which of those is correct.

21:56But this human review of decisions made by these potentially superhuman narrow machine learning models has proven to be quite a good setup. So that AI would defer to a human in case of need, in case of uncertainty. That is interesting, though. that, I mean, granted, in those very specific scenarios where the AI is superhuman in its abilities, that the best team that you can get is where essentially the AI delegates to the human when it's unsure. That is fascinating in and of itself. And, you know, maybe there are use cases where it's the converse. Now, for these more general systems, again, if an AI can recognize when it needs approvals and permissions for sensitive actions, then it does make sense to delegate those decisions to humans, at the very least, right?

22:45Just looking at the other side of this, I also want to think about the sort of cybersecurity element of this. Because as more and more agents are out there interacting in the world on the internet and so on, there are inevitably going to be people who are trying to exploit the vulnerabilities of agents. Tell me a little bit about agentic traps that people are laying. This is a scary and a fascinating topic at the same time, I would say. And I think it's one of the main reasons why these kinds of deployments at scale cannot work right. Because as we said, if there is not complete reliability of individual interactions, any system at scale that has many interactions is naturally going to statistically fail.

23:27And because these systems take a lot of compute and therefore energy and money to run, if they're not reliable, it's just a non-starter. And agentic traps are something that we have been thinking about for quite a while now. They can manifest in different ways. There are many types of traps. But it boils down to agents operate within an environment. And in this context, the environment is the web. If the environment itself is poisoned, if the traps are laid, agents may stumble upon them when interacting with the web. and then yes malicious people or malicious agents deployed by malicious people can place those traps and then compromise systems really.

24:11So I don't know the sort of the wine buying agent for the wedding goes on to a particular wine merchant where there is some essentially a prompt injector in the website that changes the agent's goals. Is that the sort of thing that we're talking about That is one way this could happen, yes. And the reason why that may potentially go unnoticed is, in terms of how web pages are encoded, there are elements there that are just not rendered visually. So if we're talking about an agent that isn't a visual computer use agent that sees the web page, I mean, the pixels the same way a human does, rather consumes the actual format of the page in its raw format, that, then it could inadvertently consume those hidden tokens that can make it do different things than what the intention was.

25:02But this is not the only way it may happen, because what malicious websites could potentially do, they could do what we refer to as dynamic cloaking as well, where they display pages differently for humans and agents. Because you can, based on the behavior on a page, make a very good guess as to whether it is a human or it is an agent interacting with the page. And then only if an agent is interacting with the page with a specific intent, do you tweak the content in such a way so as to induce some kind of jailbreaking. But just kind of going a little bit further on this, you could have agentic traps out there that, I don't know, are designed to sort of take money from you to do all kinds of things.

25:44Yes, and this has happened to people who have experimented with agents and have We've given them access to wallets, right, to do things. As you say, in the early days of this whole, when we are especially experimenting internally or anyone else is, this is done in a trusted environment. So you don't necessarily, in your early prototyping, have to deal with any of this. It's not in the wild. Yes, but once you deploy on the web, especially now with AI really being used in all sorts of places, the more agents there are, the more incentives there are for malicious people to do malicious things because they have a higher surface area to target.

26:22And I think we're at the point where even the most of the web is currently being generated by agents and consumed by agents, that the agentic use of the web is exceeding that of humans, which is maybe happening for the first time. Okay, two things. First of all, it sort of sounds like you're describing that we're entering into this phase where there's like two different forms of the web. the sort of human version and the agentic version with dynamic cloaking and so on, a sort of a version of the web where adverts don't mean anything anymore. You know, it's not sort of human eyeballs that you can possibly sell to.

26:57But I think that the second point about this is how on earth do you mitigate against it? If you don't have control over the environment, which you don't over the web, how on earth do you protect your agent from going rogue? In some sense, it's not a new problem, right? Because the security of the web has in other ways been an issue before. And computer viruses could spread if you open the wrong attachment in your inbox, right? Or you click on something on an untrusted page. So it's not the first time we're experiencing the need to certify the resources we're interacting with. When it comes to machine learning systems, let's say adversarial examples have existed for a long time where imperceptible changes in images, imperceptible to humans can jailbreak models.