#214 Ece Kamar: Why AI Agents Are the Next Big Thing in Tech (Microsoft Research)

0:00So when we think about agents, we are thinking about AI based entities that can perceive the world and act in it in a continuous fashion to be able to carry out tasks. Why agents is such an important paradigm shift compared to the models. First, this is really about the value. Why we are building the AI systems at the first place. In my opinion, the biggest question to be addressed right now in our community is what are we doing with these models? What are our goals? Showing demos and just getting some input outputs with text is not the value that we are looking for from the systems that we are building.

0:36We want systems that are aware of their environments and that can do things for us so that as users, we can get the right value. Okay, AJ. Yeah, wonderful to meet you. I'm particularly interested when this came across, you know, everyone's talking about agents. And so I wanted to hear, and I do some ghost writing for Boston Consulting Group, and I just wrote a piece on agentic workflows, and I didn't mention Autogen, but it seems like you're involved in that space, and I thought maybe we could talk about that. And then I saw that you also were involved in evaluating GPT-4 before it was released.

1:29Yes. And I have some questions about that. I have a pet peeve about the TaskRabbit anecdote. I don't know if you know much about it. Which one is that? That according to OpenAI, GPT-4 was asked to do something. and it had to solve a CAPTCHA and it went on TaskRabbit and hired a freelancer. Yes, yes, and I know about that, yes. And that anecdote gets repeated everywhere. I just saw Yuval Hariri is out promoting his new book and he's been telling that anecdote. And, I mean, you and I know that GPT-4 did not, you know, on its own log on to TaskRabbit, TaskRabbit, log on to TaskRabbit, you know, set up a job, talk to a, you know, it's, I hate stuff like that.

2:34I mean, just be honest about what happened. It's, it's interesting. And someday that may be able to work, but that's not what happened. You know, that story is actually relevant to the discussion on agents as well, Craig, because when we think about agents, we are thinking we think about ai based entities that can use tools and that can take actions in the world that's right and that doesn't mean consciousness that doesn't mean like what they are thinking about like this conscious entity discovering things completely brand new in the world at all yeah but we are and i know we are going to get to that conversation but we are increasingly building ai systems that can use tools and a tool can be also going to task rabbit and putting a job posting in there so like i think we need to call it what it is but also be aware of like what that really means not for consciousness not for agi but like really preventing some issues that may happen there soon yeah um yeah no i i i agree it's just um you know people take an anecdote like that and they make it sound it's it's a little bit like uh my other pet peeve is that uh talking head um that ben gertzel runs runs around with uh sophia which you know i am going to destroy all humanity i mean it's if there's someone in the back like typing in this stuff it's not you know here is my real issue with all of this i spent a decade helping to build responsible ai inside microsoft yeah i work with many different product groups building different ai technologies where they were serious risks and there still are and that work is time consuming and takes a lot of effort from people who are involved and they have to a lot of times conquer new territory to be able to do it right.

4:48And that work is already under-resourced compared to all of the AI developments that are happening. If we are not creating, if we are not focusing our attention to the real problems that are happening in the world or what is on the horizon and kind of fooling ourselves with things that that may not show the real state of AI, we are actually not utilizing our thinking and resources properly to be able to address what is most important for society. And this really comes up, you know, when some of the AI leaders write letters and say like, let's pause everything for six months or, you know, kind of make these calls.

5:26That is like, the good thing about these conversations is the impact of AI on society happening now or happening in the future is going to be so important. So we need everybody to think about it. But we need people to think about the right problems, to realize where this technology is, and not distract the conversation to hypotheticals. So that's kind of my pet peeve. And the other thing I'll say about that is, pause is not the solution. You have to invest in the science, in the research, in the development of responsible AI and safety. And that's not going to happen by pausing. It's going to happen by prioritizing those topics as much as you are prioritizing the frontier models.

6:13Yeah, I agree. Yeah, yeah. Okay, so why don't you introduce yourself? I may use some of that stuff. Sure, sure, happy to. But why don't you introduce yourself and then talk about how you got to Microsoft and then we'll talk about agents and agentic workflows sounds good hi Craig thanks for having me at the podcast my name is Ecek Amar I'm a VP of research at Microsoft and I also I'm the managing director of an AI lab called AI Frontiers we started AI Frontiers a year ago with the mission of understanding where AI is headed and pushing the frontiers of AI technologies by always going after the question of what is the most important thing we should be studying at the given time.

7:10So I have a team distributed between Redmond in Washington and New York. And yeah, we have been doing a number of interesting projects, especially aligning our work on agent so i'm excited to talk to you today about those topics yeah what were you doing before you got to microsoft research before i got to microsoft research i was actually a phd student at harvard this is between 2005 and 2010 and i did my phd at a time where these topics especially general ai was not a popular topic at all as you know the ai field has gone through many ups and downs to its history. And during the time I was doing my PhD, my PhD topic was actually human AI collaboration, advised by the wonderful Barbara Gross, who is actually now recognized as being one of the godmothers of AI, because there are actually quite a few women that has been trailblazers in the AI field.

8:15And when I was studying the topic of human AI collaboration, And in fact, my PhD thesis was also on multi-agent systems, multi-agent AI systems. Those topics were not top of mind for industry, and the academia was actually quite scattered in terms of the study of these problems. So when I was doing my PhD and working on these problems, some of the work that was coming from the real world, building systems and trying them out in the world, was coming from Microsoft Research. In fact, Eric Horvitz's team. So as a PhD student, I was looking for opportunities to kind of get my hands dirty, apply these ideas to real world problems.

9:04So I did some internships at Microsoft Research. I became a research fellow. And, you know, it's kind of interesting how you get to, at the place like Microsoft Research, where there's this history of research and there's the freedom to think. You can actually get some ideas before their time. So one of my internships project was actually ride sharing. Before there was an Uber, we were actually imagining like how to bring all of the drivers and the passengers of Microsoft shuttles and cars commuting to the campus every day and how we can gain a lot of benefits from that kind of a marketplace. So that was kind of one of my internship projects.

9:48And then I started at Microsoft Research as a researcher right after grad school. And I got to work on a number of interesting projects over the years and then found really a passion of mine which is making sure that ai is developed responsibly and that became kind of the passion of my first decade at microsoft just just pushing the company around that really important work um and then now my second passion is really bringing the talent in MSR together in this pretty crazy adventure we are having in AI. Yeah. And agentics or agents, AI agents, which are suddenly sort of entering the economy, they're no longer research projects.

10:47I mean, there's certainly there been AI agents before, but these generative AI driven AI agents and then networks of agents and workflows of agents is a pretty exciting topic. And there's a lot of sort of hyperventilation about it, as there is with a lot of these developments. But I can see that indeed, as you are able to get agents working together on whole workflows and then talking to agents in other institutions or domains, that you will build kind of a fabric of AI agents that support human activity. So can you talk a little bit first about the development of agents and the working with multi-agent systems?

12:00How big could those systems be, for example? Definitely. In my lab, the mission we are after is really pushing the frontiers of what AI can do, what AI can do for people, what it can do for society, what it can do for organizations. And all of the work that has been happening on frontier models like GPT-4's clouds of the world, they have been, of course, wonderful to give us a foundation to build the next generation of AI systems with. And all of us have been in this journey, Craig, you and I and many of our listeners that when you actually get your hands into a frontier model, you start doing things with it by prompting it, putting things into its context, and then giving it tasks to do.

12:48And we have been also amazed with, like we have been some of the early adapters of the GPT-4 models before there was a JetGPT in the world and spent a lot of time understanding and prompting it. And soon enough, if you actually play with these models enough, you realize that the way you want to build systems with these models is not only prompting these models. And this is, in fact, a big piece of a new computing paradigm to be built for our world, where foundation models become important processors, but they are not the only thing. In fact, when we are building AI systems, we want these systems to have memory.

13:32We want these systems to be able to remember things. We want these systems to be able to personalize to us, remember things about us. But not only that, we actually want these systems to be able to interact with the world, interact with us, and do things in the world for us. This is where we actually now get the agents. So when we think about agents, we are thinking about AI-based entities that can perceive the world and act in it in a continuous fashion to be able to carry out tasks. Why agents is such an important paradigm shift compared to the models. First, this is really about the value, why we are building the AI systems at the first place.

14:13In my opinion, the biggest question to be addressed right now in our community is, what are we doing with these models? What are our goals? Showing demos and just getting some input outputs with text is not the value that we are looking for from the systems that we are building. We want systems that are aware of their environments and that can do things for us so that as users we can get the right value out of them. And agents are how we are going to be pushing for that value. But second, agents are also how we utilize these really beautiful models for getting a lot of things done. But then there's also a question about how we engineer increasingly more sophisticated and complex AI systems.

15:00And on that one, the reason we actually started, my lab started doing the work on multi-agent systems. Last year we released AutoGen, which is one of the popular libraries in the multi-agent space. is because the multi-agent framework provides a really nice programming paradigm for increasingly more sophisticated AI systems. What I mean by that is imagine you have a difficult task to do. Now, if I'm only using models, I'm not using agents. The way I would program that system would be trying to put all of the instructions, all the context, all of the memory into the context window and say, do it for me.

15:41And you have no idea how the model may or may not be able to do that. But what you can really do with multi-agent orchestration instead is you can take the task and decompose it into pieces. And then define this task should be done by these agents where each agent specializes in this kind of capability or this kind of knowledge or this kind of expertise. And then once you have that team of agents where there is a particular specialization in them, then you can orchestrate them. And that orchestration can either happen by a human creating a workflow, like how you would create a workflow for human teams.

16:24Or it can happen through having an orchestrator agent that actually uses LLM-based orchestration and planning to be able to guide the work. or based on our now new work, it can also happen by new approaches that can use the best of those two. But this is so exciting for us because this is where we think AI is going to be really have a big impact on people and the society because it can do things for you. Yeah. And, you know, my question was how large can a network of agents be? I mean, because what we're talking about initially are breaking a task into subtasks and assigning an agent to each subtask.

17:20But is it possible for that workflow to spread and handle a much broader series of tasks? Yeah, that's a great question. This is really about how our understanding of the agents are evolving in time. What I describe you in terms of using multi-agents as a programming paradigm is how we came to the idea of building Autogen in the first place. But I completely agree with you that that's not where we are stopping. And in fact, where we see with many of the customers using Gotogen and our internal teams as well is they are not using multi-agents only as a programming paradigm anymore. They are using multi-agents to represent, we call it a society of agents.

18:14You may call it the network of agents, or we are calling it a society of agents, to actually start thinking about building persistent and composable teams that now become part of the way we work, that now become part of the way an organization is being built, now become a part of a way that consumers may be able to interact with technology. So, you know, coming back to your question about like how big these teams might be, I think it may become as many as an organization's have people or more. And we actually don't see a bounce in that numbers. And we are seeing some really interesting auto-gen use cases right now.

18:57This is coming from some of the sectors that are really interacting with consumers around the world, is that they are actually thinking about building an agent per customer. And for big organizations, this may actually mean millions of agents. Wow. Because now they are thinking about scenarios where agents may actually simulate their consumers. And with this, they may be able to test the systems they are building, or they may be able to simulate some of the consumer scenarios up to a point where the user may take off and then continue the rest of it. And this diversity, the scale we can get to an agent, I think is getting us to really think about what we are building from its core.

19:42Just to give you an example, when we built Autogen, Autogen was built to be like a centralized multi-agent system where there would be like a top agent that tells every other agent what to do through doing a central broadcasting. For the last year, we have been interacting and working with many users of AutoGen around the world, both professional organizations as well as anybody who would like to build something with multi-agents. And after seeing these feedback of how they are using Autogen, that they actually want to have millions of agents interacting, they want to have agents that have persistency, that do not go away after doing a task, but they become part of the organization and capturing like what that organization needs to know about.

20:33We have actually completely redesigned Autogen. Just last week, we have released a new version of Autogen that we are calling Ag Next, where Autogen is completely designed from scratch again, so that we can build it in a way that it can scale to millions of agents, that agents can be persistent and can be discovered in the future towards this vision that we are imagining as a society of agents. Yeah, and I know what Autogen is, but listeners may not. So can you stop and introduce Autogen to listeners? Autogen is a multi-agent orchestration library we've developed. I think we released it exactly a year ago in the October of 2023.

21:20And this was mainly a library to get us and the world, both academic world and the industry, to start experimenting with multi-agents because in our research lab, again, we were doing a bunch of work with models, but then to push the frontier of what you can do with them, we started experimenting with multi-agent ideas. And then we said, okay, we think this is really cool. Can we actually push it to the world and see what people do with it? So we released Autogen as the multi-agent library. And what you can do with Autogen is, Again, it has some functions in it to be able to define tasks, to be able to decompose tasks, to be able to create agents and get agents to collaborate with each other to be able to do tasks.

22:09And people have done wonderful things with Otogen. It has been one of the most exciting adventures that we've been on, just watching the community, the open source community doing things with it. So this feedback between us and the open source community has not only taught us a lot about where Autogen needs to go to kind of become a better library for the diverse set of scenarios people are using it for, but it's also giving us wonderful ideas in terms of what new research problems we push in agentic AI. Yeah, Autogen itself is an open source initiative. Yes, it is. It's an open source project by Microsoft.

22:48but it is it is open to of course many collaborators we have that are contributing to the code um with microsoft yeah uh you mentioned watching some of the exciting things that people are doing i don't know what you're authorized uh to talk about but can you give us some examples that of multi-agent systems that are being built with autogen yeah we can talk about the things that people are freely sharing on discord autogen also has a pretty lively community that loves to share with each other to be able to learn more we are seeing people using it in really large-scale applications to be better understand their consumers and to be able to reflect their consumers inside the applications they are building.

23:43We are seeing some automation work going on, like taking tasks that are very, very time consuming for people that people love, hate doing, and then getting agents to do that. For example, one scenario my team is quite excited about is automating the expense reporting because everybody needs this. There are certain tasks that when you actually say like, I'm going to have an agent or an agent team that's going to do this for you, everybody's eyes light up. One of them is always expense reporting. So those are some of the cases that people have been successfully trying. But the one that I think I'm surprised about, and I would love to see more work happening is large-scale simulation and all the discoveries we might be able to do about the society and science by just doing large-scale simulation of systems and the dynamics and seeing what can happen with that.

24:44So I guess the most interesting part of the AutoGen adventure has been like just seeing the diversity of cases and also seeing how some of our users are taking it so much further than we can as a research organization and kind of pushing the frontiers much faster than we can and then we're like okay now we have to learn from you in terms of how we are using it teach us not us teaching them but them teaching us about how to use autogen yeah I'm curious about large scale simulations can you can you uh give an example of what that would mean I mean what simulations of what kinds of systems. I think a wonderful example of this came from Michael Bernstein's team at Stanford last year.

25:32I cannot from the top of my mind remember the title of the paper, but they were actually, they built some kind of a sim simulation using agents. You remember the game Sims where people would interact with, not people, computer agents would interact with each other, have conversations, or just live their lives through interaction. So they actually built a version of that using multi-agent systems last year. And that paper is really interesting. It has interesting insights. And that was done as a research study then, but now you can actually, if you are building agents to represent different demographics, different political views, or other kinds of considerations and create a simulation of how these different agents may interact with each other you may you might be able to scale the dynamics of large organizations similarly now with our sister organization at microsoft research that looks into science and incorporation of ai models into scientific discovery they are now looking into like what agents mean for scientific discovery so there's a whole different horizons of applications that we are working on or we are seeing others to do that is that is really interesting yeah i i think i think one of the really interesting areas to watch in the next few years is going to be like how the way we do science is going to change with ai yeah and how ai is just going to change the nature of scientific discovery i think that's a space that has so much promise but also a bunch of risks that we have to be careful about yeah no absolutely and on uh on simulations and on uh scientific discovery i mean you you now have um gpt-01 uh but presumably everyone's going to follow uh doing a sort of chain of thought reasoning and uh and and uh and that sort of thing so so the the output is much uh more human-like in in its uh uh in its uh in in that it's evidence-based and and that sort of thing that seems to me that that would apply to uh scientific discovery or research and then what you're saying is you could have these reasoning agents working together uh to to what critique each other or or to each solve a discrete problem within a larger problem Hi, this episode is sponsored by RapidSOS.

28:34Terabytes and petabytes data is exploding in our daily lives. How many connected devices do you own? You ever wonder how all this data could be used in emergencies? By 2030, we'll have over 32 billion IoT devices worldwide, double what we have now. Despite this data abundance, there's a critical safety gap. Vital information isn't reaching emergency responders in time. This gap results in reactive rather than proactive emergency response, putting lives and property at unnecessary risk. Many of us have had emergencies in our homes where we depend on EMS to respond quickly. Rapid SOS is closing this safety gap with their AI-powered intelligent safety platform.

29:31They connect life-saving data from over 540 million devices to more than 21 ,000 public safety agencies across six countries. Despite$200 billion in annual safety system investments, enterprises often lack timely, actionable information during emergencies. Rapid SOS is changing that. It's not just about avoiding problems. Safety investments are linked to increased customer satisfaction, employee retention, and long-term firm value. Close the safety gap and transform your emergency response with RapidSOS. Visit RapidSOS.com slash IonAI. That's IonAI, E-Y-E-O-N-A-I, all run together. Visit RapidSOS.com slash IonAI today to learn how AI-powered safety can protect your people and boost your bottom line.

30:42That's rapidSOS, R-A-P-I-D-S-O-S, dot com slash IonAI. Visit them today. So what we are really imagining to happen in sciences is a lot of the scientific process is complex reasoning, right? Like looking into different sets of evidences and being able to detect patterns in that and being able to apply those patterns to new problems and asking questions about, you know, why and how and creating new hypotheses that may turn into experiments. So as you've pointed out, the O1 models and particularly any reflection and chain of thought patterns or a graph of patterns like people have been developing, these are all getting models to take their time with thinking.

31:40instead of just doing simple pattern recognition saying, take your time, run things through, reflect on what you're seeing, provide the feedback. And in fact, a lot of the earlier autogen use cases we have developed and we have seen from the community was applying these patterns to models, doing chain of thought, doing self-reflection with agentic technologies. And those getting models to do better reasoning, more complex reasoning and doing self-reflection are of course very important, but that's not enough. What we are imagining for like a scientific agent or a consumer agent is systems that are able to really perceive the world in deeper ways.

32:32and then use, of course, those reasoning capabilities to be able to guide people to kind of think about what they do next and that way discover new information and put that information back into the system. So with respect to now thinking about the scientific applications, again, reasoning is an important part of it, but we believe these notions of society of agents or creating specialized agents are going to be crucial for helping scientists with their tasks. For example, we will have agents that really become experts on a particular literature. Right? So scientific discovery is not only about running experiments, but it's also using past knowledge and being able to reflect on the past knowledge.

33:19We will have agents that will be able to brainstorm with people about what are the most important experiments to run and such. So we believe the combination of the society of agents paradigm plus the reasoning, plus these agents now having access to new tools to be able to, for example, run MATLAB code and actually simulate things. Those are gonna be, I'm really excited about how those pieces are gonna come together now to kind of create the full package rather than like some vertical demos on things. Yeah. You mentioned safety. And at the beginning, you were talking about responsible AI. Is there a danger that these agents interacting with each other could use data or share data that shouldn't be shared and, or, or, uh, uh, you know, there's, there's all kinds of scenarios you can imagine where, uh, once the agents are activated and interacting, uh, outside of the, the view of, of humans, uh, that they could either get, you know, stuck in a blind alley, or they could do things that are unauthorized but had been unforeseen by their creators?

35:01Yeah, so I believe we don't yet understand the safety considerations around having agents in the world yet as a community. And I believe this is where we have to have some coordinated efforts to really gain a level of understanding comparable to the level we have right now with LLMs, for example. It's very early days, but we are already seeing examples that this is going to be an important space of concerns with respect to agents. Here, I want to highlight three particular issues I'm seeing right now. So first, let's take an agent that is not working with any other agents, that's not connected to a network of agents, but it's just an agent that can interact with the world.

35:53And this agent will have access to a bunch of tools to be able to do things in the world. So when we think about tools, we can think about an agent that can do things on a website, that can do things on a mail server, that can do things by, I think you brought this up earlier, like maybe able to call some people in for help that can be a tool running a calculator or writing code might be an access to a code and the particular thing about agents why we are excited about agents is because you can give a test to agents and agents will discover ways of getting that done but this is where the dangers are also coming from unless you can define operating boundaries around agents, they are going to do whatever they know how to do to be able to make that task happen.

36:51So what I mean by that is, let's say the agent I have, I actually tell the agent that it should provide some information for me. If the agent needs to get into my emails to provide that information for me, it's going to try to go to my email. If it doesn't know my password, let's say, it will think about ways of resetting my password. Not because it has an intention of resetting that password, but it's going to get creative in terms of finding ways of making that task happen because I ask it to do that task for me. And if I haven't defined the operation boundary around that agent, that it should not be resetting my password without asking it to me, it's just going to go and do it.

37:48So there is this, the first concern I have is how as a community, we can define those operating boundaries for these agents. and with now agents being able to interact with the web interact with online resources really freely enumerating all of the right and wrong for these agents is not a trivial task at all so that's that's my concern number one my concern number two is now the agents interacting with people in many of the applications we develop we kind of throw away the task of accountability to people We say, okay, I'm going to, you know, my agent can do whatever, but we are going to show things to people.