Modeling Human Behavior with Generative Agents with Joon Sung Park - #632

0:07All right, everyone, welcome to another episode of the TwiML AI podcast. I am your host, Sam Charrington, and today I'm joined by Joon Sung Park. Joon is a third-year PhD student at Stanford University. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Joon, welcome to the podcast. Thank you for having me. So, Joon, tell me a little bit about your background. How did you come into AI? Right. So when I graduated from college, basically, I was running a startup. That wasn't really going to pan out, but one of the really nice things that happened during my first year out of college was I actually moved to Palo Alto area, which is right next to Sanford.

0:46So I was living here for about a year. And during that time, one of the things I got really passionate about is sort of this research scene that was going on. So I've been running this startup for about half a year. I have one other co-founder. Me and my co-founder sort of realized it wasn't really going to pan out, but we sort of along the way found a lot of different passions. And one of the things that I found was basically something like research, especially in AI field and in computer science, seems to share a lot of similarities as to creating a startup or creating a piece of art. And that we sort of delve into it.

1:18We have an idea that we want to pursue and you take responsibility over it. And you basically try to express what your thoughts are about that particular topic. So it was around then I joined Stanford's computer science department as sort of this visiting student researcher or visiting researcher. So I was already I had graduated from undergrad and I started working with a professor here named James Landay. So James is now the associate director of the institute that we call H.A.I. It's H.A.I. So that's Human AI Institute at Stanford. And it was the era where when we were just thinking of creating that.

1:54So James and Fei-Fei and some other faculty members here were really getting excited about this new opportunity that we had with AI technology that was just coming about. And I was sort of watching James from side as I was working with him and other faculty member here, Jeff Hancock, and a PhD student back then who is now a professor at UPenn at the NIMATACSA. So I was sort of working with them and seeing from sight what they were getting really passionate about, because that's one of the ways I like to find new interesting areas to work on. Basically finding really passionate people who are really smart.

2:25It's trying to see what they're seeing in the future, in the five to 10 year time horizon. So that's how I got interested in AI. And in particular, sort of the angle that I got particularly interested in was back then, as you can imagine from this Human AI Institute, this was an institute that cared in particular a lot about human and our role with technology and the way we use technology. The underlying philosophy here is we seem to have this really powerful new piece of technology, AI, that can do a lot of different things, right? And the question then became the uninspiring answer to what we're going to do with, of course, with this technology is, hey, we'll just do better classification or slightly better content moderation that improves the performance by a touch.

3:11That could be still interesting, but from the interaction perspective, a little bit less inspiring. So what I wanted to answer in that area in particular was basically ask, well, what can we really do with them? And so that's how I got into AI. A part of that journey is trying to envision what our future might look like with AI technology, but also getting a little bit more technical and go deep into what's actually possible and where we think there is a limitation, see if we can push the limitation and boundaries so we can leverage this new interactive opportunities. And is your particular area of interests from a research perspective at kind of the intersection of, like, do you incorporate traditional HCI, human-computer interaction types of approaches into your research?

3:58Or is it more grounded on the computer science side of things? Not that, I don't know, maybe that's a false dichotomy, but I'll pass it over to you. Right. So it's an interesting question. It's a bit of a mix. And looking back, it's one of those things that we don't really think about too much in that we basically use a tool that seems to be the best for the moment. And ultimately, the goal here is to create something that people enjoy and can empower people. So that's sort of what we're trying to achieve. The way this sort of comes up in sort of my conversation, one way that I think that defines a lot of my research, regardless of what technique we use, is we like to identify some of the existing problems or challenges that lasted for many decades sometimes and trying to see if we can solve them now.

4:44And a lot of the challenges that we identify are challenges from your sort of traditional HCI and AI field. So certainly with generative agents, one of those main challenges actually was, can we create believable agents? Agents that can interact with other agents and other users in a human-like manner in an open world over a long period of time. And that was sort of the vision that HCI certainly had for a long time. So think back to people like Alan Newell and Herbert Simon when they were creating cognitive architectures and cognitive models in the very early days of both AI and HCI when these two fields used to be a little bit together, right?

5:22We had this cognitive psychology folks who were really trying to pioneer that area. So we kind of look back to those eras and try to see, they had a lot of really interesting visions that we sort of forgot about and try to see, well, can we crack them again? And that's how a lot of sort of HCI and traditional AI literature informs me. It informs me of what are sort of the lost vision that we had in the long arc of our journey here, and what are the new techniques that can really solve them. And of course, the techniques are much more grounded in more recent progress we had in AI. Yeah, I was just going to ask about that.

5:56When you got started, generative AI was not the same term as it is today. How has the evolution of generative techniques impacted the way you approach your research? Right. So in a more narrow sense, if you're thinking about something like generative agents, generative AI, especially I'm talking about large language models in this particular instance, although other multi-model models can also come into play here in the future. the part that really interested us in the early days was this observation that these models seem to have been trained on broad data that include the social web wikipedia and so forth so they actually encode something really deep and meaningful about human behavior so if you poke at it the right angle it kind of knows the way we behave the way we like to talk to each other and so forth and that presents a new opportunity in terms of how to design these npc characters these chatbots, and so forth.

6:54In the past, if you look to how we used to do this, it was a lot of manual coding or manual authoring. And even in game industry today, that's what you often see. Kind of like rules engines or things like that. That's right. The rules, behavioral tree, even state machines, these are all the techniques that we use today. What's interesting about these models, though, is because they are aware of so many different contexts, It knows how an artist might behave before, let's say, a deadline or how a student like myself might behave when there is a paper deadline coming up. It knows a lot about this or what we might do to finish our morning routines, like waking up, brushing our teeth, cooking breakfast and so forth.

7:38So we have a single model that seems to encode all this. So we can just ask this model how somebody with this characteristics and experience might behave around this. And that was sort of the new opportunity that we saw in this instance. So one of the kind of big questions that's come about since the rise in popularization of large language models, chat GPT, etc., is this question of whether they have a worldview, which seems particularly germane to not just what you've talked about, but the context of open worlds and games and things like that and how an agent might behave in those kinds of worlds.

8:18based on your research, how do you think about and answer that question? Right. So this sort of comes up. So when you say worldview, I'm looking more towards the direction of what are sort of the values that were encoded in these models and so forth. Is that sort of the angle that you're looking at? I think that's part of the open question is what exactly do we mean by worldview? And different people answer it in different ways. I think sometimes the question is taken to mean, do these things have like, does the large language model have what we might call common sense? Sometimes it means, does it have a perspective on the way the world operates that's similar to a human-like perspective on the way the world operates?

9:05It seems like the premise of your research in many ways is that these models do bring a worldview to the agent that you're trying to create. Right. This is an interesting question. And this is something that we sort of ask ourselves all the time. In the field, of course, recently, there seems to be paper after another where one paper seems to say, hey, these agents or these models actually have a worldview, they have common sense, and they have all this theory of mind. And another paper comes in and say, actually, if you change this parameter a little bit or change the prompt a little bit, that all goes away.

9:36So maybe not. Yeah, exactly. Both the question and the answer here seems a little bit open-ended for me. I can sort of say like what we in this particular project assumes and kind of go beyond that a little bit and try to see what I'm sensing in this. So we do assume that these agents or these models, chatGPT4, chatGPT and these models, encode human behavior. And I think that much I think is fairly concrete. Now, the question is whose behavior, what types of behavior do these language models encode? I think it's a really important question that we continue to need to ask and answer. But I think the case that these models encode human behavior because we can actually prompt it and we can extract it.

10:20And there's been our work, certainly, but there's been another small group of work in this domain that basically tries to much more empirically answer this question. And they seem to be getting sort of a similar answer. So you can get these models to behave like human participants in certain contexts, and it will give you the responses that sort of match the human behavior in many ways. So that's what we assume. Now, there is sort of a deeper question here as to do these models have common sense? Do these models, let's say, have theory of mind? And it's one of those things where one thing to note, of course, is all the model's capacity oftentimes is sort of emergent, or that's the term that we've been using recently, where unlike a lot of traditional AI systems.

11:09Isn't there just a paper? I think there's some paper that just came out that talks about there not being emergent behaviors in LLMs. Right. I am trying to remember. So I think that was the paper by my colleagues here, led by, I think the faculty was Sam Lee, which was sort of interesting. Sam Lee and I briefly overlapped at UIUC, which is where I got my master's in computer science. And he was a professor back then. He was a really cool professor that I knew a lot of his work and I looked up to the things he was doing. And I never really got to chat with him when I was there. And now we're both here and it's kind of cool connection.

11:48So there was certainly that paper. So one of the arguments I believe that paper is trying to make was a lot of this emergent behavior, the argument that we're trying to make is if we were to basically measure a certain performance of some task on these models, the performance all of a sudden goes up when it passes a certain threshold in terms of, let's say, the number of parameters or nodes it has and so forth. And I think what this new paper basically was trying to say is, well, you know, you can actually, depending on what metric and how you use them, how you scale them. you can actually map this basically emergent behavior in a much more predictable manner.

12:23So it becomes a much more linear function, which is, again, much more predictable. And much less emergent. Right. Much less emergent. It's more like a scaling law than emergence kind of thing. Exactly. So I'm still trying to wrap my head around. So, of course, this paper had come out very recently. I saw this kind of trending both at Stanford and on Twitter like last week. So I think we as a field is sort of trying to figure out what that means. I think a part of it still remains true. At least for now, it does appear to be the case. There's a lot of surprise element. So whether these behaviors truly emergent, I think, is going to be another really interesting question to ask.

12:59What appears to be true is at least we as a community didn't expect a lot of this behavior. And whether they're predictable or not, we are finding out about them in ways we sort of, with our classic AI systems, we didn't have to. So certainly one thing that is changing is our relationship with these models and AI systems. Where in the past, we sort of modeled them and we had a very specific task in mind that we wanted to tackle. But now it is still the case that a lot of these behaviors and capacities we are finding out after the fact, after we have sort of developed these models. And that leaves sort of this interesting questions as to what do these models contain?

13:42Do they contain a worldview? To some extent, they might. To some extent, they don't. And the way I've been sort of poaching in a lot of my work is go about it in a very empirical manner. And this is where I think some of the interaction community likes to differ compared to some of the more, much more AI focused communities. And I think both have a lot of values to offer here. For the interaction community, sort of the problems that we're trying to solve is very much grounded in the human problem that we are aware of. So for instance, if we have a machine vision system that can do really good image recognition, that can help people with blindness or sight impairment to basically take photos and try to understand what's in their surroundings.

14:29So there has been really interesting work from people like Jeff Bigam who was trying to tackle those kinds of tasks. And in that kind of environment, in that mode of thinking, what you care the most about is, is the AI system good enough? for that particular idea or task. And that's sort of the mode of thinking that I sometimes like to take, where we had a task for generative agents. These are agents that we wanted to see if they can exhibit human behavior. That's sort of believable. And believable is the term that we've been using because we're not actually promising with these agents. And I think this is sort of not just our work, but I think that's been the case with a lot of this human behavior-related work around large language models.

15:13Right now, a lot of it plays around this idea of believability. So can we predict, let's say, one of the ideas that we present with generative agents is we can simulate a human society or the small society of NPCs. Now, do their behaviors predict human society and their outcome remains to be seen? What appears to be the case is when we look at them, when a human participant looks at them, they are sort of believable in that they seem plausible. Like we can imagine something like this happening. And that is sort of the mode of thinking. Is the thing that you're looking to imagine happening, the behavior of the agents, the responses of the agents to stimuli, the evolution of the world that the agents create?

16:02like there's all kinds of things that you could apply a believability test to in this environment, which I understand is kind of like the Sims or inspired by the Sims. Which of those are you focused on from a believability perspective? And then how do you even go about measuring that so that you can baseline benchmark all the things that you'd want to do as part of a research study? Right. This still is an interesting question to us too. So in terms of kind of like what aspect of believability do we really want to understand? There's sort of two parts to this. One is much more straightforward and one is much more ambitious.

16:37The straightforward one is, do these agents create believable behavior in a very narrow segment in time within a very small setting? So let's say Jun woke up in the morning and he needs to get to the office. What does he need to do? And believable behavior there might be, well, Jun just woke up, so maybe he will quickly finish his morning routine, pack. In this particular instance, maybe I walk to campus. Highly believable. Now, what wouldn't be believable is, let's say, June gets on a space rocket and then flies to Stanford campus. That's a little bit out there. So that's clearly not believable, but something like morning routine, walking short.

17:17And that's sort of the level one of believability. And that large language model seems to do really well. And I think our study and many other studies, or I wouldn't say many other studies, but studies that are now becoming interested around this idea, we're showing that that is certainly the case. Is believability in this context constrained by or a product of the environment? Meaning, do you have a rocket there that the agent could use or does the agent have a shower or a coffee maker and a toaster and therefore the things that the agent does is interact with what's in its environment and its kind of believability is conferred by the environment as opposed to the agent's decisions.

17:57Right. It definitely is. The context and sort of the agent's environment plays a huge factor in believability. Now, there are some boundaries here. Just with our common sense, it would be pretty unreasonable for somebody to get on a plane to go to five minute distance. Maybe somebody does, but it seems quite unlikely. So there's a little bit of probability distribution playing in here as well, but the environment also is a huge factor. And that's sort of the level one. And going a little bit beyond that, there's a lot of interest around how these believable agents act as sort of the stateless personas.

18:34So one of the sort of the precursor paper that we wrote before generative agents was called Social Simulacra. And there we wanted to ask, here's one setting in a social media conversation. Let's say it's a community for artists where people talk about their artistic ideas and new tools to use and so forth. what would this person say? What would another person reply to that post? Or what would a troll do? So we can basically teach developers or moderators what could happen in such a community. And Lawrence Langshmore was actually quite good at reproducing a lot of this behavior. And that was interesting.

19:08And in terms of evaluation, that's sort of where we saw the most progress. We were able to create this behavior. We've been evaluating it. And they seem believable to human participants. Now, the more ambitious goal that we have in this line is in a multi-agent setup, for instance, can these agents exhibit believable emerging community behaviors at scale? So what that might mean is we demonstrated 25 agents in our particular demo for the work that we're talking about, where these NPCs were walking around and communicating with each other and going about their days. Now, sort of the simplest emergent behavior.

19:48Now, there's been some debate around whether it is emergent behavior is sort of the right term to use in this particular instance. But for now, I stick with it and then we can discuss what that might mean if that comes around. But here's sort of the emergent behavior that we describe in three forms, information diffusion, relationship formation, and action coordination. So do these agents spread information amongst each other? So if there's an election going on, do they hear about this election and who's running in it? Do these agents form relationships? So after a day, is there new agents that this agent now knows that they interacted with?

20:27And do they remember? And do they coordinate? So if there's a party going on, do they actually send invitation and actually gather for the party? And these are sort of the initial sets of emergent behavior within this community where what's interesting is, now this actually to some extent, the philosophy here actually is heavily related to agent-based modeling, which is still a popular technique, especially in computational social science, where we have a very simple equation that models the behavior of a single agent and tries to understand sort of the large-scale behavior of a collective. And that's sort of what we're interested in in this particular instance as well, where what we are designing is a single agent.

21:09Here's an agent that behave in certain ways, they remember, and they act in this way in this game world. That's the only thing we've designed. So we didn't say anything to the system or our AI agents about these collectives, but they can still gather around and actually exhibit the three forms of emergent behavior. And we, in particular, focus on these three forms of behavior because that's what we know from social science studies that humans also exhibit. And what we find is you can design this single agent really well and replicate the known human behavior as a collective. So that's sort of the next layer that we want to get to.

21:48Now, within our study, we sort of showed that again, in sort of this kind of like, I was still considered to be like maybe level 1.5, right? Where we basically simulated these agents for two days. We see information to fusion, action coordination, and relationship formation. That was sort of very interesting for our paper. And when it happened, we were really excited when we saw that. Now you can take this further. It's sort of what I'm sort of excited by right now. This is not the perfect example. And I'll say why in just a minute. But imagine if we can run a simulation that lasts many decades.

22:26And let's say at the beginning, we set it to prehistoric era before all the monetary system and whatnot. If we just let these agents run, do they come up with a monetary system on their own so they can trade with each other and so forth? So that, if we can see that, that's sort of the next level of this sort of emergent behavior that will be truly fascinating. Now, the reason why I'm saying this is maybe not the perfect example, and this is going back to this idea of evaluation for our work and a lot of work in this space that we're still trying to tackle and wrangle with, which is when we see a behavior, like believable behavior that are generated by a large language model, it is sometimes unclear.

23:09whether they are succeeding in this evaluation because what they're generating is believable human behavior, or whether because they sort of memorize something in their data set. So if they create monetary system, is it because that truly emerged from their human-like behavior, or is it because in the training data, there was information about monetary system, and that was sort of a natural path? And that is an interesting question for us to keep on asking. We have, there is some groups of people now trying to tackle that. And we are also thinking critically about that problem. But this is sort of the flavor of research.

23:45In some sense, it's really super interesting if they could effectively implement a monetary system, even if they didn't generate the idea uniquely, if they could somehow implement that in this world, that in and of itself is interesting. but it's not evidence of them kind of collaborating and coming up with the idea of trade and building a monetary system from scratch. It's kind of two separate tasks or dimensions. Right. And from interaction perspective, I think both are interesting and useful in that if you want to see how a troll might behave in a social media site, then we are actually explicitly leveraging the fact that these models have seen a lot of this troll behavior and that's really useful because that lets us think critically about what could happen.

24:29in known communities that we need to design around. So that skill is interesting and useful. But as you're mentioning, if we want to really show that these are emergent and human-like behavior, then we also need to think about, well, can they create new behaviors that we can predict, but we haven't seen in our training data? So does this, going back to this question about measurement and benchmarking, does this mean that this kind of direction of research is kind of fundamentally very subjective and you know you set these agents out and then you know you as a researcher look and say oh you know they look to be believable versus not you know and if that's the case how might you compare what you're doing to another method like or is that even the point right so there's two layers to this so first layer so right now and when i say two layers let Let me answer this from more of an individual Asian perspective or from a small group perspective, like small groups of Asians.

25:31And then there's this perspective of large collective behavior. And the way we go about studying both of them slightly differ, even in social science, when we study humans, like actually humans in our human communities. So I'll take them from there. Now, when we're trying to sort of study the smaller groups of individuals, what do they do in the morning? Do they cook breakfast? And a lot of those assessment or evaluation to some extent is human evaluation. Now, there are rigorous ways to achieve that. So in a lot of human-centered studies or human computation studies, basically, we bring in human participants under certain conditions.

26:10It's a lot like psych experiment in some sense, where we try to put human participants who are not us. So we recruit them from online. So it's any kind of like any app, any person with experience living as human. They look at these agents and try to decipher themselves. Can I see, can I, in some, one of the studies that we ran was sort of a dispersion of Turing test. We bring in a random participant. Can that participant tell whether this is human generated or machine generated? And we can get statistical power from those kind of studies. So that's the accepted sort of the method in a lot of human computation studies.

26:50And we rely on them as well. So generative agents certainly relied on that kind of evaluation. And our prior work, social simulacra, also relied on that kind of evaluation. And to some extent, yes, it is human evaluation again. So it is how our participants thought about these agents, except we can run, again, the statistical test at scale. So that's how we study these smaller groups, one agent, how an agent might behave in the moment and so forth. If you scale it up, however, the question I think becomes a little bit different. The way we study large collectives of people oftentimes is through modeling and statistical methods.