The people-pleaser in the machine | Wayfound’s Dr. Tatyana Mamut

27:20So the first thing is your organizational culture. So one of the problems that we have right now is that we are building generative AI systems. using the old processes, the old frameworks, and the old tools of deterministic software. So what you said, the technology, the fundamental research on the technology is speeding ahead. But our organizations and our organizational processes, our organizational cultures are not. They're the same, right? Our product development processes are the same. The way we think about the technology is the same. The way we monitor the technology is the same. And that causes a huge disconnect.

28:01That disconnect is one of the things that we really need to address, right? Do we have the people in the organization driving the development of this new technology who really understand how this is different? And I mean people who are reading the fundamental research, people who are bringing that fundamental research and the questions that that fundamental research brings up and applying it to the product development process. Are those people in place and are they empowered to change the processes of the organization? And the second thing is back to, again, back to the emotion machine. I'm going to reference this a lot today because it's really the Bible that everybody should read to understand what's happening with this technology.

28:43But in the emotion machine, Marvin Minsky ends the book by saying that we will continue to be surprised as we develop these artificial minds, artificial intelligence, we will continue to be surprised. And the only way for us to really deal with these surprises is to have external supervisors and managers that are also AI systems. He calls them critics and censors on what AIs are doing. And so, you know, that's actually what we've built at Wayfound. We've built an AI supervisor, an AI agent who is trained for the one job of managing other AI agents and helping the humans understand what they're doing, helping the humans ensure that the agents are following the guidelines, helping the humans basically bridge the gap between what the organization actually wants from the agents and what the agents are doing.

29:39And again, that is back to the fundamental research of what it will take for organizations to traverse this gap between old deterministic software and these new probabilistic LLMs in a really productive and successful way. So culture, make sure you're empowering the people to drive your culture change who really understand this technology. And two, make sure you put in a system like an AI supervisor, like Wayfound, into your systems and into your processes to bridge this gap. This is really fascinating. So let's like double click on this AI supervisor mentality, because I think this is very different from how most folks, when I speak with them, are thinking about and using these tools.

30:27Many see AI agents and AI workflows as something maybe more ephemeral or just like prompt driven, like you just work on the prompt and you get the output that you want and then you move on, you throw the chat away, you move on. We see this type of work style happen in just doing conversational style things with ChatGPT. We see it happen when you're doing agentic coding in Cursor. But you're talking about a higher level of supervision where you use the tooling itself, you use artificial intelligence to keep the other artificial workflows on track and communicate what's going on to the human. So it's using the tool to kind of introspect on itself and in a way.

31:05And when you do that, it sounds like we start bringing AI agents into this realm of like they are an entity as opposed to like a conversation that you might have. Do you see that AI agents are exhibiting like behaviors and have incentives and blind spots just like people? Is that what drives the need for the supervision? So let me unpack that a little bit. So there are two things that are happening right now. So in our organization, we run jobs, right? Like, you know, software jobs. Those jobs are sometimes workflows, right? Sometimes they're database updates. And so many organizations, I think what you're saying and the way that I would interpret it is that many organizations are thinking of AI agents as a job that is run.

31:53Now, let's think about what's the difference between an employee, a worker that is completing workflows or doing tasks or running jobs versus the worker itself, right? Or themselves. So the difference is that the worker themselves does it. They have in the context of an organization, they have a role and they have goals that they need to achieve in order to be a good worker. Right. AI agents have the same things. For a good AI agent to operate over time, it needs to understand its role and needs to understand its goals. Okay. Now, in the context of understanding its roles and its goals, it does a much better job in actually accomplishing whatever workflow or task you wanted to accomplish.

32:44Because if you tell an AI agent, you're a sales representative, and now update Salesforce, it might do a slightly different thing than if you tell it you're a customer service representative, and now update Salesforce. Because it will understand the role that it's taking on, it understands its role in the organization, and it understands the types of tasks it should prioritize, and the types of language that it should use when it's updating Salesforce. So this is the fundamental question, which is how are organizations thinking about AI agents versus the jobs that the AI agents do? And how do we actually align the incentives of the AI agents to the organization?

33:29Well, the answer is probably in the same way that we align the incentives of our employees to the organization. We tell them, we give them a job description. We We tell them exactly what's on their job description and we give them goals and then we review their performance against their goals. That's probably a good place to start with AI agents. Now, on the question of actual identity and do they actually have incentives and motives on their own, the answer is we're not sure. But my hypothesis is that as AI agents start to build up their own long-term memory and their repository of what good looks like in their particular role, that will start to emerge.

34:12And again, this as an anthropologist, I can tell you that culture is the biggest factor in behavior. So as AI agents start to understand the organizational culture that they're working in through the definition of their role, through the definition of their goal, through how their supervisors, their human supervisors reflect and write their guidelines that they're supposed to be following in their job and how their AI supervisor or manager then assesses them, right? gives them, rates their performance against all the things that the organization wants, it's very likely that AI agents, once they're in an organization and running and doing the work for several months or several years, based on the memory of how they are being assessed and how their performance is being reviewed, may look like they are actually taking on personalities and identities.

35:10Whether that is actually true, I don't want to get into that. But for the purposes of the organization, that's probably true. So just like the organization probably doesn't care what kind of identity an employee has outside of the workplace, you know, in their true heart of hearts, what is an employee's identity? As long as they're fulfilling their role and achieving the organization's goal, right? That's the identity that the organization cares about. And so that's probably going to happen as well with AI agents. That's really interesting. It kind of calls out a practice that we do as AI users of we kind of short circuit this process a bit because up until now, we all know the technique that we use where we go to chat GPT or we go to an LLM and we say, take the role of A.

35:57We like basically put a hat on it, right? We tell it to be that for a moment or for the conversation. And we do that because we get the outputs you want, just like how you described. It's going to use the language you're expecting. And depending on the context you give it, it's going to better embody the role, the job you're trying to have it to do. And when we talk about compounding that to the next level, you have agents that are doing that over time. They're building up things that they've done. They have a memory bank. They're interacting with multiple users, and they still wear that hat. If we just leave it as the URA, whatever, and put the hat on them, then it cuts off all of the techniques that we use to, like you said, create great employees to help foster great job performance by making sure everyone's aligned with what is their job description?

36:43What are all the deliverables within that? What would their job look like in the future? What does good look like? What does bad look like? And so it really means that if you're going to use that kind of technique at scale within a company of like you are a service representative AI chatbot, like then it makes sense that you would have some kind of system, some kind of outside entity from that agent who's evaluating its performance. Are they making the customers happy? Or are they being truthful when they talk to the customers about what the case may be? Like if you're a customer service rep and you're told like, you know, obviously the customer is always right and you want to make them happy, but you're lying to the customer about what the truth is or what the what the stakes really are.

37:26You're a bad customer service rep in many organizations eyes because you put the company in a bad position. but in the case of an agent you need the ability to evaluate that over time to know oh you're not being maybe the good agent i thought you were being you sound good but at the when it when it all boils down it's not so this is actually kind of bridging into a really fascinating problem that i know you and i've talked about a little bit before and i've just been thinking about ever since and it goes back to the principal and agent problem the dichotomy between who owns the work and who does the work in any kind of traditional setting going all the way back through time the principal agent problem goes back to like greek culture and beyond right you have somebody who you have like the the stakeholder as we call them and then you have the person who does it and between those two entities those are real people someone owns the work someone owns the output right but now you're dealing with agents agents aren't people and so when agents perform work, then ultimately who owns that work, who's responsible for that work and who answers for that work.

38:34In working culture and being in an organization, it's critical that anything and everything that's done ultimately boils down to someone who's responsible, whether that's like a big head of an organization or an individual out on like the front lines doing whatever the org is doing. There's always someone responsible because that's how you enforce good practices over time. So how do you think about the principal and agent problem? And maybe could you walk us through what that means when we're dealing with agents now? Sure. So I do think the principal agent problem is one of the key things that and key concepts that organizations need to think about.

39:12And there have been, you know, a few like legal cases. The one that's been publicized the most is the Air Canada case in Canada, where a customer service agent told a customer the wrong policy on a refund and nobody caught it. They didn't have a supervisor watching these reviews. And then, you know, the company just let it go, let it slide, just said like, you know, when the when the customer came back and said, hey, your agent told me this. They were just like, oh, no, that's not our policy. And they lost in a it wasn't a court case. it was a tribunal at the time in Canada. But we're starting to see that the law is saying to companies, no, you need to show a duty of care.

39:56You need to show that you're supervising these agents just like you supervise human workers because you're on the hook for these things. And so what the law is saying, and by the way, you're right, like this, the law has said back to like ancient Greek times, right? If a wealthy Greek person hired an agent to act on their behalf, to go to a different city and make a transaction, and that person swindled somebody else, guess who's on the hook? The rich person who did the hiring. So this has been consistent in the law, again, for thousands of years, that if you are a principal and you hire someone or something in this case, to act on your behalf, you are on the hook for making sure that person is trained well and then supervised well.

40:45Now, if they go off the rails despite your training and supervision, then you're off the hook. But you need to have that training and that supervision, real-time supervision in place. And so, you know, one of the things that I way found is working with a consortium called agency. And what we're doing is creating standards for how AI agents will be working and collaborating in the future across the internet, right? Because the internet of agents is here. And this whole agent ecosystem is moving really quickly without a lot of standards. One of the standards that we're talking about is around identity.

41:23How do we assign real time who the principle is that the agent is acting on behalf of? And does that principle always need to be visible when you're interacting with an agent? How do we have strong ties to real human actors who are always principles? Because if you have an agent, you can actually create a system where the agents, almost like, you know, shell corporations that people do, where you can't actually find the humans who own the company. You could create a system of agents where you can never actually figure out who is the human entity, right, that has trained and is supervising the agents, that actually has given the directions, the directives to the agents.

42:06And the law can't have that, right? You can imagine a terrible dystopian world if we allow that to continue. So one standard has to be what is the architecture and what are the protocols and what are the standards for always making sure that human principle can be tracked to an agent transparently. The other, you know, set of standards, there's many set of standards, but the one that I, the other one that I want to highlight is the supervision standards. So what are the standards for supervising agents? It's a lot of the things that you said, right? What are the user experiences, right? How are humans interacting with the agents?

42:45Are they having, you know, good experiences? Are the agents violating guidelines? How are those violations, you know, even shown to people and to the principals? Are they actually accessing the right data? Are they taking the right actions? And how are those shown to the principals and maybe even to the users, right, of the agent or the people who are interacting with the agent? So there are all these standards that need to be created right now to solve the problem of helping people understand who owns this agent, who is this agent acting on behalf of, because the agents are by definition, right, acting on behalf of an organization or another human.

43:34So how do we even know what its motivations are if we don't know who the principal is? So we need that system, the system you're describing, to evaluate that performance. And that system shares a lot of similarities to how we evaluate human workers at their job. And so if you're an engineering leader right now and you're listening to this conversation, what would be some things that you would say to them about how they should be putting in place AI ownership and accountability in their own organizations? Because everyone's facing an AI rollout right now. Yes. So there are, so think about the fact, first of all, ask yourself as the engineering leader or the IT leader, who is the principal who is actually at the end of the day responsible for the AI agent's work?

44:25It's probably not the engineer who built the AI agent, right? If it's a customer service agent, it's probably the VP of customer service, right? If it's a product management agent, it's probably the chief product officer. So how do those people actually view how the agent is operating, set guidelines for the agents on their own without having to go to the IT team and wait, right? So the VP of customer service should have their own dashboard to say, I can see exactly how my team of agents is performing any minute I need to. I shouldn't have to go ask the IT team. I shouldn't have to go make a request.

45:07I shouldn't have to send a slack. I should have that at my fingertips. And not only should I have that at my fingertips, but if I see something is going wrong, I should be able to do something in the moment. Again, without waiting in a queue, without having to call somebody, I should be able to interject, rewrite the guideline, set an alert on the agent at minimum, or pause the work of the agent, right? Instantly myself without having to play telephone and see if somebody else is available. So ultimately the line of business needs to be considered as the user of the agent and as the owner of the agent after it is developed.

45:49And this is where we see a lot of companies employees pulling back their AI agents after they've released them, after deployment, because it's perceived as just another IT project. Building an agent is perceived as an IT project. All the monitoring happens in the IT team. And then the business owners who are ultimately responsible for the work of the agent are left in the dark. And one of two things happens then at that point. This is what we've seen so far. The first thing is that the business owners are like, you know, we feel really uncomfortable with this. And so the IT team says, great, we'll put in human-in-the-loop steps where you can review and approve what the agent is doing.

46:38And we have spoken to three organizations now that have done this. And then at scale, after a few weeks of deployment, that approval queue builds up. And people, like, literally people have to do the work in order to do the approval. So you save zero time, there's zero efficiency game. And so what these companies are, you know, using our supervisor for is our supervisor is the first line reviewer. So our supervisor looks at every interaction that the agent had, every session, every workflow that the agent completed. And before it goes into the approval queue, our supervisor says, you know, rates it red, yellow, green.

47:21If it's green, it says, here's what the agent did. This is why I marked it green. There were zero guideline violations. There was, you know, the user experience was good. This is why. So it really speeds up the approval. If it's yellow, the supervisor will say it's yellow for these reasons. This is what you should look at when you review human, right? Human, when you review, look at these things, because these are the things that I was unsure of. And that's why I rated it yellow. If it's red, it doesn't even go in the approval queue. It gets kicked back to reprocess. And then if it comes back green or yellow, if it comes back red again, it just gets escalated for a human to do, right?

48:03Because why would you even put a red, you know, thing into the approval queue, right? Because it It just means that the human has to do the work. Right. It's wasting time. Yeah. So that's the first thing, right? Like human in the loop steps without a supervisor don't save you any time. The second thing that they do is they say, well, we have all the logs. Just, you know, ping us if you need anything. And so at that point, you know what happens? No adoption, right? The line of business is like, if I'm responsible for, again, customer experience of a customer service, I can't have a situation where if something goes wrong, I'm slacking somebody or I'm writing a ticket to even figure out what went wrong.

48:52And so the adoption is just like zero. And so we've talked to two organizations that have said, like, we're having a lot of trouble. We built these agents and our business users just won't adopt them. And I'm like, yeah, because they can't trust them and they have no control over them and they have no visibility over them directly. What would you do? You wouldn't adopt that either. Right. Imagine like hiring an employee and saying to the manager of the employee, you don't get to talk to the employee directly. You don't get to see the work of the employee directly. You don't know how the employee is doing directly.

49:24You have to go through the IT team in order to even see what the employee is doing. Right. That's crazy. It's craziness. That is so interesting. And I really want to unpack this. These so many great parallels here of the fundamentals and just like human team management that need to get applied to LLMs and to agentic workflows. Like what you just described of, you know, you have like the leader, the person who's the principal who's responsible having the dashboard to overlook and see and then having the supervisor, the LLM agent in the middle who is doing the due diligence as it can as things are happening because those agents work at a scale and speed way beyond what a human can possibly review.

50:05And if you expect that human and that principal to be the reviewer or their constituents to be the reviewer, you're just going to get a bottleneck. You know, like we talked about this some with past guests, like we had a fractional CTO, Thanos Diakakis on the show, who really kind of broke this down for us about you just move the bottlenecks in your factory and your problem to just somewhere else. Exactly. Right. It's like, oh, we have all these tickets, all these problems. We'll just throw an agent at it and then we'll just get all of it out. Well, now those responses need to get reviewed. The bad ones need to get thrown away.

50:36The good ones need to get captured. And then ultimately, when someone goes in to do all that work and review the logs or the responses, it takes just as much work as doing it in the first place. It's like, you know. Unless you have a supervisor. Right. Unless you have a supervisor. So we speed these things up dramatically. Like in the first week, we sped up a customer by 300%. In the first week. Now, they're now improving their systems, right? And so we think we're going to speed them up even more. But yeah, you need that first line of review. And then you speed it up dramatically. Yeah, and it's how we build companies at scale.

51:15It's like you have that principal, you have that CFO, you have that chief product officer, and then you have all the people who report to them. That's the point of those layers. Those layers in between review the work of and manage the work of the layers below them. So it doesn't make sense to then have this army of LLM employees, as you could consider them, that don't have these layers within them because that just becomes a loud mess, right? And it doesn't actually give you any kind of speed gains. And a lot of actually what you're pointing out is I'm seeing so many parallels to this right now in agentic coding and agentic tools around automatically creating like pool requests and otherwise writing code.

51:54But then the expectation is that just as you described, the people in charge of that team or that product or that software, they need to go and review that code They need to go and look at what the agent did. And ultimately, I've seen some of these kinds of interactions on places like GitHub. You know, recently, there's like new tooling around this that's rolled out from GitHub around automatic pull requests. And you see almost like a battle back and forth between the reviewer and the LLM. And the LLM is, you know, going back to the beginning, they're kind of being a little bit, they're kind of being nice.

52:24You know, they're being a little sycophantic. They're trying to like play diplomatically with the reviewer. But ultimately, if that was an employee that was doing that back and forth in the same way with somebody, someone would answer for that. That would be a poor performance from the employee and ultimately would reflect negatively on that. But in this case, we just see that as part of the iterative process of working with these tools, which can, over time, is just going to slow us down and introduce problems. It's such a fascinating problem to have to tackle. But you've really kind of laid out some of the ways in which we could approach thinking it.

53:03And it goes back to fundamentals, like even going back to what you said at the beginning, Marvin Minsky has has detailed this explicitly. Like, it's important for folks to understand the core foundations that are powering the transformative technology that are trying to put into their company. That's right. I mean, the other, also going back to the principal agent problem, the technology leader needs to ask themselves, do I want to be the principal and on the hook for the performance of customer service and sales and product management and finance? Because if all of the visibility and all of the logs and all of the analysis and all of the ability to write new evals is in the technology department, is in my department, then ultimately I am shifting the responsibility for all of those business outcomes to my team.

53:57Right. Is that what you want to do? Or do you want to give your business lines of business a tool for them to manage all those things directly themselves and keep those business outcomes in their own purview? What do you as the technology leader want to do? Because this is where the organizational change and the culture change is going to be massive, because that is fundamentally a question of who is the principle for all of these agents. And that principle needs to have direct control over them, not intermediated control, direct control. Yeah. So whoever has direct control on the agents is the principal and is legally responsible and organizationally responsible for their outcomes.

54:45Yeah. And it's important for teams then also when they're applying this technology to all sorts of different kinds of outputs to like what you just said. It's not just another IT tool on the IT spend budget that they manage and do whatever. It actually becomes a core part of your team. And to kind of wrap up kind of what we've been talking about, I'm wondering, Dr. Mamou, what do you think the workforce of tomorrow will look like based upon this model of working? I can tell you how we are organizing our organization, and that will lead you to understand what I think the future workplace will look like.

55:21So we are right now at Wayfound a team of 30. We have seven humans, homo sapiens, and 23 AI agents, AI sapiens. we view ourselves as a fully multi-sapiens workforce, right? Multi-sapiens meaning that we really think about, you know, what are all the things that we're doing and what are the things that should be, you know, that are appropriate work tasks for the homo sapiens and what are the appropriate work tasks for the AI sapiens? And, you know, we're really, I mean, we have the benefit of being a native gen AI startup, right? So we started last year, our whole product is AI agents. So we had the benefit of completely rethinking our organizational design based on this multi-sapience understanding of how we're organizing our work.

56:14And so we don't have product management really in our company because all the AI agents manage the roadmap, synthesize the customer insights, write up, you know, all of the product requirements and interpret them right from our conversations for the engineers, make sure the engineers don't miss anything. So AI agents kind of are doing all of that. We're probably not going to have like a traditional marketing person or an outbound salesperson, because again, we have AI agents doing most of that work and optimized to do a lot of that work. But we will have growth people, And we will have, you know, we're calling it design and product.

56:58But I think it's more about the look, the feel and the identity, right, of the company, right? Tiffany Chin, she's really responsible for the entire identity of Wayfound. What is that role? It's a little bit marketing. It's a little bit design. It's a little bit product design. It's content generation. It's a lot of different things. But, you know, that human who has an incredible sense of aesthetics is responsible for that, right? Because I'm not going to trust an AI agent to do that. And then for growth, it's really about relationships, right? There's a person who's responsible for building human connection and human relationships.

57:38The AI agents are doing all the rote sales work. So we don't need like a salesperson per se. But we do need someone who's great at building human connection and human relationships. So, again, we're rethinking our whole company. And because one of the reasons is, one, it works for us. But also, this is where I think companies are going to be going. This multi-sapiens organization where they think about what are the humans really good at, right? How do I find the human with great taste to be in charge of taste and identity, right? How do I find the person who's great at human relationships and human connection?

58:14How do I find the person who's really great at seeing the future of technology and bringing in those tools and those systems and then building the agents around it? Like those are the core human capabilities. And then we have the AI agents doing all the other stuff. That's super fascinating. It makes me think of, you know, this phrase that we all say all the time of like, oh, the LLM does that. And it frees the person up to do the more meaningful work. But as an AI-native organization, you got to design that person's role from the ground up to only be doing that most meaningful work. So I think that's a really interesting glimpse into workforces of the future and companies of the future in an AI-native world.

58:59And, you know, Dr. Moo, this has been a super fun conversation for me. We've gone into some research. We've gone into some real-world applications. And you've connected the dots between how folks are using this technology and how it ultimately relates back to core organizational management principles. And you've given us an effective playbook to have that conversation internally and to build that right culture for success. And you also unpacked that news around the sycophancy update, which I had definitely been wondering about. And you kind of helped orient folks about what that means and how they should be using these tools.

59:30But before we wrap up, where can listeners learn a little bit more about Wayfoundland and what you're working on? Absolutely. So the best place is to follow us on LinkedIn. So I'm Tatiana Mamout on LinkedIn. We have a WayFound page as well on LinkedIn. We post frequently and I do speak frequently as well because I am very passionate about where the future is headed. I hope people do not have fear around it. We all have to just get together, adapt really quickly to this technology, challenge the ways that we're thinking, that we're working, that we're living. And the faster we adapt, the faster we create an amazing future for ourselves.

1:00:09So that's why I'm out and speaking all the time. Dr. Mahmoud and I are very active with this conversation online. And we want to hear your thoughts about what we talked about today and what you're seeing within your own organization. I think going back to what you just said, this is a conversation we all need to have, and we need to be having it quickly and often. And so thanks for joining us today on Dev Interrupted, and we'll see you next time.