What Is an AI Agent?

0:03Today, we're discussing one of the busiest and most confusing terms in AI right now, agents. Are they just fancy wrappers around LLMs, full -blown autonomous workers, or something in between? A16Z InfraPartners, Guido Appenzeller, Matt Borenstein and Yoko Li break down the technical definitions, pricing models, use cases, and why the term agent means so many different things to different people. If you're building, buying, or just curious about what agents are and aren't, this episode is for you. Let's get into it. As a reminder, the content here is for informational purposes only. Should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

0:51Please note that A16Z and its affiliates may also maintain investments in the company's discussed in this podcast. For more details, including a link to our investments, please see a16z .com forward slash disclosures.

1:08So I think there's some things which are probably kind of easy to say, which is say, there's a good amount of disagreement. What is an agent? We've heard a lot of different definitions of it on the both on the technical side as well as say on the marketing and sales side in some case, because there's some sales models associated with it. So let's start with the technical side. I think there's sort of continuum here. The simplest thing that I've heard being called an agent, especially just to clever prompt on top of some kind of knowledge base or some kind of context, that has this off a chat type interface.

1:42So from a user's perspective, this looks like a human agent would look like. So for example, I ask it, hey, I have a technical problem with my product, x, y, z. it looks at the knowledge base and comes back with a canned response. But there doesn't have to be a knowledge base. It doesn't even have to be a knowledge base. I see. Got it. So maybe it's just a trained model. It's on the model weights the knowledge. So it's even simpler. So an agent could just be an LLM. But they're chatted in a face or something like that by some definition. I think on the other end of the spectrum, there are some people who basically say for something to be a real agent, it has to be something fairly close to AGI.

2:18Right? It needs to persist over a long period of time. It needs to be able to learn and it needs to have an knowledge base It needs to work independently on problems. If you take them the most extensive definition Is it fair to say that doesn't work yet? I think so. It doesn't work yet. Will it ever work? That's a philosophical question. Alright. Fair. Fair. Fair. Fair. So if we take that continuum in between is at least a way to to to chop that up into a couple of categories of, so if they maybe degrees of agentech behavior. And different types of agent. There's some artsy agent that help artists to come up with new, bezier curves.

2:59There's coding agent, which we like to talk about as the agent of the day. Which we use, yeah. Which we use. There's agent that's just a wrap around half of LLNs. That's right, you know. I may be the contrarian in this group. All right. Look, I kind of think agent is just a word for AI applications, right? Anything that uses AI kind of can be an agent. Now, before we started this talk, I actually went online just to refresh myself about some of the more interesting AI agent perspectives out there. I found a really cool talk from Carpathy that he gave a couple of years ago about agents which I can describe a little bit.

3:34But the really funny part was on the YouTube recommended videos to watch next. It's like AI agents are going to revolutionize your lifestyle and the rise of super intelligent AI, you know, it's just kind of like marketing. And so I actually do think that's what's going on a lot of ways. The cleanest definition I've seen of an agent is just something that does complex planning and something that interacts with outside systems. The problem with that definition is all LLMs now do both of those things, right? They have built -in planning in many cases and they at least consume information, you know, So at least from the internet, maybe from some servers and exposed information through MCP or some of their protocol.

4:13So the line really is very blurry. And it was so interesting about the Carpati talk. Is he basically, he related to autonomous vehicles and said, AI agents are a real problem, but it's like a 10 year problem. It's like a decade problem that we need to work on. And I think most of what we're seeing in the market now is not the decade version of this problem. It's like the weekend demo version of this problem. And this is why we sort of generate so much confusion. You have this kind of poorly defined, nebulous thing that LLMs are kind of consuming themselves over time. And so I don't think anything we have or actually agents is kind of an agent itself, maybe a poorly defined and kind of overloaded term.

4:49But if someone's willing to do the hard work and define exactly what it's like to kind of be a human but in digital form and spend 10 years to make it actually work, you know, that's sort of what I'm excited to see. Okay, so the finding agent is a difficult job. Maybe it's easier to talk about what people use the tools they call agents and what are the different degrees of agent behavior. I wonder if part of a conversation is redefining agent because we all know that agent as a term is not a great term. It means so many things to so many people. If it's interesting to dissect like what do we mean?

5:21What do different people mean when they say agents where are different ways? We could utilize this process we call agents. So it seems to be this, if we're trying to define agents, well maybe even degrees of agent behavior which might be a little easier. There's something like a user interface aspect to it, right? Where something that's a pure co -pilot where basically user goes back and forth with an LM to work in a particular task that's not called an agent, is that fair? There's a little bit the co -pilot's versus agent's UI models. Yeah, I guess like what are the elements we will think that goes into to agent -like behavior.

5:59Like, mass -dimension planning could be one. There could be decisions made by the agent. There has to be LLM somewhere, but curious about your takes. So I think another definition, we heard from Anthropic recently was this idea that an agent is an LLM running in a loop with tool use, right? Which there's two important parts. One is this notion that it's not just a single prompt, and not even just a single static sequence of prompts, right? But something really LLM takes the output of a prompt feeds it back into itself and based on that makes decisions on what the next prompt and likely also went to a board so I hear went to when to complete a task.

6:36I think that for the Relay agents or the more agenteic behaviors. I think that's a reasonably good definition And I think the other thing is by that definition isn't every Chatbot effectively an agent then in this world, right? Like if I go just to chat gpt .com and use their latest reasoning model with web search. It right isn't using tools and like feeding it's outputs into a new prompt in order to do kind of chain of thought. Chain of thought is a little bit in between. If it's just a single prompt that comes back with a result, then it wouldn't have this notion of planning and doing a more long -term concept and deciding itself when it is complete.

7:14If you have a chain of thought reasoning where I'm giving a more complex task, that's so into look at it, I agree. I just think it's really tough to define a system based on what someone says to it, right? Because these are by design unstructured inputs. These systems will accept literally anything. And so, sure, if you tell it, you know, what's today's weather, I would agree that's not a gen tick, right? That's just fetching, you know, from an API. If you ask it, define a new philosophy of weather, right? It'll happily go do it, right? So it's like an agent if you ask it one thing, but not an agent if you ask it another thing.

7:47I think that's kind of a lot of the confusion in the market around this. And if we spoke in the terms that you're talking about, Greedo, of like, hey, this is an LLM in a loop with a tool. Like that's actually a much more productive way to talk about it, I think. Yeah, I mean, that's it. It seems like we're seeing some degree of specialization of user interfaces in software directions, right? There's, let's say, a cursor or something like that, which really emphasizes the tight loop between the user, the tight feedback loop between the user and the LLM and the thing I'm working on. So I want immediate gratification when it do something, and response time matters.

8:20Then there's more the back -end as source code management system type plugins, where it's more about throwing something over the wall, maybe answering a couple of questions, and then you try to maximize the amount of time the agent can work independently. So it seems like, I think you're right that there's no clean system definition split between the two, but there seems to be a little bit of an interface specialization. Is that the first statement? It almost felt like for all the use cases with described, there's one element that all agents have, which is Reasoning and decision. Would you call just a call to L and to say translate this text to JSON?

8:57That's probably not an agent, but then if you ask L and to say hey decide where you know This response goes and route it for me. It feels more like an agent than before So it almost felt like planning, I'm not actually not sure if that's the agent need to apply or it doesn't need to decide. Maybe both. I actually feel like it's like multi step -al I'm change with a decision tree. A dynamic decision. A dynamic decision tree. Yeah, I think that's fair. I think we've all just been nerds -niped. I just think, you know, it's like humanities people love classifying and you know, they draw kind of like fine distinctions between different types of things, entities, whatever.

9:34We're computer scientists. It's like, you know, another is anything wrong with humanities, but we're just not that. So I think we're not well equipped when it's a bit isn't just zero or one. It's maybe something in between and we just talk about it a lot. It's like, try to like coerce it to one value or the other. Of course, agents are more than pure technology. They're also becoming products, which means they need to be marketed and how someone positions their product has a major effect on how they price it. What's more, the ultimate value of any given agent, which is still to be determined for the vast majority of them, is to what degree they can actually replace or simply augment human workers.

10:13The very interesting point, which is, I think there is a marketing angle to agents. I've heard this narrative from a couple of startups that they're basically saying, hey, we can price the software that we're building much, much higher because this is an agent, so we can go to a company and say you're replacing a human worker with this agent, the human worker and makes, I don't know, $50 ,000 a year. And therefore, this agent you can get from the $30 ,000 a year. This sounds really compelling from the first glance. And actually, I mean, there's some value to it in the very early days because it essentially, it's very easy to understand comparative pricing for somebody who's to make buying decision, right?

10:46Now, on the flip side, we all know that the cost of a product over time converges towards the marginal cost of production, right? And so today, if I used to use a translator, maybe to translate a page of text today, I use Chatchy PT. I do not pay Chatchy PT like I paid my translator. I paid a tiny fraction of a cent, right, which is the API, which is the actual cost. So I sort of wonder how much of the agent debate is different by marketing and pricing. I just actually think this is a really interesting topic. What fields can you think of that are actually suffering complete replacement from AI or AI agent?

11:20And this is a setup, I'll warn you. I have another extreme point of view that I'll say afterward, but can you think of fields where this is actually happening? Not completely, but definitely partially, because there's a lot of, for example, voice agents that replace receptionists. I don't know if we should name. Replace people who would get back to customers. So there's definitely a lot of workloads that have been offloaded from the folks who traditionally did the job. But I don't think there are 100 % replaced. They can do something else. But we are seeing headcount growth in some areas are slowing.

11:56So it's not that existing jobs are being replaced. It's more like they're hiring net new humans slower. I think it's exactly right. I mean, I think in a few cases, humans will get replaced by AI. In most cases, two humans will get replaced. One human that is more, by month human, that's more productive for AI. Or then. Or maybe they keep the two employees. Maybe they go to three employees because now they're more productive. Yeah, right. It's just a really interesting question. And the reason I think it's really relevant to agents is, I think part of the ethos and part of the confusion around agents is this idea that we actually will develop human replacements.

12:30And that this thing we called an agent, which by the way is a name for a person, right? Before we had AI, we had people called agents, and we still have all kinds of people called agents. And it just doesn't seem like that's happening, right? Not in the replacement sense, right? You mentioned Yoko with agents. We've always had customer support automation. We've had 1 -800 numbers where press 1 for sales, plus if that's existed for a long time, this is a much better form of that, obviously. Translation is a great example, too, we know. These systems can perform translation extremely well, but you're probably not going to just stick something to Chatchy PT and then publish it on your website.

13:04There is actually work that needs to take place. I think the reason for this is there's just fundamental creative work in most things that humans do. I think from our kind of purchase in Silicon Valley, we can And forget that. Sometimes, people all over the country and doing all sorts of jobs actually have hard jobs and not just hard in the sense of someone's got to do it jobs, but hard in the sense of it does take thinking and human decision making, which I just don't know that AI kind of has what we would think of as decision making or intent. It's a system that still, somebody has to push the button.

13:36It may be running somewhere, it may do a great job, but someone still has to give it a prompt and hit go. And to me, that's a lot of the confusion around agents. We're all thinking at some point a human person with intent and creativity and thinking is going to be replayed. I'm just not sure that even as theoretically possible. It's almost just like a catch -22 to say an AI system is thinking for itself because somebody has to have sort of created. This is old sci -fi, the lost film getting into now, but I actually do think it's a big reason for the confusion that we sort of experience now. It's interesting because there's two types of agent, or are you talking about?

14:08There's one type where the agent is replacing humans, work with humans. Do you think humans can do? there's the other type of agents that's more low -level system processes. They work with each other, they hand off tasks to each other. To some extent, agents are like technical details in the system in that way, but we mean both when we talk about agents. In that case, it's actually a difference between an agent of the function. I think so. I think agent will be multiple functions with our own scene in the middle. If I have a low -level agent and I'm giving this low -level agent a task, and I get back a task result, It looks a little bit like a classic API call.

14:43But was the L in the middle to make decisions on what to do for that API call? So, but that's sort of how this function works internally. Yes, to some degree. Yes. Right? Yeah. So from the outside, would I care? You wouldn't care. It's like most of the time when we see AISDRs, what we talk about AISDR agents, what we mean by that is when the agent can go to the CRN, pull something out, and then filter the list, drop the email, and send the email. So that feels very process level instead of a female level. So that's what I meant. If you don't know how this thing works internally, a classic function in an agent become indistinguishable.

15:25Totally. I absolutely agree. But as a programmer, you'll find the function. You will define agent that does this. I see. Inflmentation. We'll get back to pricing shortly. But first, let's dive a little deeper into this discussion of how interacting with an agent is different than, or similar to traditional software -based functions. So here's one interesting thing to think about on that topic. I totally agree with you, Guitart. And I think you sort of agreed to. It's really a function if you kind of just look at it that way. Shareable, reproducible functions have never really been a thing. This has been one of these long -time goals that people in the market have tried to say, I can just write a function and then anybody on Earth can use it.

16:09We have packages that you can download a whole package with various functionality, but literally just one function that you can share. If you squint a little bit, that exists now with AI, because you have these models that's trained by somebody. Somebody else may download it, fine -tune it, train a Lora, package it up into some new and interesting way. Then it's actually immediately available for someone else to use on hosting services or hugging face or something like that. It does seem to be just an implementation detail, whether you're using an LLM or not. But there is this interesting thing where the model itself takes up so much of that functionality in the function.

16:41And it's just a different kind of animal compared to normal code. It's actually more, it's kind of shared by default in a way, because nobody's going in and training their own model every time they're writing code. You know, it's obviously heavy, right, it's harder to move around. There are all these different characteristics from normal functions that some of which are actually very desirable. Some are kind of, you know, bad right characteristics you don't want, but many of them are kind of interesting. And I think we'll actually see new infrastructure, new dev tools kind of built around this in the long run.

17:06I think it would make sense. I mean, when we go back in time, the last time we sort of invented a major new component for building systems, which was probably networking, right? How we thought about calling a function before networking afterwards changed a lot, right? Totally. Totally. Totally. The complexities of APIs and the infrastructure on that is completely different today. This is such a good point, because now I think about it. I feel like humans are just functions too. Like if you have a thought experiment and then replace L, I'm seeing the program to a human Like how the kind of answers will give to the program is not that different from what L will give to the program So so if we actually all get hooked up to servers one day and can be called as a function from Lambda Then I will agree that a agent's in created that's an agent isn't mechanical Turk exactly that or maybe even your email inbox There's an Amazon.

17:57There's an Amazon goal. Supermarket, a wall bag, and so on. I think they were advertising that it's computer vision models behind the scenes, identifying what you took from the supermarket. But then people found that they hire a lot of people behind the scenes to actually label the data in real time. So the humans in that case are the functions that today may be. Secret agents. Right. Replace by all ends was. Well, but this was exactly my point though, right? There actually is important creative work, even in a grocery store checkout clerk, right? You could naively think, oh, this is an easy job.

18:31Actually, it's not an easy job at all, right? And so you can take this work and kind of shift it, right? And you can squeeze it down with automation. It's up, but it never really goes away. Oh, yeah, absolutely. Yeah. All right. So given all of this, how should companies think about pricing their agents per seat, per token, per task, hint? It might be too early to truly tell. Usually, if you introduce brand new product category, right, you often initially put a pricing that prices against the status quo, right, whatever you replace or augment in some cases. But let's assume we have a diabolical placement, right?

19:07So that's, I think, where this idea from, oh, this replaces a human, which it doesn't. But if it were, right, then you could charge X amount for it. Usually, over time, competition kicks in, right? And you're effectively priced by how much your competitors are charging, right? And you start sort of an erosion, then it depends on many things like how much of a motor you have to have customer lock -in, right, and so on. Long -term, converging against the marginalized, marginal cost of production, right? Which, I mean, look, if I look at most agents today, it's probably very low, right? From any agent, you can purely model and software the couple of LLMs calls, you can run at a very low cost.

19:46The cost is decreasing over time. And I would sort of argue that's kind of already what's happening that in practice most AI applications, and in particular, if you want to call them AI agent applications, they have their sales pitch around, you should pay us X because we're saving you. It's like a classic ROI calculation. That's the value of that. Yeah, exactly, value based pricing. But in practice, I think most buyers are actually pretty sophisticated about what's going on under the hood. And to your point, they know it's pretty simple stuff happening. And so it's like, hey, what does it cost you to run all these GPUs and we'll pay you some premium over that?

20:22And I think that's how a lot of vendors are pricing in practice these days. I mean, long time we expect pretty healthy margins, just like in SaaS, right? Which software traditionally is very good margins. It's so funny because we always advise companies to not price based on the margin, but price based on the value you add, whatever that could be, it could be compared to other vendors on the market, it could be compared to just what it is building in -house. And traditionally for In -Fra, a rule of thumb, not always the case is that if the surface is used by a human, it's a per -seat pricing. And if it's a service is used by other machines, it's a usage -based pricing.

21:00And I actually don't know where to put agents here. So you think? It could be used by either, right? It could be used by either. It could be used by either. I think your analysis is exactly right. And the reality is most AI companies don't know what value they're generating. Yet this is so new and so nascent that it's like, hey, we're just gonna charge something that we're not gonna lose money on. And in the case of open AI, they have how many millions of users. They probably don't have a very strong sense of what they're all using it for. And once they do, right, and you see this more, they're trying to verticalize a bit more and have kind of specific products for a specific use case as code obviously being the big one.

21:36Then you'll be able to see the pricing catch up. It's kind of my hypothesis. This reminds me of the open AI point you brought up. I was thinking about AI companions, because that's the closest to perceived human pricing. Like, you can't charge someone every sentence they talk to their companion, although some of the foundational. They're our services that will charge you per response. I haven't used them, but they do exist. I see. Wow. Okay. So usually it's kind of weird to charge someone like by tokens of how much they talk to the companion, whether they're like a flat monthly fee. It doesn't feel like a true friend exactly.

22:15It's very transactional. This is, look, this is all theory, right? People love sitting around and talking, oh, we're going to charge per person, per task, per, you know, world economy that we rescue. You know, it's like it's all made up, right? I think Guido's thing was exactly right. Let's look at the actual technology underlying what we're calling agents right now where they're being deployed and why and Honestly, the pricing the marketing the sales tactic all of this kind of follows from what they're actually selling if I'm selling something that looks like an agent But I haven't truly figured out the value I'm providing to my users How do I justify the jump to a higher price point when I do figure out that value?

22:50You just need to be selling a solution rather than a product right? This is really well -worn and expertise and enterprise go to market. Code, you can somewhat see the decoupling of price from the underlying technology. Now, because it really works, there's a very clear ROI to people who use it. And so, as a VP of engineering or a CTO, you can look at this and say, okay, I'm actually saving a lot of money and my guys are getting a lot more productive. I can value, I can do a normal job. The happier. Yeah, so you're kind of buying a solution, right? You're buying from a vendor something that solves a problem for you, which again, Microsoft, Oracle, Salesforce, people, and doing forever.

23:22Once we start to see more of that, it's going to be these things that become real products and kind of decouple pricing and look kind of like real businesses, I think. I think it's dictated by the high level applications, so I'll give you an example. So I'm a Pokemon Go player. So for those who have played Pokemon Go, once you've collected enough Pokemon, you are out of storage in your pocket. So you need to pay extra to buy a new bag, a virtual bag that you can put more Pokemon in. and as an infrastructure investor, I invest in storage businesses. And then when I look at how much I need to pay for like 30 extra Pokemon, it was thousands of types more expensive than what storage is.