Transforming Customer Service through Company Agents, with Sierra’s Bret Taylor

0:05Hi, listeners. Welcome back to KnowPriors. Today, we have Brett Taylor, whose legendary career spans from creating Google Maps to serving as the CTO of Facebook and co-CEO of Salesforce, founding two companies along the way, as well as chairing the board of Twitter and now OpenAI. He and Clay Bevor have started Sierra, which is creating company agents for the next generation of customer experience. I'm thrilled to have such an amazing technologist and leader at all scales with us today and longtime friend. Welcome, Brett. Well, thanks so much for joining us today, Brett. My pleasure. Thanks for having me.

0:40Let's get right into it. Do agents work today? How do you define agents? Or do you want me to define agents? You define agent. You're the expert. Agents mean something different in academia than I think they mean in industry right now. I think both definitions are important. Just starting what I view as sort of the classic academic definition is an agentic system is one where software can reason and take action autonomously. And it comes from the word agency. And as a consequence of such a broad academic definition, I think it becomes sort of the proverbial inkblot test for people using the word.

1:14In industry right now, there's probably three categories of agent that I think are on the cusp of working. The first, I think, which a lot of people online talk a lot about is personal agents. And I think that's probably the earliest of the three categories that I see, but maybe one of the more exciting ones. And this is the agent that will triage your inbox, schedule a vacation, help you prep for a meeting, manage your calendar, all of that. And the reason why I think that's earliest, I think it's really interesting to make some demos. But I think the human-computer interaction and even how the agents interact with all the systems we depend on as people is quite complex.

1:52You can think of sort of the surface area of both reasoning and systems integrations as almost infinite. And so as a consequence, I think it probably a prerequisite for a great personal agent probably demands more technology than is currently available. Though there's lots of interesting startups in this space. And you could imagine some interesting companies carving out meaningful niche use cases that expand as the technology improves. The second category of agent, and I think this one does exist in some categories, is what I call persona-based agents. So they're agents that do a job, a very specific job.

2:26You know, there's companies like Harvey that, you know, serve a legal function. There's all the coding agents. I think there's some fairly effective ones right now that serve the job of a computer programmer. I think this is really exciting because I think when you narrow, I call those cases narrow but deep. If you're just trying to... Both task scope and perhaps integration scope. That's right. The tools you access and even how you evaluate the effectiveness is, you know, if you're building a coding agent, there's actually really good benchmarks already. Similarly, compilers emit error messages and you might have integration tests.

3:02You end up with this scaffolding that actually, practically speaking, limits the scope of sort of the true research that you have to do to accomplish it. I think broadly speaking with the advent of foundation models, a lot of effective AI right now is where you've taken areas of research and you've made them areas of engineering. And I think you can engineer very effective persona-based agents for certain domains where the technology applies, like the law, like areas of software engineering, things like that. My take is the domain of personal agents is probably of the very large consumer companies like Apple and Google and OpenAI and others that have big consumer brands.

3:41For the persona-based agents, I think there's probably meaningful companies in each of those spaces because I think to do those effectively, it involves sort of the confluence of AI expertise and expertise in that domain. The other category, which is the area that my company, Sierra, works in is what I call company agents. And it's really less simply about automation or autonomy. But in this world of conversational AI, how does your company exist digitally? I always use the metaphor. If it were 1995, if you existed digitally, it meant having a website and being in Yahoo directory. In 2025, existing digitally will probably mean having a branded AI agent that your customers can interact with to do everything that they can do on your website, whether it's asking about your products and services, doing commerce, doing customer service.

4:31That domain, I think, is shovel-ready right now with current technology because, again, like the persona-based agents, it's not boiling the proverbial ocean technically. You have well-defined processes for your customer experience, well-defined systems that are your systems of record. And it's really about saying in this world where we've gone from websites to apps to now conversational experiences, what is the conversational experience you want around your brand? And it doesn't mean it's perfect or it's easy. Otherwise, we wouldn't have started a company around it. But it's at least well-defined.

5:01And I think that right now in AI, if you're working on artificial general intelligence, your version of agent probably means something different. And that's okay. That's just a different problem to be solved. But I think, you know, particularly in the areas that Sierra works and a lot of the companies that you all have invested in is this saying, you know, are there some shovel ready opportunities right now with existing technology? And I absolutely think there are. Can you describe the like shoveling cycle of building a company agent? Like what is the gap between research and reality? Like how do you what do you invest in as an engineering team?

5:36Like how do you understand the scope of different customer environments? It's just like, what are the sort of vectors of investment here? And maybe it's hard to interrupt, but as a starting point, it may even be worth also defining, like, what are the products that Sierra provides today for its customers? And then where do you want that to go? And then maybe we can feed that back into, like, what are the components of that? Because I think, obviously, folks are really emerging as a leader in your vertical, but it'd be great just for a broader audience to understand what you focus on. Yeah, sure.

5:57I'll just give a couple examples to make it concrete. So if you buy a new Sonos speaker or you're having technical issues with your speaker, you get the dreaded flashing orange light. you'll now chat with the Sonos AI, which is powered by Syrah to help you onboard, help you debug, whether it's a hardware issue, a Wi-Fi issue, things like that. If you're a SiriusXM subscriber, their AI agent is named Harmony, which I think is a delightful name. It's everything from upgrading and downgrading your subscription level to if you get a trial when you purchase a new vehicle, speaking to you about that.

6:29Broadly speaking, I would say we help companies build branded customer-facing agents. And branded is an important part of it. It's part of your brand. It's part of your brand experience. And I think that's really interesting and compelling because I think just like, you know, when I go back to the proverbial 1995, you know, your website was on your business card. It was the first time you had sort of this digital presence. And I think the same novelty and probably we'll look back at the agents today with the same sense of, oh, that was quaint. You know, I remember if you go back to the Wayback Machine, you look at early websites, It was either someone's phone number and that's it, or it looked like a DVD intro screen with lots of graphics.

7:08A lot of the agents that customers start with are often around areas of customer service, which is a really great use case. But I do truly believe if you fast forward three or four years, your agent will encompass all that your company does. I've used this example before, but I like it. But just imagine an insurance company, all that you can do when you engage with them. Maybe you're filing a claim. Maybe you're comparing plans. We were talking about our kids earlier. Maybe you're adding your child to your insurance premium when they get old enough to have a driver's license. All of the above.

7:39You know, all of the above will be done by your agent. So that's what we're helping companies build. And Sierra's initially focused on facing, like, consumer-facing companies. Yeah, the vast, vast majority of our customers are consumer companies. Technically speaking, there's not a huge difference between sort of a B2B company and a consumer company, except for the volume of customers that you have. And I always like to think from first principles about what does this technology enable that was impossible before? And if you think about the typical cost of a conversation, so if you call into a call center today, one of the key metrics for most service teams is their cost per contact, which is like, what is the all-in cost of the labor and the technology to fulfill that phone call?

8:21You know, for most phone calls, it's called$13, you know, to service that phone call. Now with AI, you can bring down that cost to well below a dollar. And so all of a sudden, you've literally decreased the cost of a conversation by an order of magnitude. And so if you're just doing the math on that, what companies would benefit most from that cost? And I'm not sure, depending on the math equation, see the numerator or the denominator. But if you're measuring in millions of consumers, obviously the value is really different for consumer companies. For a lot of consumer brands, because having conversations is a really expensive thing to do, you don't necessarily make it easy.

9:02You know, there's entire websites devoted to finding companies' phone numbers. Because often, in some ways, are really consumer-friendly. You push these towards these digital self-service experiences. I'm really excited about now that having conversations with your customers is an order of magnitude cheaper. Maybe you can do it an order of magnitude more. You know, what does that actually mean? And so with these technology trends, I think you often start with just digitizing what you currently do. But I actually think the second order effect will be, gosh, now that having a conversation isn't a formidable cost center, how do I actually want to incorporate having a conversation as a key part of my customer experience?

9:39So going back to your question, I think that's a much more meaningfully different conversation with a large scale consumer company than it is with, say, a B2B company with 100 customers. It doesn't mean it's not valuable. I just say the level of impact and the difference in the decisions you make are quite different. Can you describe some of the key challenges in taking the capabilities of Foundation Malts today and then making them work in the company agent context? One of the techniques, and I think you all probably talked about on your podcast that's very common today, is it's what's called retrieval augmented generation.

10:12And essentially what that means is you take a large language model and rather than using the model and its innate knowledge from the pre-training process to emit answers, you combine that model with a database of content and you say, use the content as a source of truth. And you ask the model to summarize selected content from that database. And that's kind of a roundabout way of saying if you can ground the agent and knowledge that you provide it, but also you can take off-the-shelf models and integrate it with proprietary business data. So it's a really popular technique right now. I would say that's a really exciting area, but what we found in practice is that broad category of technology investment is woefully insufficient for almost any meaningful customer experience.

11:02If you think about, you know, all of the interactions you've had with brands that you care about, what percentage of those conversations were asking questions? Probably none of them. It's all about taking action, right? It's upgrading or downgrading a subscription. It's returning an order. It's a warranty exchange. It's a, you know, filing a claim with an insurance company. All of those are not only not simply answering questions, but also taking action against probably 10 plus systems of record. It's probably a very complex process. Often that process has both business goals. You know, how do we, you know, prevent you from canceling or convince you not to?

11:43There's probably compliance goals. If you imagine being a, you know, HIPAA compliant, you know, healthcare adjacent firm, you have a lot of restrictions on what you can and can't do. You might be in a truly regulated industry. And all of that means that this idea of sort of building agents that can be grounded in content is a great demo, but actually not necessarily an impactful product. That's the area of technology we've really tried to solve. We are really trying to create a platform where you can orchestrate a process of arbitrary complexity, not simply have agency in the AI, but also have guardrails as well.

12:18Broadly speaking, most software systems for the past two decades have been rules engines that execute really quickly, whether the rules are implemented as source code or perhaps in a low-code platform. And now we're moving to a world of goals and guardrails. And so people, businesses now have the opportunity to express a business process, not simply as a set of rules and a decision tree, but saying, what are you trying to achieve? Where do you want the AI to have agency? I.e., where do you want it to have creativity? And where do you not want it to have creativity? And it's a remarkably interesting technical problem.

12:52It's also a remarkably interesting, I would say, social and business problem. A lot of companies will start out saying, I want to control precisely what the AI does, which is a fine goal. And actually, our platform does support it. But if you do that, it can be fairly robotic. And you're actually removing a lot of the magic that people feel when they engage with things like ChatGPT, which is fundamentally the creativity and agency innate in some of these models. On the other hand, if you turn that knob up to, you know, this is Spinal Tap 11 on agency, you know, you could get hallucinations. It could violate your policies.

13:27Or more subtly, it could just not be a great brand ambassador, you know, for what your brand does. So I would just say that I think there's a really deeper thing that we're trying to build, which is how do you program against non-deterministic creative software? What are the abstractions that we need to build to express goals and guardrails so that you don't remove the creativity and the agency that I think make these experiences delightful? It's why ChatGPT got to 100 million users faster than any service in history. but also you can represent to your board, your CEO, your customers that there's the right guardrails in place.

14:04And then there's like where you actually are comfortable. Where are you comfortable with this AI having agency? So it's a really fun technical problem. I think it's also really a new design problem, almost a philosophical question about where you want to seed certain amounts of creativity to software in a way that just wasn't a conversation one could have more than a couple of years ago. How much do you think those different aspects you mentioned, the guardrails, or in some cases I've seen people working on agents build their own reasoning engines and other things, their own modules that go on top of the core foundation models or LLMs.

14:36How much of that do you think is a company you need to keep doing yourself versus will eventually get integrated into the core model companies like OpenAI or Anthropic or people like that? If you don't mind, I'll zoom way out for a second to give you my view of the marketplace, and then I'll jump into that question. There's a Mark Twain quote, history doesn't repeat itself, but it rhymes. I think the AI market will rhyme with the cloud market of the past 15 years. And if you look at how that played out, broadly speaking, you ended up with a small handful of infrastructure as a service providers that represent the vast majority of the CapEx investment in cloud, i.e.

15:14most software as a service companies pay rent to one of those infrastructure providers like Amazon Web Services or Azure or Google Cloud. And again, because there's economies of scale and data center development, it didn't make sense for a startup to either build their own data center or for a startup to actually build infrastructure as a service business. Just the capital expenditures required and that positive feedback loop on CapEx just didn't work out. I think that will probably play out with the frontier models. We'll end up with a relatively small number of companies doing pre-training, which is the really capital-intensive part of model building.

15:54Not because they're the only places with good researchers, but again, if you look at the CapEx requirements, to actually make a return on that CapEx, you want to lease it out to a large number of people. And then for a lot of companies, especially startups who have done pre-training, They're finding making a return on that is questionable mathematically. Do you think in the long run that just ends up being the main cloud providers or cloud providers plus one or two other players? Because fundamentally, to your point, there's a CapEx and ability to afford it side. The second piece of it is if you're actually running all your application, all the data, everything else on one of these cloud providers, pinging out to a third-party service just adds latency.

16:33So you add the round trip, you add a second sort of buying behavior around approval, budget, security. et cetera. So do you think it's just going to roughly consolidate around the clouds, plus or minus? I do think it will roughly end up the cloud providers in partnership with the big research labs, which is roughly the current landscape. I'm not sure I completely agree on the security and latency front. It's possibly true. It was interesting, I think, that most companies, most large enterprises now use multiple cloud providers. Most of them use software as a service and don't necessarily care where it's hosted as long as the security and reliability requirements are met.

17:15And there's obviously some exceptions to this, but I think thanks to 20 years of software as a service, people have sort of evolved their expectations to not ask, you know, where do you get your power? And just say, what is your SLA for this service? And I think that's probably a positive trend. So I do think there's probably meaningful latency and security issues to overcome, But that all being in the same substrate, I'm not sure I make that leap. I might be wrong. I just, you know, I view the evolution of software as a service having evolved that. But going back to my history rhyming point, I think you'll have a relatively small number of foundation model builders doing pre-training.

17:54I think there'll be a market of tools companies. You know, a great one in AI might be Scale AI. You know, Snowflake was a great example in cloud that might also be an example in AI. and I think all those tools companies, and it's the proverbial pickaxes in the gold rush. If you're trying to transition to the cloud, what software do you need? If you're trying to transition to using AI in your business, what are the tools and software that you need? And then the final category would be solutions. Just like in the cloud era, you can take the services from Amazon Web Services or Azure or GCP and build almost anything, but most companies don't want to.

18:30Most companies want to solve a problem. And the total cost of ownership of building your own CRM or ERP system is nonsensical. And I think it took a long time for companies to realize that, but certainly they have now. I think the same will largely be true of AI. If you want to automate customer service, working with CIRA is much easier and lower costs than building it yourself. If you want to automate parts of your legal process, talking to Harvey is probably a much more logical path than trying to roll your own for all the same reasons it was true of software as a service. So broadly speaking, going back to your question, how do you build technology and what will the foundation model providers do?

19:13I think the higher order bit is what is the value you're providing and how do you decouple and are you adding enough value on top of models to be a real company? And the answer is if every time there's a new release of an AI model, somehow it decreases your value, it probably indicates you're not actually a solution. It might be a slight value add on top of the models. I think there's a number of startups that unfortunately sort of smell like that. You know, it's not necessarily a lot of value. What happens for Sierra when models improve? If we're doing our job right, our platform gets better.

19:44You know, so, you know, I think that our customers, you know, in our platform, which we call Agent OS, are essentially defining the goals and the guardrails of their customer experience. and every time we have new technology available to make that work more effectively, we plug it in and you get better case resolution, better customer satisfaction, fewer negative experiences and that's just great. In the same way when any web service that you provide from a software as a service company just gets better when the technology gets better, that's effective we want to provide. But what our customers are hiring us to do is not related to the models, it's related to their customer experience.

20:21So fundamentally, that's the way we think about it. And as an entrepreneur, I think there's danger if you don't fit into one of those lanes. At least that's my opinion, because there's a real question of when a model improvement comes out, if that was 50 % of the value provided, you're in the sort of uncanny valley of value. But I do think the idea that all use cases will come from foundation models is probably wrong. I mean, it's hard to predict the future right now. But I think that would be the equivalent of saying, you know, 15 years ago, gosh, there's not going to be a single software as a service company.

21:00Everyone's just going to build their own or from the Lego bricks provided by. There were enterprises that said that. Yeah. Yeah. Yeah. Yeah, and I actually think perhaps the opposite came true there. And most businesses, I always like to know, where do you want to innovate? With the relatively few engineers you have, if you're a large retailer, you don't have the resources to implement everything yourself. Where do you want to stand out? Where do you want to stand apart? And for most companies, they benefit from the rising tide, lifting all boats of investing in a software-as-a-service platform.

21:34I just see the same thing happening here. So I'm very bullish on going back to our definition of agents. All the companies creating persona-based agents, they'll obviously compete with each other. But I think there's meaningful companies in that space. And I would probably work with them over assuming it's coming from the foundation model providers because they're solving all the unique problems. This would take a coding agent of developer workflows, of security, of different programming languages, all these things. I actually think there's a ton of value there. And I also think there's probably second-order effects of relying on coding agents, how they incorporate into your team, governance, code reviews, all these things that I'm not necessarily thoughtful enough to enumerate right now.

22:17But that's why there's a company in this space. And I'm very bullish on that company existing for the long term, not even knowing half the names. I think to your point, the analog with SaaS is a really telling one because people always talk about wrappers on foundation models and how those companies will go away. And you could argue that a lot of SaaS is like a wrapper on a SQL database. It's kind of like the same thing. Yeah, I think the same was true, probably said, of Shopify, Salesforce, ServiceNow. Yeah, exactly. And those are all great companies, you know? It's interesting. They ended up being very, like, let's say, the database vendor to Salesforce and being a very important vendor for a long time.

22:50And, like, it did actually get yanked out eventually. For what it's worth, like, what's really interesting about the cloud market, if you buy my analogy, and analogies are always sometimes dangerous, too. And, well, let's say the exceptions to the analogy can sometimes be hidden. But the foundation model providers will all benefit from this investment. I really do think these foundation models have a ton of innate value. And so any solution built on top, the foundation model providers will collect attacks on all those amazing use cases. And that's really great for everyone involved. As you sort of alluded to, different application companies can decide to use different models at different times.

23:28So it creates a lot of probably healthy competition there as well. And the most important thing for, say, Sierra customers is we're future-proofing our customers from that. Both future-proofing so you don't end up in the dreaded situation of something breaking when new technology comes out. But in a more meaningful way, when there is great new technology coming out, can I just turn it on? Can I benefit from it? And I think for a lot of the solutions and applications companies in this space, that will end up over time one of the main values they provide. You know, for anyone who's experimented with prompt engineering or prompt engineering with tool use and kind of, I would say, the low level of these models, it's not like your prompt just work with future.

24:10I mean, the model could be better, but it's not strictly better. It's actually, you know, there's a very tight fit between the tokens and the model and all these things. And, you know, while there's lots of interesting tools there, I'm not sure that's like the layer that most companies should or will want to be operating. Just like your company doesn't want to know that you're doing a database migration. It's boring but important. And what software as a service provides is you don't need to care about that. Just no downtime. Yeah. The other thing that's interesting is business model. At Sierra, we're really focused on what we call outcome-based pricing, charging for the job done.