Atlassian CEO on the SaaS Apocalypse, AI Agents & What Comes Next

20:01Or maybe like Intuit, again,

20:05Alex Rampell:this is where I think it's so fascinating. It forces us to rethink our businesses, right? Is Intuit filling out the tax code for you? Or does Intuit know the tax code as well as anyone else can? What they're helping is you to take your life data, your understanding, they're asking you the right questions. Intuit's almost more like a McKinsey. It can be considered that way. It's their process and their special ability is how to ask you the right questions to fill out the tax code rather than the filling out of the tax code. Yeah, that's true. And all these businesses are having to look at, maybe I have 50 processes internally that I think are my secret sauce and unique.

20:39Alex Rampell:Maybe only 20 of them are, but now I have to really consider which of those processes are actually unique and which are not because we haven't had to think about it in that manner before.

20:50Mike Cannon-Brookes:I think it's also kind of a question of how, like there's this Goldilocks zone probably of like, is it worth doing yourself versus not? Like if you take this kind of like third, not third rail, but kind of this independent variable of should I now cloud code myself some X? Well, if it's like 99 % of my cost and like my business is going to fail because this evil company is overcharging me for software, it might make sense. If it's like a dollar a year, it probably doesn't make sense. And then not all systems of record are the same. So like, you know, I kind of think of a system of record as like the atomic unit of something for a business.

21:27Mike Cannon-Brookes:Like it could be calendars are a system of record for time. Or I don't know, ERP is a system of record for inventory. Like you have all these different systems of record. But like the example I was giving somebody is, if I have an office in Miami that I don't go to very often, and there's a system of record for conference rooms, there is a system of record for conference rooms. It's like Google Calendar. Like, am I willing to change that system of record? Yeah, because it's like Miami office, I only go there once a year, like who cares? versus like this is something that touches my revenue. It's not that expensive.

22:03Mike Cannon-Brookes:Am I really going to grow my own food for something where, I mean, actually, this is the cool thing about like farming, right? If you kind of take that metaphor, it's actually a lot cheaper to go to a restaurant. If I just want like one hamburger versus like get myself a cow and feed the cow and wait till, it's just a lot of food is actually cheaper if you consume it in a restaurant because of comparative advantage and economies of scale. So there probably are systems of record where it's like there's some where outside of any of the factors that we're talking about, they're more susceptible just because they overpriced or they're just not as valuable in terms of what it is that they're storing and keeping records for.

22:42Mike Cannon-Brookes:I mean, like Carta keeps track of cap tables for a lot of companies. How often do you access your cap table? Not very often, but it's super valuable. You can't F that up, right? Like I'd probably rather use Carta for that and they don't charge me that much money. Like, sure, I'll use Carta. And it's not like a daily use kind of product. So it's not even like that dimension.

23:03Alex Rampell:I think the vibe coding thing is so fascinating to me because, yes, there's someone in software, they're like, oh, people are just going to vibe code all these replacements to tools. I'm like, the idea I would vibe code my own workday and then run it is terrifying. I have some really smart engineers. Firstly, I have other stuff for them to do. Secondly, I'm like, wait, I feel like that has way more downside than upside for me. However, and so that's the sort of replacement theory. There is a great gain we are seeing internally in extensibility of software using things like Vibe Coding. So most of these applications are highly configurable, customizable, you know, in our case, all the way through to true extensibility.

23:43Alex Rampell:You can write pieces of software, apps that run on top of our platform that have all sorts of different areas. and lots of customers do, but those customers need to put a technology team on doing that job. Their ability to quote-unquote vibe code extensions, customizations, very tailored applications to their very specific use case of something. I want an app for the Miami team to do conference room booking and Miami has some weird HR policy so that app needs to look at Workday and this and that. It's used by 20 people. I probably wouldn't have been able to afford to put the IT team internally on building that because the bill would have been too big.

24:21Alex Rampell:But now maybe I can build that, right? But that uses Workday's data and rules around the world underneath. But it just gives me a very custom interface for, I don't know, the person on the front desk in Miami to do something very specific to what they need. That is super powerful, but it's not a replacement for poor Workday. I feel like Anil is like the butt of a lot of these conceptual examples. That's really powerful, right? That actually makes Workday stickier in the enterprise and more valuable because you can build all these applications on top, which is the power of AI and vibe coding and creativity to make it more tailored for what I need.

25:00Alex Rampell:But we're going to have to be really careful about these sort of layers of stability and rules and process versus customization, right? And you could argue, I don't know, OpenCore and stuff is an example of building very personal apps just for me. Most of those people aren't software developers. They're building apps that work just for them on top of their Gmail or something else, right? But it still uses Gmail as a Rails. They still go to Gmail to read their email and do their email, but they build some specific thing for themselves to solve a problem they have and probably only they have. A couple of them maybe turn into companies.

25:32Alex Rampell:Most of them are just solving some stuff that they needed themselves. That's it. And that's great. That's really powerful. That's why I'm curious about,

25:39Mike Cannon-Brookes:maybe I'd call it my bucket two of this pricing fairness where the backend is not the front end. So if you think of Salesforce, they charge for licenses. Like I think we have 600 people at our firm, might have 600 Salesforce licenses. I've never logged into Salesforce, but I bet we pay for me. But I use the output of it sometimes because it actually is the system of record, not to overuse that term, but it stores like all of our relationships. But I am like part of a table in a relational database of it's like, you know, I'm user ID number 422 here. And then whenever I meet with a company, like, oh, well, like, user ID 422 is matched in this other database, but we really just want to pay for a database.

26:25Mike Cannon-Brookes:So like in a world where the front end is not the back end, I mean, that's the thing. It's like for Workday, I kind of think they've come up with a very clever pricing trick. The trick undersells it. I mean, I think it's a powerful pricing paradigm that feels fair. It's like the more employees that you have, and why is that fair? Because GE has more profits than a 10-person company. GE's going to pay more for this thing. It's still a drop in the bucket. It's totally within the Goldilocks zone of pricing. I don't think anybody's going to vibe code that. They're going to add all this AI revenue.

26:53Mike Cannon-Brookes:But most importantly, their pricing feels fair. Whereas for these things where it's like the front end is somewhat divorced from the back end, that one is, I don't know what's the fair format for pricing. What will happen to software pricing? And obviously, if nobody's going to vibe code their own thing and there's not going to be any competition, then pricing will stay unchanged. But you can imagine a world where people are building things on top to read from the database, right? Because, I mean, a system of record has a database represented. That's like the abstraction layer beneath everything.

27:26Mike Cannon-Brookes:Will the pricing, will there be any pricing pressure on any of these categories? And for me, I think it's like if the front end is not the back end, there's more susceptibility than if they're like very, very tightly intertwined. Like QuickBooks is used by small businesses. They don't have seats. It's like the owner of the business just logs into QuickBooks. So the front end kind of is the back end versus, you know, Salesforce, where you can imagine like nobody gets rid of Salesforce, but maybe they have fewer seats because they need fewer front ends, but they really still need the back end desperately.

27:58Mike Cannon-Brookes:They're not going to go, you know, they're not going to eliminate or do anything with the back end.

28:02Alex Rampell:It depends on a way. Like, I think your fairness and optics in pricing are really, really important. People understanding what they pay for and feel like what they pay for is

28:16Alex Rampell:relates to their usage in some broad way, right? I would say that a 10 ,000-person company paying for workday, the 20 ,000-person company probably plays twice as much plus some discount because they're buying more because they generally have twice as much complexity of stuff and they see that as fair. That's what you mean by like, it seems reasonable that I would pay by employee for my HR system. I think the question with a lot of these things is, you know what what processes when we talk about front end and back end as an example it's not a database it's a database plus a set of processes we used to call it business logic when i was growing up that those business logics are not irrelevant so in the world of what why does a business have them because it runs as a collection of processes and they want standardization of process to some level right so the two teams work the same way so someone can manage them understand them, track output.

29:08Alex Rampell:You know, I don't know if I have a bunch of car factories, I want to track the total amount of cars in and out consistently across them. The business logic where it gets baked in is somewhat where the value is. Because you may need, and again, maybe A16Z is not a great example of a Salesforce customer, right, that actually has a huge amount of sales going on in terms of traditionally. the processes you bake into that for your sales teams are totally valuable to you and you would think that's a fair way to pay. The question is your sales adjacent teams, the sort of collaborator rather than the core user, how much do they need those processes and how much do they not?

29:52Alex Rampell:So I don't know, I assume Salesforce, Sales Cloud, I guess we're talking about, Sales Cloud has an MCP server. That MCP server doesn't go to the database. It probably involves your processes and the rules on the way through. So the question is someone sales adjacent, I don't know, they're in marketing or they're in customer success or something like this. If they need those processes and governance and controls and rules and, you know, hey, we only do X for customers in Japan, we do Y for customers in this area, that sort of stuff. Even their MCP server is going to need an account. Whether the customer thinks that's fair, that's a different question, right?

30:26Alex Rampell:It's the challenge of like, how does that get priced? I'd say, because we get this all the time, I'm talking about consumption-based pricing, usage-based pricing, outcome-based pricing. There are a lot of categories where that makes sense. I definitely do not believe that it will be the majority pricing manner for all software, for all SaaS-based software, because when you talk to customers, they hate it. They really hate it. Where, asterisk, it is not related to the value they consider that they put in. So I have usage-based pricing for Splunk. If I set them twice as many logs, I pay more money, I get it.

31:01Alex Rampell:But the logging is up to me. Right? I can log more. I can log less. I can yell at teams where I'm like, hey, how come you're logging so much? This is expensive. And, you know, are you using these logs? I can control the amount of data I put in. Same with storage in S3 or something. Canonically, I put in a gigabyte. I put in two gigabytes. Fine. Right? The problem is those are relatively transferable and controllable by me as a customer. A lot of the examples people give of either outcome or consumption-based pricing are not in control by me as a customer. and not exchangeable. So the AI token world, the AI credit world, is really, really difficult for customers because I'm like, I don't really understand what this casino token you've given, casino chip you've given me is, right?

31:45Alex Rampell:I can take a gigabyte from AWS and go put it in Azure and I know how much they're going to charge me because the gigabyte is kind of constant. When I have these AI credits, I'm like, I don't know if your credits are the same as yours or the same as yours. And by the way, you keep adding features which chew up my credits because my users use them. And I'm like, wait, I don't know what they're doing with those credits. Like, it's not the company choosing to use them. It's the vendor adding, like, features that make the software better that seem to just happen, right? I can 10x my customers' credit usage overnight by adding a whole bunch of stuff.

32:19Alex Rampell:Like, hey, I built these great summaries for you. And they're like, wait, I didn't do that. So I think the outcome-based usage, when you talk to customers, they want seats. probably because today they understand it. And secondly, they've been burned by a lot of this consumption base that the bill just goes up massively and they're like, wait, how do I control this? Right, it's predictable. That will take some adjustment. Yeah. It will be certainly present in a lot of categories. You know, we have a bunch of areas of our business at Atlassian that are, you would argue, consumption-based pricing or literally just consumption-based pricing.

32:49Alex Rampell:But we try to stick to areas where customers do twice as much stuff, they get twice as much value, they pay twice as much money, and it's in their control. A lot of these other things aren't in their control. And the last example of outcome-based pricing is those outcomes are also dynamic. So the problem with, say, customer service where I've saved you, you know, you used to spend$20 on customer service. With our tool, you'll only spend 10. That's a great sales pitch in year one. In year two, the customer goes, but I only spend 10. Now I want to spend five. Otherwise, you didn't deliver any value.

33:20Alex Rampell:And the vendor goes, well, if you took me out, you'd be spending 20. And it's like, wait, but I don't spend 20. I spent 10. So like my ability to save you money each year is difficult from an outcome basis, right? I'm eliminating tasks.

33:33Mike Cannon-Brookes:I think also like from a sales perspective, I've started two payment companies and it was really, I used to, this is why I know Workday is I envied them and I would talk to my sales team about Workday because they know from the outside in how much money they make from GE. They're like, okay, GE uses PeopleSoft. They have 330 ,000 employees. maybe we charge them$4 a month but probably$5 per employee per month this is how much money you make from that account and it's so much easier to scale a sales team if you're selling a software product or anything by the way if you know that company will pay us$3 million versus like you know when we were starting a firm we signed up 1-800-Flowers we have no idea how much we're going to make from them and it turned out like you know what really made the business work Casper the mattress company It's like, what?

34:24Mike Cannon-Brookes:Like this stupid map? But it's like, you just don't know. And you think like you get like a big deal. Like we got Walmart, didn't really work out that well in the beginning. We get Casper, the mattress company. Oh my God, incredible. Workday has the, it's predictability in both directions, right? It's predictably for the spender of the money, which is the customer. But it's also the predictability for the management team, knowing that you should spend your time trying to sign up GE and not sign up a 10 person company because GE is bigger than a 10-person company, whereas it's crazy in internet land where it's like Stripe might make more money from a 10-person company than GE.

35:00Mike Cannon-Brookes:And I guess you can get to higher levels of predictability there, but when you have outcome-based pricing or consumption-based pricing or something, I mean, consumption-based pricing is not bad per se, but if you don't know from the outside in how much you can make from an account, it just becomes exponentially harder to scale a sales and marketing team because you just, as an entrepreneur,

35:20Erik Torenberg:Mark, one thing I want to go back to sort of dealt with how you guys are adapting in this era. Can you share more about the biggest ways in which that's manifested for you and, you know, how it's made you change your business?

35:33Alex Rampell:Look, I think the way that we think about it is we, look, we sell collaboration tools that solve human collaboration problems, right? In lots of different areas, service teams, broad business teams, HR finance, software teams, like lots of different types of teams by different sets of apps from us, collections and sets of apps. Fundamentally, they're all collaboration problems that involve a lot of text. So this is really good for us. What are those people doing is probably the important part, right? The technology world often runs to, we're going to reinvent everything and that's the way of the future.

36:10Alex Rampell:and that generally is true in the medium to long arc of time. Our challenge is always we have a lot of customers that work in today's manner, today's workflows, in today's set of apps, and they're very smart. They want to get to tomorrow, but they also have to move a lot of people. So when we're building AI features, and I can give examples of any of these, we need to understand what that technology is, how it can help us. That's how we think about it, firstly. Secondly, what fundamental platform componentry do we need to build for whatever that future will be? Because this stuff's accelerating so fast, right?

36:42Alex Rampell:So that's how we got to our AI gateway and the teamwork graph and the enterprise compliance and controls. You have to separate that out from the features you're building for customers in a given app. Then have to build features for customers that they use, right? So where do you put those features? What are those features? A whole bunch of them are in existing workflows to help the customer do that existing workflow faster, better, higher quality, more efficiently. Those tend to be very unexciting from a magic point of view in terms of what sells a 30-second animated GIF on X, but they're incredibly exciting from the customer because they can use them today.

37:21Alex Rampell:Like their existing way of working just got better. They're like, this is amazing. Like they rave about that stuff. And in the AI world where I'm like, but that's pretty simple. And it's like, but it actually helps them today in a massive way. I tell people internally though, and you can give an example in service, that's not enough because you also need to use their existing workflows with new apps or look at new workflows and be able to handle that as well, right? So we have to do all of these things. So if you look at, you know, Jira's canonical example, you know, in the service collection, in our HR and IT service management products, summarizing a ticket is something we can do way better than we ever could because there's a lot of existing workflows we have in an enterprise, maybe a four or five, of six people work a ticket internally to try to resolve a problem.

38:06Alex Rampell:The fourth person that shows up, there are a whole lot of attached files. There's a lot of conversation. There's a lot of different things going on. They would normally have taken 30 minutes to like read it all and understand what's going on. So then they can bring their expertise to bear on the problem. Literally just summarizing that, and it's not a simple stick it into, you know, an LLM and get back summary. You have to be very careful about the context is so powerful for them, but they haven't changed their workflow on IOTA. It's still Alex saying, hey, Eric, can you come help me with this ticket.

38:33Alex Rampell:Eric shows up. Eric has to bootload his brain with all the things. So that's like an existing workflow where we can use LLMs just to make that customer way better. And they love it, right? They rave about all these types of features. But they're very simple. They're usually not agentic. Then we can say, cool, but that service workflow, we need to put agents in at various spots, right? And most people are taking a workflow and finding, you know what, this step trips us up a lot. This costs us a lot of time. Can we make this step faster? and that's absolutely something that we have to provide agent frameworks ourselves.

39:05Alex Rampell:We have a pretty great agent framework that uses all the teamwork graph and all the context you have. It's pretty simple. It's pretty, very affordable. Or you bring your own agent framework, right? Most businesses, I think, will have three to five large-scale agent platforms running internally and they say, hey, I use Agent Force for this or I use Gemini for this. Great. Bring that agent and we'll pop it in the workflow here and we'll make that work, right? We have to be able to do that. But you're still all in the existing workflow world. You're just doing the old task and then doing kind of a new and efficient task, but in the existing workflow.

39:34Alex Rampell:Then you get people like, what if the service ticket didn't exist at all, right? So you're reimagining whole categories of software to new workflows, and we have to help our customers make it across that gap because they don't generally have one service team. They have hundreds, right? And if they have hundreds of different service desks running, they might say these 20 are going to work in this new way, but they have to manage them all. So I guess we're trying to bring data in the teamwork graph together with this and also from a customer-driven lens. I think that often gets left out here, right?

40:10Alex Rampell:We're trying to take them five years into the future. It's our job to actually get them one year and two years and five years into the future simultaneously, which we're trying to do. And the last thing I'd say is we're investing a lot in design. And I think that always in any conversation gets left out because there's a lot of foundational design to do in how this works, right? We're seeing the first elements of this. But if I look at the mobile era, the first set of apps were kind of just canonically taking desktop or web things and sticking them in a phone. And then we evolved new patterns of interaction and experience, right?

40:45Alex Rampell:Not even the visuals. How do we use these things? What push notifications for? They didn't exist at the start, right? Drag to refresh is like a very obvious, simple example. It's a pretty canonical design pattern that generally it's successful here and it gets moved across. But the whole like, how do I use my mobile and my desktop together? How do I move back and forth? We have so many design challenges to solve that actually help people to understand what's there. The average customer we have, the average user, they don't want to understand. If AI doesn't exist for them, that's fine. But they want the outcomes of it, right?

41:20Alex Rampell:They don't need to know all of the technical detail. It's our job to hide them and just give them the answer they're looking for or make a task more effective or efficient. I feel like in the technology world, sometimes we get so obsessed by like model quality. You know, it's almost trite now to say the models are far ahead of the actual value they're delivering now. The underutilized capabilities are so big. A part of that equation is actually design and experience. Right? How do I get this? Give people a chat box that can do unlimited power and they're like, uh, tell me a dad joke. like it's like unlimited power but it does it's very hard to help them utilize that power which is where a huge amount of our challenge goes in terms of bringing agents and all the the power of them into workflows and collaborative loops and and having humans and agents work together

42:07Mike Cannon-Brookes:i i love the skeuomorphic point on both you know it's first it's like you had pieces of paper the early web was just like a web page that's why it's called a web page it's like eight and a half by 11, right? And then mobile, oh, we'll make it a tiny webpage. And then it turns out if you don't just go into the schemorphic world, but you just think from first principles and take advantage of the power of the device, you do all sorts of other things. It's like, you know, the scroll to refresh, right? Like the pull down to refresh. That was a new concept that came from mobile, right? So I was thinking about this the other day.

42:37Mike Cannon-Brookes:Have you tried Nano Banana 2? Yes. It's really good, right? So one of my colleagues just said, hey, for an American tourists visiting Japan make an infographic about what to do and not to do. And it's like, it one-shots something that's amazing. How do you edit that output? Right? And that's where it's like, you know, it feels very, it's like, well, you could edit the text, you could edit the graphics, you could just one-shot something new, or, you know, what is the state of, I guess this is my question for you, is like, what do you think the state of the art is or should be? And how have you been thinking about this, just because you mentioned design, for editing the output of the the AI output, right?

43:18Mike Cannon-Brookes:Because they're like, they're the classic, it's like, oh, I'll use a GUI and click here and change that. But it feels like that's very skeuomorphic.

43:27Alex Rampell:I would zoom out two levels from that to answer that question because it's a great question. First is, customer trust is really hard in these areas, right? When you go talk to users, you sit down, you do research with them, you sit, you ask them questions, you ask the five whys. they're very scared of AI, not because of its power, because it does stuff and they're like, hey, how do I know that was right? What did it do, right? It's like the idea that, oh, don't worry, my AI bot's gone and sent 15 emails in Manager Inbox. Your inbox is empty. And you're like, okay, did it, I don't trust it yet.

44:01Alex Rampell:So I have a trust question on generally AI doing things really quickly. To gain trust, it has to come back to you and say, here's what I'm about to do. You sure you want me to do this? without being annoying, like just effing go and do it. So like that's a whole design question. How often is it? How do you build trust with any of these tools? The second is, does it have enough data, right? So much of AI is one-shotting things. Sit on X, you'll see a thousand like, hey, this is the magical prompt incarnation, Harry Potter spell that does this, like runs your one person billion dollar business. Just put this prompt in and paste it.

44:35And like, that's like kind of ridiculous

44:38Alex Rampell:because the reality is you also have a lot of iteration on the data side, right? One-shotting things is really useful, but you often need to go back and edit the output and the input, right? I'm not very good. I've used this example for a while where you say, hey, go write me an essay for my homework. It'll spit out an essay. And you're like, wait, no, no, it's a history class. They're like, oh, okay, well, let's draw a live an essay. And like, you're actually changing the input and somewhat this is chat like iterations. But if you've ever tried to do that image editing with chat iterations, it's super frustrating.

45:11where it's like, oh, no, you changed the thing

45:13Alex Rampell:I didn't want you to change, and you're going back. You're like, ah. So there's an input design and experience problem. Part of that is how do I have the right amount of context? And then there's an output and iteration problems. Our teamwork graph can access largely all of your organizational knowledge. It's insanely accurate. It's got great search. It's got amazing relevance. And you're like, sweet, I have full organizational memory. Now, the teamwork graph knows that I used to write code in 2002. And it knows that because it has this insane memory. And I'm like, it's actually not useful. Don't use that to answer any query I give you other than one thing.

45:52Alex Rampell:Mike used to be a developer. Maybe a bad one, right? It wouldn't get hired nowadays anywhere. But maybe that helps in explaining something to me in a way that, oh, you have a computer science degree. I can help explain it to you in this way. but I don't want to know all that information. Why is that an input challenge? You kind of see all these boxes at the moment where it's like, search the web, don't search the web, search my organization, don't search my organization. Like you're asking the user to make all these choices that I don't quite understand. That's not in a design flow, right? Where it says, hey, this question, I suspect you want me to do this and that.

46:27Alex Rampell:Is that correct? You see that a little bit in deep research, but it's a bit frustrating. And it leads to this whole like, man, I've got 17 different agents running off and doing stuff. And I'm like, it's like the problem of having a lot of interns. We're like, the problem with having 50 interns is you get a lot of work done. The problem with having 50 interns is they ask you 50 questions a minute. And you're like, all you're doing is answering questions for interns. So there's an input problem of experience that you really need to solve. Then you get to the iteration problem, which in a corporation is much more difficult, right?

46:58Alex Rampell:Because we gave this great example of, you know, brainstorming where it's not usually one person brainstorming. So in our whiteboard and confluence, you can bring in agents and say, hey, I want to brainstorm about this topic. They are really good at going off and getting all the information from your organizational knowledge through the teamwork graph and coming back with a really good brainstorm. And we get better and better at drawing it and putting the cards in the right places and everything else. If you just take that randomly and say go, you lose human input and trust. So actually, usually what happens then is we've got a bunch of data, we're going to have a meeting, we're going to get people together, we're going to go and say, well, what do we all think?

47:38Alex Rampell:Add our intuition, the brain matter, which of these are useful and not useful? And then that information has to go back into some other agentic loop to say, cool, now we've kind of voted, although the voting is like the output of a human process, then you're going to go and do something. Then we're going to work out what to do and did we do it correctly and all these things. It's, as you said, it's very non-deterministic in the quality of output, but it requires, I think, this human agent loop, right? And getting that right is a design problem. Too many loops, it's frustrating. Not enough loops, you lose trust and it just happens.

48:15Alex Rampell:And so we see that we just shipped, you know, agents in JIRA in a lot of ways so you can like assign work to an agent and it goes off and does stuff. And when we test it with people, they're like, Like, well, what's it doing? I'm like, do you want to give us a thousand steps? And they're like, why are you telling me all this crap? I'm like, wait, because you said you didn't know what it was doing. And so there are lots of design challenges with just bringing them into workflows. And back to the business processes, like the, I don't know, the security team is involved in a lot of places. The accounting team, the finance team, there's lots of places, like even in sales, finance usually has sign off on a deal or someone in finance does.

48:48How do you do that and make that workflow better

48:52Alex Rampell:where you're just assigning to agents? You need to be very careful about the experience. How does it come back? When does it come back? Is it frustrating? Does it come back in a new way? Can I interrogate what it's doing right now? Like our agent, first or third party agents, but running Enduro, if they're off doing a task, you can chat to them while they're doing the task and say, what are you doing? Which helps you build trust in the short term, we believe. But in the long term, if you trust it, this particular agent doing this task, man, it's got it right the last 20 times. The odds are right, it's good.

49:23Alex Rampell:it, I'm just going to ignore it. These are all, I would argue, a fundamental foundational design and experience problem. They're not a technology problem, right? They're getting millions of people who use our apps every day to trust this and the gains they get and removing the blank box, I can do unlimited things for you, which just leads to paralysis, I think, for most people.

49:45Mike Cannon-Brookes:It's an open question, right? It's like, because it's clearly like, it's not the yesterday version of like click your mouse here and it's not the today version of just do a new prompt. It's like both, it's like it actually is like a, as long as humans are involved in some way, shape or form, which I firmly believe they will be because these tools serve humans, you need to be able to get your head into the model, both from a trust perspective and from an iteration perspective. And it's a design problem. And I don't think anybody's quite nailed it yet, right? I don't know, maybe they have, but it feels like we're at the very, very beginning of this process of coming up with a better design for modulating, not even modulating, but just kind of editing the one-shots, which impressive as they are today, it's just like, that's not going to be, I just don't believe it's just like, ah, Harry Potter spell incantation.

50:35Mike Cannon-Brookes:I'm going to steal that phrase. That's a good one.

50:37Alex Rampell:I think one thing, one interesting example is just writing documents is something we all do so naturally. And there is a huge design challenge, an experience challenge, which I can describe with AI, document writing. but secondly there's also a huge like people learning challenge so like we sometimes forget pretty much people in technology know what a prompt is and what it does and what the lm's doing in the background you go to people in the broad business world they don't have time to learn all this they kind of probably know what chat gpt is they don't quite know how it's working and the reason it's a design challenge of document creation is um we have a whole set of features which we call create with rovo which instead of writing a document by giving you a blank page and just starting to write, okay, I've got a heading, I put some text in, I put another heading, I put some text in, I put a table, et cetera, right?

51:24Alex Rampell:We've all been trained for decades in KnowledgeWorker to write a document that way. With Create with Rovo, you can literally say, start with a prompt, right? Hey, I want a document that roughly does this or looks like this shape. Give me a template and I'll spit out a template. You can say, hey, I want a document. Can you go off and research this, that, and the other and bring it back? But most of those documents, the research is actually a small category of tasks It's like, help me get started with my document in some way. Teaching users that they should start that way is really, really hard.

51:56Alex Rampell:Once they're running, though, they now have two panes, right? They have 75 % of the screen is the document itself and 25 % is a chat window. Think of Microsoft Word without a toolbar, but with chat only. Now I can type text in, I can edit it, I can change it. And you need to say, hey, you should be totally comfortable change everything on the left. but you can do operations on the right. Like, hey, I want you to add a new section that goes and researches other stuff and put it after like the summary and it'll go and do that. Trying to watch power users are like, this is amazing. And they're like moving back and forth and they're getting the whole paradigm and they're like doing things and they can write commands like, you know what?

52:34Alex Rampell:Make every heading blue, which you can't do in Word. And they're like, bang, it's all blue. And they're like, this is cool. I can kind of give it commands across the document and I can go get more information. and I can like, hey, can you re-summarize it quicker? Or man, how do you think, they can ask questions like, how do you think the board is going to read this document? As a board member, is it simple enough? And it'll give you information in chat that you may say, cool, go action that or don't. It's a completely different paradigm to writing a simple document, which is just at the end of the day, you know, headings and bullets and texts and stuff.

53:04Alex Rampell:And when you watch power users, they love it. Normal people, like regular business users who are very smart, they're like, so I just type on the left that's all I do I'm like well yes it's it's a it's a whole paradigm shift I suspect as we get more of these tools and experiences just like mobile two years now five years now that'll be very standard they'll all say yeah I get how to do this right maybe the first time someone looked at excel they were like wait where do I type the paragraphs or something and you're like oh no you have to think differently about it now it's just like oh yeah I get excel I know how it works um that's the experience challenge we have I think to kill all this power and put it into something as simple as writing a document with all my organizational knowledge.

53:45Like, I get the maths of why that's possible, but now help me actually help people

53:49Alex Rampell:do it. Massive amount of challenge there. Massive amount of excitement, right? When they get it, they're like, this thing is amazing. But it's going to take us a lot of time to get the experiences correct for people to learn. That's a great place to wrap.

54:03Erik Torenberg:Mike, thank you so much for coming on the podcast. It's been an excellent discussion.

54:07Alex Rampell:Yeah, no worries, guys. Hope it was great. a minute, Mike.

54:13Erik Torenberg:Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our sub stack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

54:52Erik Torenberg:Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures.