1. Podcasts
  2. The a16z Show
  3. OpenAI: When AI Agents Use Computers Faster Than You
The a16z Show

26 Oct 2026 · 45m · 17 chapters

Episode artwork

OpenAI: When AI Agents Use Computers Faster Than You

In short

OpenAI’s computer-using AI agents (Astra, Sol, and “Dots”) that can operate software faster than humans, with enterprise controls for safety, auditability, and human approval loops; future focus on superhuman speed, multimodal real-time collaboration, and more coherent consumer experiences.

Guests (backgrounds)

Ari Weinstein and James Sun from OpenAI; they discuss Astra’s computer-use model, enterprise safety controls, and consumer “Dots” workflows.

Key claims

Astra is faster using fewer “thinking tokens,” more reliable (99% vs 97% framing), more aligned, and better at resisting prompt injection. Computer-use agents can perform any task a person can do on a computer, enabling automation across legacy enterprise systems. Enterprises can sandbox agents (allowed apps/websites) and set human-in-the-loop approval dials (e.g., downloads/uploads, new websites) plus audit/evals. Speed is nearing “superhuman” for expert computer users.

Notable examples

Astra drawing in Microsoft Paint; Astra beating video games (e.g., Fallout in ~30 hours); Dots booking haircuts/buying online; Dots flagging missed calendar invites from Gmail; Dots pulling Sentry logs via cloud computer and analyzing with Codex; procurement via spreadsheets into procurement software; finance ledger reconciliation; WebMCP to let agents and humans co-browse.

Written by AI. May contain mistakes.

Chapters

0:00

The Rise of Computer Using Agents

Learn about the advancements in computer using agents and their potential.

“We've gotten computer use now to a place where it's faster than the average human, and I think we're getting really close where it is superhuman.”
2:27

Deep Dive into Astra's Capabilities

Exploration of Astra's advancements and its impact on computer tasks.

“Our computer use models were already so good before Astra.”
4:25

Consumer vs Enterprise Use Cases

Comparison of how computer using agents are applied in consumer and enterprise settings.

“between consumer and enterprise if you were to pick?”
6:13

Innovative Uses of Dots

Insights into how Dots are utilized to enhance daily tasks and workflows.

“Yeah, no, that was going to be my question on dots.”
8:23

Balancing Speed and Control

Discussion on balancing the speed and safety of computer agents in enterprise.

“for at least these proactive and kind of long-running tasks where they're not the same shape every single time.”
10:10

Impact on Legacy Systems and the Web

Exploration of the implications of computer agents on legacy software and web usage.

“in which models people want to use for which tasks.”
14:00

Exploring Agent Capabilities

Discussing how AI agents can handle various tasks and workflows.

“but then the person is kind of using it together.”
17:35

Building Trust with Enterprises

Strategies for making enterprise users comfortable with AI agents.

“Do you think there's anything you have to share with like an enterprise user to get them comfortable with that?”
20:35

Identifying Automation Opportunities

Identifying workflows and areas where AI can automate tasks effectively.

“And I guess I'm curious, like in a broader sense, It's like, okay, now we have like capable models.”
24:22

The Future of User Interaction

Discussing the evolution of user interactions with software and AI.

“Yeah, I think kind of similarly on the consumer side, there's a lot of things where it's sort of automating drudgery or just doing things you don't want to do.”
Show all 17 chapters
28:00

Personalized Software Development

Learn how personalized software can enhance user experience through craftsmanship.

“So like my wife loves, recently got into playing this like, you know, board game.”
29:20

Human-AI Interaction and Software Design

Discover the importance of UI and human-AI collaboration in software design.

“But I don't think it will just be like, yeah, everything will have an MCP and an API and, you know, computer use will never be useful.”
31:00

Startup Considerations in AI

Understand what startups should focus on when integrating AI into their solutions.

“And so, like, any B2B user still needs to figure out, okay, like, why did that payment get made and go back in and get a summary of what happened and be able to say, okay, pay this person at this time.”
33:00

Challenges and Opportunities in Computer Use

Explore the challenges and potential in utilizing computer use for complex workflows.

“So like being thoughtful of like, how can I kind of like, humans are used to collaborate.”
37:40

Future of Computer Use and User Experience

Examine the evolving landscape of computer use and the focus on user experience.

“What are you excited about that's on the frontier that you can share around computer using agents?”
40:00

Growth and Potential of AI Products

Learn about the growth trajectory of AI products and their increasing utility.

“And they have to, the better the UI and controls and the easier and more enjoyable it is to use, the more they're going to get out of it.”
42:04

The Future of Computer Use and AI Agents

Discover how computer use is evolving with AI agents and the potential for broader adoption.

“It's still the case that only some, you know, portion of our user base is using computer use.”
A man walking through an airport, asking his phone a question

Ask about this episode

Get the gist, explore an idea or ask about something said.

Claude

On The a16z Show, what are the key points of this episode?

Searching your listening

Transcript

Automatic transcript. May contain errors.

0:00Fabrizio Serafini: We've gotten computer use now to a place where it's faster than the average human, and I think we're getting really close where it is superhuman.

0:08James Sun: What sorts of tasks do you think are ready to hand over to the agent versus the ones like, okay, it still requires some checking and correcting?

0:15Ari Weinstein: Computer using agents now since Astra is now capable of doing anything that a person can do. We think a lot about like what new consumer experiences can we enable with a faster, better model? And then on the enterprise side, that's kind of naturally why the safety becomes more interesting. How do we give the ability to control the amount of human, the loop there is, and then the auditability?

0:34Fabrizio Serafini: It's unlike any other thing that we've built where the developer has to do some work to adopt it. Someone has to build a connector, MCP server, you know, computer users work with everything.

0:43Seema Amble: Now we have like capable models. They're getting like cheap enough to run at scale. What should kind of companies think of automating?

0:52Ari Weinstein: I think of it as in two big categories. The first category is like...

1:23Seema Amble: across legacy enterprise software, finance, procurement, and sales, alongside the technical advances driving progress, including why the harness surrounding a model can nearly double its speed. They also examine how enterprises can control and audit agents. And what comes next? AI that proactively works in the background, collaborates with us in real time, and eventually navigates software faster than expert humans.

1:52James Sun: Welcome to the A16Z podcast. I'm Sema Amble. I'm joined by my colleague Fabrizio Serafini. In August, we wrote about computer using agents and how they're starting to do the real work inside companies. Today, I'm joined by Ari Weinstein and James Sun from OpenAI, who are going to help unpack what is going on with computer using agents from their perspective and share some thoughts on the future. Ari, James, thanks for joining us.

2:16Ari Weinstein: Of course. Happy to be here. Yeah, thanks for having us.

2:18James Sun: Ari, kick it off and tell us a little bit about what in Astra has made computer using agents move forward and what's really unlocked.

2:27Fabrizio Serafini: Yeah. So Astra is an amazing model. It's good at so many things. Our computer use models were already so good before Astra. You know, like Sol itself is amazing at computer use. So Astra really helps with a few things around the edges in my sort of anecdotal experience. One is it is a little bit faster on average because it uses fewer thinking tokens to do the same task. It's better at certain kinds of creative work. Like, for example, we had this demo where in our launch video where you were talking to it and it's drawing things on Microsoft Paint. It's way better at that sort of thing, which I think probably transfers to other kinds of creative tasks also.

3:03Fabrizio Serafini: And then the last thing I find is it's just more reliable around the edges. It's kind of, I feel like I trust it a little more. It's sort of the kind of thing where there's a little difference that is hard to perceive, but really important between something being 99 % reliable and 97 % reliable. And I feel like it's sort of just a little higher in a way that feels pretty, pretty concrete.

3:23Ari Weinstein: Yeah. And I think adding on to some of those things, Astra is also our most aligned model. Yeah. Right. And so talking about trust a little bit. Yeah. If you look at the system card for it, it is much better at being resistant to prompt injection. It's much better at not taking any kind of unintended actions. And so when you think about it for like enterprise work, especially for companies using it for sensitive things, whether it's procurement or talking to vendors or talking to customers, you're going to feel a lot safer kind of using it. And that's really, really exciting. And then there's some fun things on the creative side.

3:54Ari Weinstein: I think internally we had Astra play video games against 5.6 and it wins every time. And so there are some cool tweets that people have made where Astra is able to beat Fallout in 30 hours or something like that. And so it's also just really amazing at playing video games. So I don't know, that probably translates to something. But yeah, we're excited.

4:12James Sun: Maybe just a double click on that point. Consumer versus enterprise, use cases of computer agents, very, very different implications around kind of like all the scaffolding around, I would say, the computer using agents. How do you guys think about where things are more exciting between consumer and enterprise if you were to pick?

4:30Ari Weinstein: I think our goal at OpenAI is really about building products that's useful for everyone. And so actually it's interesting because Ari is definitely more of a consumer guy here. And I spend a little bit more time on the enterprise side. I think the consumer is always the place where we try to push the frontier on what's possible. With Astra, like Ari mentioned, it's much faster compared to the previous models. And so we think a lot about what new consumer experiences can we enable with a faster, better model. And then on the enterprise side, it's kind of naturally why the safety becomes more interesting.

4:57Ari Weinstein: Because I think at the enterprise, whenever we talk to large companies, I think the use cases for computer use is actually fairly obvious, right? Inside enterprises, there's lots of legacy software that doesn't have any MCP tools. They are software that's been used for 15, 20 years. And so a lot of times it's actually kind of hard to use for a normal person. Lots of UI that is from the last decade. And so you don't even know how to operate some of the key things. And so we've seen companies build skills that kind of teach the model to be able to use them better. And so I think maybe the way to think about it is enterprise is where the capabilities have been there maybe for a little bit longer.

5:33Ari Weinstein: and the companies are catching up and trying to understand how to deal with it and consumers where we're trying to figure out the next types of use cases that's not maybe possible today with software.

5:42Fabrizio Serafini: Maybe you can share them more. Yeah, I mean, the consumer use cases are so fun for me because it's kind of like, how can you put your agent to work doing the things in your daily life that are so annoying and save you tons of time and just have more time to focus on the things that you're excited about? I think it's been really fun to see DOT where each DOT gets its own computer and you can ask it to do things for you in the cloud. It gets a full web browser. It can buy things for you. It can book haircut appointments and anything else you want it to do. So that's been really fun. And I think there's a lot more on the horizon that's going to be really fun.

6:16Seema Amble: Yeah, no, that was going to be my question on dots. I think, James, you probably use a dot to walk through product feedback. It would be cool if you could share more about it. I'm just so curious to know on your end, guys, like how are you using dots and like how does computer use play into kind of what you can do with them?

6:34Ari Weinstein: Yeah, I mean, I think Dots is really cool for a few reasons, right? One is really proactive. And so I've been very surprised a lot of times when in both my personal life and work, when Dots will just flag something that I just completely slipped through the cracks, right? Just the other day. So internally, like on our email system, whenever we get an external email for an invite, it doesn't actually show up on our calendars. And so I end up missing a bunch of like external meetings. And so Dots now is like catching these things for me. It's like, oh, there's this invite that's in your inbox. It's not on your calendar.

7:04Ari Weinstein: Do you want me to like accept it in your Gmail so it shows up? So that's been a lifesaver. It's a small thing, but it matters a lot. But I also have it proactively kind of looking at, I think it's really good at proactively looking for tasks where it's the shape of it is not super, let's say concrete every single time. So an example that I also have it set up to do is we use Slack internally. And there's a ton of messages that's going on at all times. and I used to have this Slack job using automations and codecs that's every hour or every 30 minutes go and try to pull things. And I've just ended up tuning out a lot of the notifications because it's just sometimes useful, sometimes a dot.

7:40Ari Weinstein: With dots, I can give kind of a high level goal. It's like, hey, if anybody in my reporting chain says anything, flag it to me, right? If anybody who's giving some critical feedback, use your judgment about certain products, go for it. And then because it has a cloud computer, I've had this pretty cool workflow where when someone reports, They usually have a Sentry log. I could have it go use the cloud computer, actually drive my local browser while the Dots can control both your local computer as well as the cloud computer, log into Sentry, pull the logs, analyze it, give me feedback. And so it's saving that entire loop of figuring out which feedback's important, looking at it, then going to Sentry, downloading the logs, feeding into Codex, asking for thoughts, and all that can now just kind of be automated.

8:21Ari Weinstein: And so, yeah, Dots is great for at least these proactive and kind of long-running tasks where they're not the same shape every single time.

8:27James Sun: We talked a little bit about reliability, speed, token usage, and efficiency. How do you guys think about balancing all of these things as you keep making improvements on what you're releasing?

8:37Fabrizio Serafini: I think we're really excited about computer use at OpenAI. We've invested in it a lot, and we've gotten to a place where I think we have the fastest, cheapest, and most reliable computer use product. And I think there's actually so much further we can take that. We spent a lot of time looking at speed. Speed is really interesting because on one hand, the power of computer use is it can do tasks in the background. So in some cases, it doesn't matter how long it takes because you can be going and doing other things while it's accomplishing the task for you. But also sometimes you want to watch it to make sure it does it correctly.

9:08Fabrizio Serafini: You need to wait for it because it needs to complete this before something else can happen. And so the speed is so important because you don't want to get this feeling like, oh, I could have just done that myself and it would have been so much easier. So we've gotten computer use now to a place where it's faster than the average human. at using a computer. And I think we're getting really close to a place where it is superhuman, meaning it is faster than an expert computer user like probably one of us at this table. And I think that's going to actually be a sea change when that happens. I think it's going to change the way we all interact with computers because all of a sudden, anything that you could do manually, it could do faster.

9:44Fabrizio Serafini: So unless you really want to do it yourself for some reason, which I think actually there may be many reasons people would want to do that, but in a default case, you probably just want the agent to do it for you. And I think that's going to let us build all kinds of cool user experiences. So speed is something we think about a lot. And then obviously we have a strategy of offering multiple different models at multiple different qualities and price points. And so for different use cases, you may want to use a different model. At the same time, our less expensive models are getting much better at computer use over time.

10:09Fabrizio Serafini: And so I think there will actually be more variation over time in which models people want to use for which tasks. I think that's what I was saying.

10:16Ari Weinstein: Like, I think the other thought is like, how do we give customers, you know, control over the safety, reliability, autonomy versus speed, right? Like if you have like, you know, when the model is slow, you can kind of watch it do stuff, but then you're limited to that, to the tasks that you can do because you can't just sit here watching the thing doing it. Like it's actually less effective, right? Because it's slower than you. But once you can get to a point where you can no longer observe and audit everything it's doing, how do we give, especially enterprises and businesses, the ability to control the amount of human the loop there is and then the auditability, right?

10:53Ari Weinstein: I think a few months ago, I was part of Astra a few months ago, it's part of Astra, you know, we kind of launched a set of enterprise controls that let businesses choose one, like how much human the loop is there? Like, do they want the agent to automatically approve everything? We have this really awesome system called Guardian, right? I think auto-approve in the app that is a really smart model looking at every single action that the computer use agent is taking to decide if it's safe, if it's aligned to their original intent, a consumer might be like, I just want it to do everything and I don't want to be asked for approval.

11:26Ari Weinstein: Whereas an enterprise is able to kind of decide which type of actions actually go through. And so I think the control part becomes more and more important as our computer use agents become also superhuman in doing things.

11:38James Sun: Totally. I think as you try to, you know, roll out more enterprise use cases, I think employers are excited about the efficiency, but they're also wary of what's sort of being unleashed. Maybe even on the other side, have you guys seen the response from application software around, okay, now there's computer-using agents accessing my legacy workflow software, and do you guys think at all about that? Are you more focused on just the agent that's being used by the consumer enterprise user?

12:09Ari Weinstein: Yeah, I think there's like, I think this problem becomes really interesting on the open web, actually. I think when it's like within software that is like approved and used by the company, it's actually more straightforward because it's like, oh yeah, 100 % we want to have people automate and do work in here that they don't have to do tediously themselves. But when it comes to the internet, I think you kind of have a few different camps of folks, right? One is we run a business model is not in line with having agents use our product, right? You think about advertisers, publishers, whose business model is entirely on advertisement.

12:46Ari Weinstein: If the agent's using the website for them, then where's that revenue coming from? And I think most places, an agent's ad impression does not actually count towards real revenue. And so that's an interesting question that we're always working through and working with our partnerships team on. There's another question kind of around for the companies who do really want to embrace having computer use agents using their sites. A lot of them are adopting kind of standards that let humans or people and agents to be able to use it together. So like WebMCP is a standard that we're really excited about.

13:17Ari Weinstein: We add a support for it in the browser. And so you can kind of imagine these scenarios where the agent is elevating the experience of using the site, but it's not entirely replacing it. So like, I think browsing the internet is still very fun in my opinion, right? I still like doing shopping. I still like looking at all the hotels and flights and places. And so you can imagine places where the agent does all the tedious work, but it kind of carves out an easier experience where you're doing the browsing. You're like, okay, now I want to finish the transaction and then put in my credit card and do all this stuff.

13:53Ari Weinstein: Then you can hand it back to the agent. And so WebMCP is kind of an interesting way where your interface with the website is still the site itself versus an API, but then the person is kind of using it together. And so those are some thoughts.

14:05James Sun: Shifting gears, I want to double click a little bit on what it means to do the job or get the end-to-end job done. We'd love to hear a little bit from you guys around, you know, what works well? How do you guys define the right narrow repeatable task? You know, and what sorts of tasks do you think are ready to hand over to the agent versus the ones like, okay, it still requires some checking and correcting on the user side?

14:31Ari Weinstein: I mean, generally, I found the model to be pretty capable to do anything that you can do on a computer now. And one exercise that I try to force myself to do is that whenever I have a task, the first thing I think about is, can I just have the agent do it? Now, there are obviously some sensitivities around, if I'm sending a message to someone, I want to feel like it's coming for me. But in general, I think the way I think about it is the computer-using agents now since Astra is now capable of doing anything that a person can do. and there are questions around, I think before, of like, oh, there are some like squishiness around like specificity and requirements.

15:09Ari Weinstein: I've generally found that the model is getting pretty good at whenever you're asking it to do something and there's some vagueness, it will actually, you know, with Astra, I think we launched this like flexible input, right? And so we'll actually ask you about, hey, like I want some details. And if you don't answer it in time, it'll just keep going and then pull the context that's necessary. I think there are some things around, you know, if you have repeatable workflows, it's nice to be able to kind of define the requirements or at least similar to writing a product spec but in general I generally feel like unless there is something incredibly sensitive that you want to be in the loop for I just reach for the capabilities model for everything I do now I'm curious if it's the same for you

15:48Fabrizio Serafini: Yeah it is, I'm trying to build that impulse to reach right because it's so good and sometimes it's so easy to get locked into the old way of doing things

16:01Fabrizio Serafini: I think the really cool thing about computer use as a technology is the universality of it. You know, it works with every application. It's unlike any other thing that we've built where, you know, the developer has to do some work to adopt it. Someone has to build a connector or MCP server. You know, computer use works with everything. And it works by using the application the same way a human uses it, which I think is something worth understanding if you haven't spent much time with this technology. Like, computer use uses computers the same way a human does. It makes the agent have all the capabilities you do, which I think is so powerful.

16:37Ari Weinstein: Yeah, and I guess, like, adding on to what I was saying, like, part of it is, like, I think over time I've stopped thinking about it as, like, am I using them on, like, a computer use task or a non-computer's task? Frequently they're, like, connected, right? Like, the way we think about computer use is that, like, computer use is not, like, a thing you pick up and just say, I want to do something with it. it's really like the universal kind of API that the model has to do whatever it takes. And so if you have an ambitious task and you want the agent to do it, likely the more complex it becomes, the more likely it will probably touch computer use at some point in that journey.

17:12Ari Weinstein: And so like maybe the only time that I don't reach for computer use for some cases is if the task is so simple that then it's easier for me to just go and like click two buttons and do it. But, you know, as computer use gets faster, even faster, then the question becomes, maybe I don't even need to do that. I just, you know, drive voice and it just uses the computer with me. So those are some interesting thoughts there too. Yeah.

17:35James Sun: Do you think there's anything you have to share with like an enterprise user to get them comfortable with that? Obviously, you guys are totally pilled on computer use, but, you know, I think there's obviously like hesitance sometimes on enterprise user of like, okay, well, yes, the human would click through it and probably do it worse. But there is that feeling of like, Like there's accountability and, you know, that person's trained and, you know, all the classic kind of rebuttals. I'm curious. Yeah. Yeah. I have to convince people around.

18:03Ari Weinstein: For sure. Lots of conversation in enterprises kind of over the last little bit. And I think the main thing is we think about it as two layers, right? The first layer is like how do we create deterministic controls? I think that's really what gives, you know, enterprises a lot of feeling of safety is when they can, even though our model is the most aligned ever. And it's, I think, better than most people at protecting against prompt injections and unintended things because it's like just, you know, the intelligence is so smart. But I think there is just a feeling of safety when the enterprise admins can basically say for certain tasks or for certain data sources, just too sensitive to be used.

18:40Ari Weinstein: And so we see kind of enterprise, so we launched was two things, right? One is the ability to be able to deterministically create a sandbox around your agent. So being able to say it can use certain apps, it can use other apps, it can use certain websites and other websites. And then two is, you know, this kind of part of a human in the loop. Like what type of actions does it require a human to be intervened? And so we might say, okay, for any time there's like a download and upload, a person has to kind of approve it. Anytime it goes to a new website, a person has to approve it. Now, again, these are all like dials and enterprise can kind of go through.

19:13Ari Weinstein: And so it's really like that's one. And then two is really just showing the enterprise admins just how, you know, all the evals that we've run internally around how aligned these models are, even compared to like, you know, both internal and kind of third-party evals.

19:27Fabrizio Serafini: I think there's also some baseline stuff that we do across the board to just try to help people build trust with these kinds of products. You know, we know it's so new. It's such a magical and sometimes alien experience for people. And so, you know, we put a lot of work into the design of the product. So you can always see the cursor if you want to watch it perform a task and it sort of animates in a fun way so it's not boring to watch. And then, like James was saying, this is very customizable for enterprises, but also for a regular user. You know, we ask you for permission before using a new application or website, depending on which sort of modes you're in.

20:03Fabrizio Serafini: And I think that helps people sort of not feel like it's going to go do some runaway stuff on their computer. you kind of want to have this trust-building moment where you figure out what it feels like and what you want it to do for you and then you sort of feel more comfortable over time.

20:17Seema Amble: The Codex cursor is so cute. I love it. It's like such a, I know it's a small detail, but like watching it use the computer gives you like confidence and it makes the experience nice. So I think there is a kind of like a user experience point that's worth, that's definitely worth investing on. And I guess I'm curious, like in a broader sense, It's like, okay, now we have like capable models. They're getting like cheap enough to run at scale. In your view, what should kind of companies think of automating? Like what would you, where do you see the most promising kind of use cases within enterprises?

20:56Seema Amble: Are there some workflows, maybe something around the ERP, maybe something around kind of how like, again, humans and agents can work together to get tasks done. but if you had to pick where are you most excited to see your models perform in the world?

21:12Ari Weinstein: Yeah, I guess from my perspective, I'd love to hear Aries. I think of it as in two big categories. The first category is automating tedious workflows. That's the more boring one, and I'll give you some boring examples.

21:25James Sun: Boring is opportunity. Boring is opportunity, yeah, exactly.

21:28Ari Weinstein: Like ERPs is a really big one. All the acronyms you can think of, ERPs, CRMs. All the systems of record. Yeah, every single system of record, right? And so, you know, we've seen folks, you know, procurement software. Like, you know, you're going to vendors and every year you're ordering a bunch of lab equipment. Instead of having this giant spreadsheet with like a thousand rows that then the ops team, it's like, you know, work-to-line companies, like normally what they have done is they have this giant spreadsheet that gets filled out by the teams themselves. Then an ops team takes that giant spreadsheet, puts it into an internal, you know, procurement software.

22:03Ari Weinstein: and then that gets like, you know, finalized as a bunch of lines and then sales team then goes, procurement team then takes that giant software output and then goes out and reaches out to vendors. Computers can automate all of that, right? So like procurement is a big one we see. Finance is another really interesting category we've seen. And so, you know, there's a lot of different ledgers within a company, right? And so you have like ledgers for expenses, ledgers for all these things. And so at the end of the month, there's a lot of reconciliation. And a lot of these financial software don't have APIs to be able to quickly pull things.

22:38Ari Weinstein: And so the finance team being able to have computer use, look through all these ledgers, reconcile, pull out anything that's like interesting to flag either to a person or then do some investigation. Also a very boring but useful workflow. For their sales team, a lot of sales teams uses a lot of different software for like transcripts, for getting data, all these kind of things. And so having computer use go and do prospecting and then going and putting that into a CRM or some kind of like, you know, outreach software and then all that. So all the system records is the boring part. And then like market research, another one that we see a lot.

23:18Ari Weinstein: And so, you know, in my free time, I've been building this like video annotation software for Badminton, a lot of Badminton. and like I like to film myself playing the game and then having the agent kind of go and look at stuff. And I was like, okay, how do I make this thing really useful? There's some software out there for like pickleball, for squash, for tennis. And so I just had computer use, go log in, create an account on these softwares, try it, take screenshots, give me feedback and then try to build it back into the app. And so these are all kind of like tedious workflows that could be done.

23:49Ari Weinstein: And then I think some of the more like interesting creative work of like enabling more ambitious work is like, you know, we talked about the model now being really good with like visual design. And so circuit design, you know, being able to like architecture work, hardware engineering, game design. These are also areas that I think we're really excited about where the Astro can take. Less, you know, less common for enterprise to think about those right now because they're thinking about the boring opportunities. But I think as folks understand the capabilities of the model more, that's where I think we'll kind of see more usage next.

24:23Ari Weinstein: But curious for any thoughts, Ari.

24:25Fabrizio Serafini: Yeah, I think kind of similarly on the consumer side, there's a lot of things where it's sort of automating drudgery or just doing things you don't want to do. It's filling out DMV forms for you. It's helping you unsubscribe from a bunch of emails in your inbox. And then I think there's also things that are more like, I'm hearing from a lot of people's stories of people doing things that they just wouldn't have time, they wouldn't have had time or wouldn't have thought to do otherwise. like I don't for me personally I'm um uh maintaining spreadsheets about my finance and spreadsheets about meal planning and uh I have a friend who's a personal trainer who's making customized things for his clients and um you know some of these things um would just be so so time prohibitive to do in your in your personal life um but this makes it so easy I use apple numbers which is like a obscure spreadsheet program but the model still knocks

25:14James Sun: it out I don't think I've ever heard someone use apple numbers it's a surprisingly great

25:18Fabrizio Serafini: spreadsheet software. And then I think there's a third category of thing, which I'm kind of the most excited about, which is just starting. And I think, you know, on the horizon, you know, which is really just having a different way to use your entire computer or interface with software. I think you saw in our Astra ad, really cool examples of people just talk to their computer with their voice. It's such a natural way to use your computer. And I think over time, we're going to see the form factor of software itself change due to this technology. And I'm really excited about that.

25:50James Sun: On that point. Okay. Form factor of software changing. Imagine if you're an enterprise user, you're used to using whatever clunky ERP from 30 years ago. There's a world in which you now unleash a set of agents that just operate that software and you don't really care. There's another world in which folks are a new like AI native player is using a computer using agent. And I'm curious if do you think the world is going to go more in the direction of like keep the legacy folks and just have this layer on top? Or do you think computer using agents can absolutely be part of new software players? And this is something I think a lot about.

26:36James Sun: So I'm curious to get your guys' thoughts.

26:38Fabrizio Serafini: It is such an interesting moment where simultaneously the cost and complexity of creating new software is so much lower. So you can't imagine software changing so much. But also there exists so much legacy software in the world that computer use is so well suited to interact with. So I expect both of those things to continue for a long time. You know, if you look at the software that's in production today, a lot of it was written 10, 20, 30, 40 years ago. And I don't think that that stuff's going to go away overnight. I do think that in some way the existence of computer use may put more pressure on companies to make software work better with agents because all the software is going to work with agents by default.

27:16Fabrizio Serafini: And I think WebMCP is a great example of this, and I think we'll see more stuff like that. So yeah, I sort of expect both to be super important, but I do think it'll like kind of shift over time. What do you think, James? Yeah, I agree with you.

27:30Ari Weinstein: I think it's like the way, there's like a lot of arts and crafts, and you understand this because Ari built a beautiful product with his team. The cursor was all in. And so I think there's like a lot of art and craft into building like great products that people can hold in their hand. And I don't know, like for me, if the internet becomes just this like faceless, you know, thing that is just agents talking to each other and people are not using anything anymore, I would be very sad. And I don't think it will be like that because like there is like something about craftsmanship and artisan software that's like just feels nice to use because the person who designed it really, really understands like the experience that the customer wants to have.

28:17Ari Weinstein: So like my wife loves, recently got into playing this like, you know, board game. I forgot the name of it. It's where you connect like five pebbles to like win. I think it's like OMOC is the name of it. And so anyway, so she's trying to get good at it because she's playing online with friends and is getting her ass kicked. And so I was like, all right, I'm going to build you like a tactical trainer for, you know, going through different gambits so you can like get better at it. Like that's, for example, something that like I, you know, created a first version. I was like, oh, I know like she likes to do it in a certain way.

28:50Ari Weinstein: She likes these type of ways to learn more than others. And so I was able to like, you know, craft the software to be something like feels really good for her. And like these kind of personalized software, I know it's like, I think maybe today still feels like only really tech people will do. But I think if you like, think about this a few years out, it's like, if anybody can build anything, and they can build these really craft software in a very short amount of time for like niche use cases that are like so well tailored for people, I think that's a beautiful world. and again, you know, at the end of her training the lessons and mistakes she made that could be an agent that summarizes it and gives it to her in a text format but when she's actually playing with the AI and trying to get better that's something that the person has to do so I do think it's I'm bullish that I don't think UI is going away and I think kind of a little tangent to the original question but I do think that the pressure of cheap software will make it so that software has to be better.

29:51Ari Weinstein: But I don't think it will just be like, yeah, everything will have an MCP and an API and, you know, computer use will never be useful.

29:56Fabrizio Serafini: I love this point so much. I think that like humans are so hardwired for direct manipulation. You really want to be able to, you know, move and do things with your own hands. And I think that it's really exciting to think about how software in the past was so one size fits all. It had to be, you know, often a lot of the software we interact with every day is designed for millions, tens of millions, hundreds of millions, billions of people to use. And I think we will enter an era where any piece of software could be designed for a hundred people or one person and just how much more personal and unique that kind of software will be.

30:33James Sun: Yeah. My mental model is I think there's probably two buckets. One is on the consumer side, you've got either something you just get, you get joy out of, you enjoy using, or you learn from, there is this, it's not the utility of it. It's the joy of interacting or learning or whatever it is, and you want to spend time. On the B2B side, you may want your agent to complete a bunch of the tedious work, but you also, there's this concept of like the audit log. And that could be personalized in how you want to interact, but the product really is me as the human user setting the goal and then being able to understand the logic, but not necessarily need to run through, like, the entire chain of command of how it happened.

31:16James Sun: And so, like, any B2B user still needs to figure out, okay, like, why did that payment get made and go back in and get a summary of what happened and be able to say, okay, pay this person at this time. Yeah, at 30 days, this much discount should be applied, et cetera. They want the agent to take care of, like, the execution of it, but they want to be able to at some level, like, some level of control still needs to be put in there. And I think that's like something that does require a UI of some sort. And it has to be, I think if we're moving more towards believing that AI will do more of this, that part has to be enjoyable to use too, right?

31:52James Sun: Why not? Like, why should we settle for shitty software?

31:56Fabrizio Serafini: I do think it'll be really interesting to see if we, and I am optimistic that we will see new kinds of software where it has traditional user interfaces, but is also using agents under the hood. and then maybe sometimes those agents are themselves using computer use to automate legacy software by giving you some much nicer, more purpose-built experience. I'm like really excited for that class of thing.

32:18Ari Weinstein: Yeah, it's kind of like the one where we like to use internets, like the shared intelligence layer, right? Like you have your agent using legacy software somewhere else and doing things at other places. You, like if you're using an application, the application wants to get access to that context so it's able to deliver a better experience. Like I think if we had, every website has its own agent that doesn't talk to your agent. It's a very fragmented world. And so figuring out like how that plays together where the agent can plug in and use your other software, I think it's going to be like a really interesting thing to play out for the next few months.

32:52Seema Amble: I love that you guys, yeah, talk about this because I think it's so underappreciated the importance of like human AI interaction, especially if you want like a product to be successful. So like being thoughtful of like, how can I kind of like, humans are used to collaborate. So you want any agent to be able to like kind of fit into the team. I'm just curious, like if you apply that to a new startup, let's say you're like, I don't know, starting a company tomorrow and you want to use computer use to like solve a workflow within the enterprise. How would you think about it? And would you be kind of what would be the things that jump to mind in terms of like, oh, I want to know when it fails.

33:28Seema Amble: Well, I want it to be able to communicate with people or I want it to be like, I don't know, a black box. Just like maybe if you had to talk to founders right now, what advice would you give them as they try to use your models and capabilities to just solve problems for enterprises?

33:44Fabrizio Serafini: In my mind, it depends on, it depends so much on the modality of, you know, are you trying to, are you a small team and you want sort of individual on the team to be able to use some piece of software that your team is using, but in a more efficient way? or are you maybe building something that itself wants to interact with existing software in which case maybe you have some you know machine in the cloud somewhere that's doing computer use in the background i think that will really affect the modality of how you want the human agent collaboration to be right um and um yeah you know in the former case i think a lot of our products are serving this use case um you know where you sit down at chat gbt and uh you know there's a lot of great, you know, collaboration you can have there.

34:36Fabrizio Serafini: And in the other case, you know, if you're building something with the API, you're doing your own computer use implementation, then, yeah, then it really is interesting to figure out when do you want the human to be in the loop? Is the agent telling you when something's wrong? Is it telling you when something went right? You know, is something summarizing that so that you don't get bombarded with notifications saying, hey, this happened, this happened, this happened, which actually you're going to ignore. I don't know. What do you think, James?

35:04Ari Weinstein: Yeah, I mean, you know, both Ari and I were startup founders before we joined OpenAI. And so, you know, I'm still very bullish on startups. You know, I think the question, right, kind of comes down to like, what, is there still a mo, right? Is there something that startups can still tackle while labs are doing things? I mean, in general, I just think there's like, like two things. Like one is like, I think there's so many things to build, right? And like, even though we're quite big now, thousands of people, we are still feeling very stretched thin internally. And like, you know, it's like the teams are not very big for all the problems.

35:33Ari Weinstein: And so if you have a problem that you really deeply understand, you have industry expertise, you understand how regulatory works, the go-to-market motion, how to service those customers. It is still, I think, a very green field out there. And the tools that you have through our APIs, like Ari mentioned with computer use, I think will still let you accelerate. And speed really, I think at the end of the day, speed and pain is kind of the mode that exists today. I think the other thing I think about a lot is like there are, in my opinion, still a lot of like vertical industries where the diffusion of AI is like much slower.

36:08Ari Weinstein: And so, you know, I was talking to a company the other day who uses computer use to help restaurant groups to, you know, take pictures of like their back storage, you know, and then using that, the vision looks at kind of like what is going to be missing based on a projection for tomorrow's traffic. And then it takes that to then use computer use to go to like a Shopee or some kind of like procurement software to then order your groceries for the next day. And then based on that, it will go into your ERP to adjust like the timetable for your like employees who are, who you might need more people to help.

36:45Ari Weinstein: Because in the last year, at this time of the day, there was a lot more traffic coming in from like holiday or something like that. And so like, you know, those restaurant groups are probably not like tech companies that are super, you know, familiar with all the tools that are available. And so I think like, you know, living in San Francisco, we see a lot of thinking, wow, like the labs can do everything, the codecs and like, you know, why doesn't it just install codecs and just have it do stuff? But like for the average American and the average small, medium business owner, the average enterprise, even understanding what the tools are capable is one.

37:19Ari Weinstein: Connecting tools in a way that makes sense is two. Deploying it is three. Servicing is four. And so there's still a lot, in my opinion, where if there are complex workflows that require a lot of disparate software, computer use is a great way to join them and complete the entire Intendor flow. And then building it in a format and UI that feels good to hold for those customers feels still like a really, really big opportunity that, you know, I think is still open.

37:44James Sun: Tell us about what's next. What are you excited about that's on the frontier that you can share around computer using agents?

37:52Fabrizio Serafini: Well, I've sort of gestured towards it already, but we're really excited about speed. We're really excited about making computer use superhuman. and we're really excited about things that blend together different kinds of interfaces that bring together sort of multimodal inputs and outputs so you can have more kind of fluid experiences with your computer. I think right now a lot of our computer use product is very background focused. It's very much like let me give the agent a task so it can do that while I do something else which is actually a tremendous strength, right, of the product but I think that in the future we'll also see more of this like real-time collaboration between you and the agent.

38:29Fabrizio Serafini: so I'm really excited about this area and otherwise just sort of better, cheaper, more available easier to use

38:40Ari Weinstein: yeah exactly, being able to actually access everything I think there's with Dots right, we're giving folks these like computer in the cloud also has a browser in the cloud honestly like for me one thing I'm excited about is just making the experience feel a little bit more coherent like you know today you can install the Chrome extension and that uses your like password manager and everything in Chrome. And then we have an in-app browser in the desktop app, which has a different set of cookies and different passwords. Your computer in the cloud also has different things. And for the average consumer, it's confusing.

39:13Ari Weinstein: It's like, why do I have these four different browsers that all don't talk to each other? If I open up a tab in one, does it like show up in the other? And so like, I think a lot of what I'm excited about is how do we make computer use go from this product that is used by like enterprises and like really hardcore tip of the spear kind of users to something that's actually understandable for people. It's kind of like an oxymoron because I think computer use is actually the most easy to understand AI product because it's an AI using a computer. But then because of the like complexities of like the different like states you have to manage and the different places you're using it, the implementation details sometimes seep through and then it becomes kind of hard to grok.

39:52Ari Weinstein: And so being able to kind of make that consumer grade and easier to understand and more coherent is something I'm really excited about for next quarter.

40:00James Sun: Yeah, I think it comes back to the point that there is still a human that's setting off the agent, right? And they have to, the better the UI and controls and the easier and more enjoyable it is to use, the more they're going to get out of it. And that's still principle.

40:15Fabrizio Serafini: On that point, one other thing I want to gesture towards is, I think another challenge of the product today is you have to put some thought into how you're going to use it and you have to remember to use it and you have to convince yourself to do things that are different from how you're used to doing them. And I do think dots, like James was talking about earlier, have this really powerful proactivity where they can sort of come to you and identify situations and help you solve problems. I think we'll also see a lot more proactivity. I'm really excited about a world where the agent can sort of adapt to you instead of you adapting to it.

40:55Fabrizio Serafini: And I think that we're going to see a lot of that. And I think it will make it a lot easier for people to actually like relate to and understand and incorporate this technology while also still being the one sort of setting and steering the goals, if that makes sense.

41:10James Sun: One of the things we haven't talked about yet is the momentum behind computer use and how much has changed. And Fabrizio and I chatted a little bit about this in our piece. Would love for you to just kind of talk about where it's, you know, how it's the trajectory so far and where things are at it.

41:24Fabrizio Serafini: Yeah. I mean, James and I have been working on computer use for some time now at OpenAI, and it's been an incredibly exciting past like six to nine months, right? Because we started working on these computer use products, and very few people were using them. The models, all of a sudden, it felt to me, became great at these kinds of tasks earlier this year because of amazing work the research team did. And then we've been working super hard with a bunch of teams internally on how to create actually good product experiences around this. And then just in the past couple of months, all of a sudden, you know, the product's growing like crazy and people are really seeing the value in it.

42:03Fabrizio Serafini: But I think it's still so early, you know. It's still the case that only some, you know, portion of our user base is using computer use. And I think we're going to see it grow quite a bit. Yeah, totally.

42:16Ari Weinstein: I mean, you know, I joined OpenAI in January and I joined working on the Atlas product. And I love Atlas, Rip Atlas. But I think, you know, one of the things that we realized was in the Atlas product, the agentic experience, I think for a few reasons, wasn't actually like a big selling point. Not many people used it, partly because I think most people treat it like a normal browser. and over kind of the last few months because the model capabilities have gotten better, we've now incorporated browser and browser use into kind of all of our existing products where there was already agentic usage.

42:49Ari Weinstein: We've just seen, you know, it's like YC level of growth week over week and then always surprised by how fast it can go. And I think this next like big inflection point is like cloud, right? Like how do we, like desktop is still very much a power user thing. I know we probably use our desktop all the time, But like most consumers are on their phones. And so how do you give every single Chachity user, how do you give everybody who's just using their phone for their day-to-day life, a powerful computer that they don't even need to understand how to use, that it just kind of makes things work for them.

43:23Ari Weinstein: And so I'm really excited about how that kind of brings it to, you know, building users.

43:27James Sun: It feels like computer using agents have had a ton of momentum behind them. And now it's the moment of like, okay, how do we get it into more people's hands? They trust it. They feel like they can really use it. And your point earlier about coherent experiences, like that's a real, I think as that gets a lot better, I think consumers will feel more comfortable using it and it'll feel more natural.

43:48Fabrizio Serafini: I think that's right. I think we're at the scaling moment of computer use, both for consumer and enterprise at the same time. And I'm really excited to see where it goes and what people build with it.

44:00James Sun: Nice. Us too. All right. Well, thank you so much for joining us. This was awesome. We love talking about this topic and it was really fun to hear it from you guys directly. Thank you.

44:09Seema Amble: You had a lot of fun as well. Yeah, thanks. Thank you. Thank you, guys. Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or a review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

44:54Seema Amble: Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures.

From the publisher

AI agents are approaching a point where they can use computers faster than expert humans, turning the user interface itself into a universal way for AI to interact with software. OpenAI’s Ari Weinstein and James Sun argue that computer use is now entering its scaling moment, as better models, faster execution, and stronger safety controls make agents practical for more consumer and enterprise work.

Ari and James join a16z’s Seema Amble and Fabrizio Serafini to explain how computer-using agents are already navigating legacy ERPs, reconciling financial data, handling procurement, and operating software without APIs or custom integrations. They also discuss why the harness around the model can matter as much as model intelligence, how enterprises can set deterministic guardrails around agent behavior, and why computer use could simultaneously extend the life of legacy software and accelerate the arrival of new AI-native applications. 

Looking ahead, they make the case for a more proactive and fluid relationship with computers, where agents work in the background, collaborate with users in real time, and increasingly adapt to us rather than requiring us to learn how to use them.


Resources:

Follow Seema Amble on X: https://x.com/seema_amble 

Follow Fabrizio Serafini on X: https://x.com/fabserafini 

Follow Ari Weinstein on X: https://x.com/AriX 

Follow James Sun on X: https://x.com/JamesZmSun 

Related a16z research from Seema Amble and Fabrizio Serafini: Can Agents Use a Computer Yet? We’ve Got the Data

Stay Updated:

Find a16z on YouTube: YouTube

Find a16z on X

Find a16z on LinkedIn

Listen to the a16z Show on Spotify

Listen to the a16z Show on Apple Podcasts

Follow our host: https://twitter.com/eriktorenberg

Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.


Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

More from The a16z Show

All episodes