#313 Jonathan Wall: AI Agents Are Reshaping the Future of Compute Infrastructure

0:00I thought there was, it would be interesting to apply the lens of infrastructure to kind of the coming agentic revolution. Got together with some former teammates from my previous startup and hired some new folks to start Runloop. Right. And so the agent operates in its own virtual machine, not necessarily a web page, not necessarily a browser. Or is it a browser? Agents work differently enough and have different enough compute patterns that they need their own new compute primitive. And that is core to what we're building here at Runloop. We call that our dead box. So there are some folks that make agent builders and that is super useful and super necessary.

0:42Some of them are paired to different frameworks. Like the folks at Langchain have a cool agent builder that is tightly coupled to Langchain, for example. But where we're playing is a little lower. It's a little closer to the infrastructure layer. And we ultimately want to be the runtime where the agent is deployed and where it executes. What I generally do is have you introduce yourself, give your background as far as it's relevant, and how you got to Runloop, and then talk about what Runloop is. And then we'll talk. There's some of the stuff in the materials that was sent over that was interesting to me.

1:27you talk about agents changing the shape of compute. Yeah. And I'd like to hear all of that, but let's start with the introduction. Okay. Yeah. Hello. It is. It's great to meet you, Craig. My name is Jonathan Wall. I am the founder of Runloop AI. I suppose the CEO as well. I cut my teeth early in my career at Google. I joined Google not long after it went public, where I was the tech lead of the Google file system. So that was the seminal early days of huge scalable distributed systems. So I had a front row seat and seeing like kind of an explosion of the internet being realized and new technology being built for it.

2:10Around 2009, another engineer and I went out and founded the Google Wallet. So that was the first NFC solution for tap and pay. When you see people tapping phones to pay, that was the technology we created in 2009. And at the time, I didn't think about it in as much detail, but it was picking up on the next big platform revolution in technology at the time, which was the mobile phone. And the kind of the new capabilities that form factor enabled. Went from the build out of the internet and at scale infrastructure to working on payments and mobile. From there, I left and started a company with another gentleman from the Google Wallet effort called Index.

2:49And we did secure payments infrastructure for tier one retailers. And that company was ultimately acquired. It was named Index. It was ultimately acquired by Stripe and became the Stripe Terminal Products, the Stripe's card present product. So I spent a little bit of time at Stripe. And after leaving Stripe, found the next big platform change or the next big technology revolution of AI and thought about how I wanted to participate in this particular revolution. Given a lot of my background as an infrastructure, I thought there was, it would be interesting to apply the lens of infrastructure to kind of the coming agentic revolution.

3:28Got together with some former teammates from my previous startup and hired some new folks and started Runloop. And we're really trying to build a platform as a service to be the place where you run these agents. And much like the infrastructure build out of GFS back in the day was necessary for the internet, we think that agents work differently enough and have different enough compute patterns that they need their own new compute primitive. And that is core to what we're building here at RunLuke. We call that our dead box. Yeah. And there has been this kind of explosion of meta platforms that sit under agents on top of foundation models to orchestrate and have agents talk to each other or talk to models and all of that.

4:20I'm familiar with Boomi. I don't know if you know them. I don't think I know that one. Yeah. And there's another company. I just had a conversation with them yesterday called DevRev. And they have a product called Computer. And a lot of these, they have a platform where you can build agents and then manage agents and then have agents work with each other. How does your company fit into that emerging ecosystem? Yeah, so you're pointing out that there's a few different kind of layers to this stack. So there are some folks that make agent builders, and that is super useful and super necessary. Some of them are paired to different frameworks, like the folks at Langchain have a cool agent builder that is tightly coupled to Langchain, for example.

5:15But where we're playing is a little lower. It's a little closer to the infrastructure layer. And we ultimately want to be the runtime where the agent is deployed and where it executes. I think if you contrast the needs of an agent to that of a traditional server, traditional servers, they get a request. It's usually schema, right? People use REST or maybe gRPC and protobuffers and REST and JSON, right? It executes some deterministic code. Usually it'll store some state in a database that also is strictly schemaed. And the way agents work is a lot different. Agents, they need access to a lot of tools.

5:50They need access to context to perform their task. And they have unpredictable behaviors compared to traditional deterministic code. They might use a lot of resources. They might write code for themselves and need to execute it. And what you really need for these agents is a runtime, a place for the agents to execute where that stuff is safe. And you also are able to empower the agents with all of the tools they need. And we think that the dev box, which is effectively an execution sandbox, it's more or less giving each agent its own computer is the right pattern for this. So we can set up the sandbox with security boundaries and isolation so that if the agent happens to do something unexpected or potentially dangerous, it's isolated on its own sandbox environment.

6:34On one side of the coin, you're saying, hey, I want to prevent the agent from doing something accidentally bad. The flip side of the coin is you can say agents perform better when you give them tools. If I give this agent its own computer, that is effectively the most powerful tool. And by having its own sandbox, it can go to the internet, it can download documents, It can learn things it needs. It can parse documents. It can keep track of its own process. Not to anthropomorphize it, but like when you're working, you probably have a notepad next to your laptop. You have your own computer. You can go find information.

7:07You can keep notes about what it is you're trying to accomplish, maybe a to-do list and check it off. And we think in this manner, agents work a lot more like people and they really need their own computer to succeed. And that's the compute primitive we're really trying to bring to market. And so the agent operates in its own virtual machine, not necessarily a web page, not necessarily a browser, or is it a browser? No, you're correct. It gets its own virtual machine. So we let people define container images. Containers popularized by Docker are effectively a de facto way to define a compute environment these days.

7:48So we launch a container, but we put it inside of a micro virtual machine so that you can tightly control what it can and can't access and make sure to keep it isolated. But you're correct to observe, it's not a webpage, it's not a browser. It has full access to a computer of its own and it lets it do really powerful things. I don't know how much you've played with some of these agents like Cloud Code or Codex, Langchain Deep agents, but you'll see they routinely use the shell, the terminal environment of their computer. And that's really taking advantage of 50 years of Unix programming that is at their fingertips to just dynamically get things done how they see fit.

8:27We think this is pretty important. And let me ask, so that's for an individual agent, and I apologize for bringing these other companies into it, but it's a way of me triangulating where we are with Runloop. Manos, for example, was the first, the Chinese agent platform was the first that I saw that's using a virtual machine, but it's a browser. It's using a browser. Is this a trend of having rather than the agent using resources on the user's computer to have the agent operating in the cloud somewhere on a separate compute infrastructure? Yeah, for sure. I think there are many reasons for that.

9:23And just to clarify, I think with Manus, I believe they actually run the agent inside its own virtual machine. And then they have a web page that is just like the product surface area for how you do it. I see, yeah. But I think each of those agents probably, I believe, does get its own virtual machine. And yeah, I think that this is the kind of pattern that people are going to move towards. You have a nice isolated unit of kind of execution, meaning, hey, here's your container image. You run in a virtual machine. I'm not worried about you getting out and causing trouble. But also I can put anything into the container image you need.

9:59I can make sure you have context and I can make sure you have all the right tools to get your job done. And I think by and large, what we'll start to see or really what we already are starting to see is people have some nice web-based UI that then routes to these agents running inside their own virtual machine. And I do think it is rather useful for you to bring up these other companies. as examples because you made a point that you're seeing a lot of companies popping up at different layers in the stack and i think you could even there there are people above us in the stack like agent builders and different agent frameworks there are also people below us in the stack like we predominantly run on aws and gcp but you can look at people like oracle and core we've saying hey there's enough of a shift in computing here happening here that bundling gpus is now really important too.

10:50And they're trying to effectively go compete and build new cloud offerings that go head to head with AWS and GCP. So across the board, the status quo has been maybe thrown up in the air. And at all these different levels of the stack, you have different people trying to provide new primitives to build towards this new economy. Yeah. This layer that you guys are building, it's like taking the Manus concept, not that you took Manus's concept, but for my mind, and creating an agnostic infrastructure layer that you can, then all these agent builders can operate on top of. Is that right? Yeah. That's a great way to put it.

11:37We believe that in the future, which is happening now, you'll see more and more people building agents and saying, hey, I will have a product front end. The way for me to deploy my agent is to spin each of them up with their own computer. And that's why our platform is designed to satisfy that use case and to make it as easy as possible. We make it really easy to specify what you want in their container image to then launch these things and connect to them. So we can launch, we have some of our customers launch like 6 ,000 of these things at a time. And then in terms of connecting to the agent so that your product front end can actually pipe all the way in and interact with the agent, We support, we have cool connectivity capabilities.

12:17We have a product feature called Tunnels where you can connect to these things via WebSockets or HTTP. You can tell us to open Tunnels. By default, we don't open a tunnel unless you ask us to. But yeah, we think that this is an important new primitive and yeah, excited to be a part of it. Yeah. And so you're not building agents yourselves. Is that right? No, it's really up to our users to build their own agent. And maybe let's walk through the life cycle of how someone goes about doing this. Yeah, that'd be great. Yeah. There's another trend that maybe is worth mentioning that's a little orthogonal to us, but is, I think, somewhat important.

12:53And that is one of people now starting to move towards agent harnesses. So you have the Cloud Code folks have produced the Cloud Agent SDK. OpenAI and Codex has produced the Codex SDK. And then great folks at Langchain, we've been fortunate to partner with on a few things, have produced a harness called Deep Agents. And these harnesses are very capable kind of agent building blocks that you can build on top of, that you can customize for your particular use case. So I think a very natural way for people to build agents going forward is going to be to choose one of those harnesses to start coding and to include that thing as a dependency in their agent, to code it locally and say, okay, great.

13:36We're like, this agent's good to go. You then need a way to deploy it. You say, okay, I need a cloud deployment. I'll use Runloop hopefully. You use our APIs to deploy your agent. Maybe you build a front end on maybe a company like Vercel or a company like Railway that makes it really easy to build web front ends. And then whenever someone comes to your site and says, hey, I want to talk to an agent, it makes an API call over. It starts a dev box on our system for that one agent and you connect the two to the front end and off you go. And that's, I believe that's effectively how Manus works. I obviously haven't seen their source code.

14:10I don't think it's open source, but I think that is the product pattern people are going to start building against. Yeah. You mentioned someone spinning up 6 ,000 agents, and I'll get to my more run loop focus questions in a minute. But I've always wanted, not always, obviously, but in the last couple of months, wanted to go in and sit with somebody managing 1 ,000 agents or 6 ,000 agents. What does that look like and what are those agents doing? Because again, just from talking to people, I'm getting the impression on the enterprise side that there isn't yet the level of trust or reliability among the agents to let them to get 6 ,000 running at once unless it's 6 ,000.

15:09employees monitoring their email boxes or something very basic. So in that 6 ,000 example, regardless of whether that's a hard number, but how effective are these agents that you see people building and how varied are the activities that they're involved in? Yeah. So these are like, I think very important questions. In terms of the varied activities, when we started our company, we really focused a lot on the coding vertical. So we have a lot of the companies that are built on top of us, people building coding agents. So there might be coding agents that attempt to discover security vulnerabilities or attempt to reduce technical debt in your code base by writing unit tests and upgrading dependencies and doing things like that.

16:04So there's a wide array of kind of companies of that sort. Increasingly, we're starting to see people building like financial technology agents as well, where they're trying to solve finance type problems. So given some set of documents from an S1 filing or something, can I extract useful information and stuff like that? So it is pretty open-ended. It is pretty broad. So those are the types of problems with some folks in crypto, some people in health tech. So it is widening. There's a dispersion of use cases on our platform. But you asked another really important point that I think is something that the industry is still getting its hands wrapped around, which is one of accuracy and of being confident that your agent is producing useful output.

16:48At the core of our platform is this notion of a dev box, which is effectively an isolated execution environment for an agent to do stuff. We actually built a product on top of that called Benchmarks. And the purpose of the benchmarks is to help people with the accuracy problem. And what the benchmarks are is let's say that one of our customers is building a security agent for the sake of argument. You create a benchmark based on a dead box in a known state with a known desired outcome. Meaning given a dead box that started in this state with a code base, I know ahead of time there's two or three vulnerabilities.

17:29Let me run my agent against this thing again and again, and then let me verify that it always finds those vulnerabilities. So the benchmarking solution that we have lets our customers build domain-specific tests to verify their agents are routinely succeeding at the types of things they're trying to get them to do in the wild, in production, in live production traffic. And I think it's a pretty important part of the development lifecycle because Because, hey, Anthropic just dropped Cloud Opus 4.5. Presumably it's better. All the benchmarks they released seem to indicate it's better. But how do you know if I switch my model or my agent that's talking to that model, will it perform better?

18:12If you've been running these benchmarks as part of your production lifecycle, you can change which model you're using and see if your scores get better. Or if, who knows, they might get worse. And so we think the benchmarking is actually a very important thing just for normal development as well. Even if you were to do something as simple as say, I'm going to change the prompts for my agent to try to make it friendlier. Maybe someone said, the agent's a little abrupt. Let's make him a little friendlier or her, a little friendlier. You would think that encouraging the agent to be friendly wouldn't change its underlying behaviors.

18:51But you don't know, right? What you're doing is taking your agent's entry point into a large stochastic model and perturbing it. You're shifting it. You're moving around how it enters that large stochastic model. And you don't really know what the side effects are going to be. And so we do think that the benchmarks are very important. And I would say that some of our more advanced practitioners on the platform use the things like the benchmarks, but then they also have application layer of things that they do to try to improve accuracy. So they'll have an agent that tries to solve a task and says when it thinks it's done, and then they'll have an agent that runs after it, that tries to verify it actually did its job, and then tells it to start over if it thinks it was wrong.

19:37So there are a number of ways people can push towards accuracy, but we think having something like benchmarks and then just constantly measuring your accuracy is a key point. Yeah, that makes a lot of sense. And again, going back to this society of agents, as somebody I spoke to called it, where you have a thousand agents in an enterprise or 6 ,000 agents doing stuff. Is your sense that most of those agents are on call for employees to use like a coding agent? Like it's not running on its own and performing tasks in the enterprise. Or are there agents that do run continually on their own, maybe watching for expense reports to come in and then handling them?

20:35Yeah, it's a great question. I think out there you see basically a mix of everything, right? I think the human in the loop, human driven, hey, we're collaborating thing is a very popular pattern. Like you see that with Cloud Code and with Codex and stuff like that and Gemini CLI, right? Where you probably have a human that's really driving the like work input, evaluating the work result. But the agent is working at the S of a person. But I think similarly, as you were suggesting, there are all kinds of systems out there that have events they throw off. So like a ticketing system can fire an event.

21:12Most ticketing systems, like Zendesk or something like that, when someone files a ticket, it'll email the support team or something like that. That can easily be a trigger that invokes an agent and gets the agent to try to resolve an event as well. We see this. Some of our customers have agents that fire. every time you push a change to GitHub, it will fire a request, start an agent, and have the agent review your code too and see if there are any obvious mistakes. We also have customers that hang their agents off of Slack. So you can say, at agent name, hey, can you help me with some problem?

21:48Then that's yet another source for a trigger. I guess that's a little more human-driven, but I think you're really seeing a blend of everything, and I think it'll probably continue that way. Yeah. And your benchmarking layer is really important to make sure that the agents are performing accurately, but presuming that accuracy improves, do you see a day when agents are really the underlying fabric of the enterprise where you have them handling all the back office of stuff, all the quality assurance, whatever, depending on your domain. Yeah, I think it'll be yes, but I think it'll be like a per workflow sort of a thing.

22:35And my guess would be that probably what will happen first is you'll have agents do 70, 80 % of specific types of workflows, and you'll probably still have a human being that audits or approves a final result. An interesting observation is that in the coding space, so if we set aside agents and AI for the moment, when I go to write some code, I have in my mind, or maybe someone filed a ticket and assigned it to me, I have in my mind some sort of a change I want to make. I'm going to go make the change. I probably make sure it builds and it passes all our tests, but then I assign it to one of my peers to review.

23:13So someone else on my team says, okay, John thinks he solved this problem. Let me look at his code. And then ultimately that other person approves it. And then the code gets landed and merged. I think that you might see those types of patterns, but where instead of me writing the code or me performing some action is maybe an agent that does most of it. And then there's a reviewer that's okay. I trust this agent mostly, but I'm going to look at the conclusions they made. I'm going to look at how they did this work. Maybe I'll look at the input requirements. And then I'll approve it. And I think this is an interesting component that it's going to need to be added to a lot of enterprise workflows, which is this kind of acceptance review and accept stage, right?

23:58It's very natural in the kind of coding tooling ecosystem, but I think it's going to need to be added to a lot of the broader enterprise ecosystems where maybe a Zendesk ticket comes in, maybe an agent reads the ticket, looks at some knowledge base, does a bunch of work, proposes updating the customer's record in Salesforce. but before it gets to do that, a human being is yes, no change. So I do think this is coming, but I think it'll be selective workflows first and probably human audited a lot of the time. And then eventually there'll be the workflows where you have such high confidence that instead of the human being requiring every reviewing everything, they only review 10 % of things or spot check, something like that.

24:40Yeah. And most enterprise that exists today are trying to adopt agents simply because of the competitive pressures. But there are other startups in industries where there are legacy players that are dominant. Do you see agent native companies coming up and is it possible to challenge a legacy player with a predominantly agentic workforce, do you think? I think so. And I think it'll just become increasingly easier. Yeah, that's amazing. So in the way that the internet forced a reshaping of the economic landscape knows that it adopted it early, figured out how to use it. And then some startups coming in and challenging legacy players, it really, some big players faded away and new players arrived.

25:48Do you think we'll see that kind of churn in the economy? I would guess so, yeah. And I think an interesting place to look at this is maybe the series C and D companies that were born before AI, but you can look at how rapidly they're adopting it. So you can look at a company like Notion or a company like Figma, right? So both these companies predate and AI agents being really a thing. And they both were challenging larger incumbents, but both of those companies being still rather young and nimble have supplemented their existing product with AI. And Google is doing this all across the G Suite stuff.

26:30Microsoft is trying to do this with co-pilots everywhere. So there's a little bit of a foot race here for the incumbents to say, okay, how can I like supplement and improve my product with agents to fend off that challenge? versus there will be new startups that are just more purely agent first. They can grow rapidly and maybe rock the cart a little. Cursor would be an example of this, right? Where they were born of the AI era and they have a really cool IDE that is agentic first. And they're making some waves and growing quite quickly. Yeah. Can you walk us through a concrete, you did a little bit before, but a concrete example of an agent job on run loop from spin up to running tools to shut down so people can picture what the platform actually does under the hood?

27:24Yeah, sure. So maybe I'll give an example of maybe someone who runs, I don't know, has like a code review tool. I think the important piece of this is to, is it's important to call out that there's still a product layer for these companies that exist outside of front loop, right? This is really where the agents are running. So let's say that you are starting tomorrow and you're going to build Craig Review, right? And this is your code review agent from Craig, right? So you might use the Cloud Agent SDK or Langchain Deep Agent or the Codex SDK, or you might just hand roll your own agent. Who knows?

28:00I don't know how ambitious you're feeling. Once you've built that agent, you're going to get it ready to run on run loop. So we have an agent API that lets you upload your agent and store it on our platform. You can also bake it into a container image. So it's just part of a container image. So now that you have built your agent, you figured out how to deploy it to the run loop runtime. Anytime you want, you can spin up a dev box and point it at a code repository and a review you want it to go annotate. This is good code. This is bad code. you should do this other thing, right? But now you need to build the product front end for your product.

28:36Maybe let's say that you're also a fan of, for the sake of argument, Vercel and Next.js. So you might build your web front end where people go sign up and put in their credit card or whatever. And then they give you authorization to receive web hooks from GitHub. And then every time someone opens a pull request on a GitHub repository, you get a web hook at the product layer. You're going to receive that webhook and be like, oh, great. RepoFoo pull request 22. I'm going to stand up a dev box on run loop and tell them to code mount this repository at that review. And I'm going to invoke my agent to say, hey, is this good or bad?