Most Valuable Skill of 2026: Managing AI Agents

0:00I don't think it's crazy to say that the people who know how to run teams of AI agents are going to be able to outperform everyone else in this AI age we're in. So how do you actually run teams of AI agents? What does this mean? What's the step by step way to actually do this? And what are the tools you need to know, like Dev and AI to actually go and do this? Well, on today's episode, I brought on Ryan Carson, and he clearly explains all of this for you. He even gives you a tour of his insane setup for how he manages his AI agents. So by the end of this episode, you're going to understand what cloud agents are, how you can set up automations, how you can create a software factory that creates software 24-7.

0:42And you're going to be able to understand it just really clearly because he does such a good job at explaining all of this. So enjoy the episode. I'll see you at the end and have a creative day.

1:00Ryan Carson back on the pod. By the end of this episode, Ryan, what are people going to get out of this?

1:06Ryan Carson:So you're going to be able to do three things really well. You're going to be able to run cloud agents. You're going to be able to do automations and you're going to be able to ship faster. So whether you're a founder, a solopreneur, a stay at home parent, you're just going to become an agent pro. by the time we're done. Okay, because I think a lot of people listening to this might have played with agents, cloud agents, but you're going to help them just get the confidence basically to be world class. I think that's what I've been thinking a lot about. It's like, okay, there's a bunch of people who have played with agents, but how do we become world class agent operators?

1:50Ryan Carson:Essentially, you are a manager of agents now. So no matter what you used to do, whether it was a people manager or an IC, you are going to become a manager of agents now. And you need to be the best in the world. And it doesn't matter whether you're a VC, a founder, an IC, a mom, a dad, like a college student. You should manage agents and be the best at it. And we're going to show you some tips for that. Okay. So where do we start? Okay. So let me give some context, right? So people understand where is this coming from? So I've been a founder CEO for like 25 years. I'm working on my fourth company now.

2:29Ryan Carson:At my last company, Treehouse, which was acquired, we had, you know, I think up to like 110 full-time employees, taught a million people how to code. You know, so I've done the whole build organization thing and manage organization thing. And you learn a lot about managing people on that. And now this startup, which I'm running, I am one employee right now. So we've raised the seed round. We're scaling really fast. Our revenue probably is going to 4X this month. And I'm probably going to be hiring my first employees. We are essentially an AI divorce agent for divorce firms. So we do the dirty topic that no one seems to want to tackle, which is family law.

3:10Ryan Carson:So I've learned a lot about how to scale me. Right. And so I'm going to help folks understand how do you manage a lot of agents? Well, how do you check the work? How do you automate things? How do you work from your phone? You know, I do almost 50 percent, probably more of my work from my phone. and it's pretty wild. So that's where we're going to take you. And I will say as well, this is going to change every three to six months. So we're going to share a lot with you, get your hands dirty, try it. And the best way to learn is to do. Cool, yeah. I'm particularly interested in the phone stuff because I actually do 0 % of my stuff on the phone.

3:56What?

3:57Ryan Carson:Yeah. Okay, let's talk. So let me start by just showing you, well, when I'm at my desk, what is the world like? Right. So right now I'm in my home office. I work from Connecticut and I do all my desk work here. And I've got this big 52 inch Dell monitor. I used to have like a couple monitors and I was like, screw it. I just want a huge monitor. So let me share my screen. And all right. So what I do is I set up eight screens on my monitor. And I also enjoy food and snacks as I code. But the key is to have eight things up at once because I have to multitask a lot of agents at once. I also use a paper system to keep track of some of my to-dos, which is a little bit weird.

4:48Ryan Carson:I use the UG Monk system, which is kind of fun. And I've got a killer Razor mouse. I was getting a lot of wrist pain, actually. So I decided to go vertical mouse on that. I also have a button which actually is connected to Whisperflow. So I almost use Whisperflow for all of my stuff. So I've got Slack in the top left. That's where I get notifications and things about what my agents are doing or what is happening. That's the primary thing I use Slack for. And then what I do is I lay out various other screens. So I've got a 1Password always set up. It's very important to keep your keys safe, secure, and separated from the agent, right?

5:33Ryan Carson:So I've got all my prod write keys in 1Password. I do not give my agents prod write keys. So production writing is dangerous, right? And your agents will do something bad. And so you need to set up a system where they ask you for the key when they need to write to prod. And then you go into 1Password, you copy and paste it into the session, and you're both very clear about what you're doing. So that's one tip. Keep your keys secure. And then I use Devon. So Devon is created by Cognition. I think it's one of the best software factories in the industry. It's not cheap, but it's good. The things I'm going to talk about today are not Devon specific, but they do work very well in Devon.

6:19Ryan Carson:And one of the keys is cloud agents. Then I usually have two screens set up where I'm testing my app. It's just a lot of testing. And then, of course, I've got X in the top left and top right. And then on the right, I've got Codex, which we'll talk a little bit more how I do most of my work in cloud in Devon. And a lot of that work, like we talked about, 50 % is probably on my phone. And then whenever I do desktop or whenever I'm doing things locally, I'll do it in Codex. Usually that's in the bottom right. So that's kind of when I'm at my desk, my setup, and what I do when I'm here. But the truth is I do a lot of work from this, my iPhone.

7:04This is my beautiful wife.

7:07Ryan Carson:It's just I get a lot of stuff done. I'll be in the shower and I'm thinking, okay, I need to check on that PR. Or maybe I want to land that PR. Grab it, talk to Devin, which is just done in a browser on my mobile. And I get a lot of stuff done that way. So that's my setup. So I keep hearing this, that Devin is the most powerful coding agent. And for folks who haven't heard of Devin or who haven't played with it, why is that? What is going on? So about two years ago, maybe two and a half years ago, Devin came out. And they pitched it as this idea of, like, it's a software engineer, right? And honestly, it didn't work very well, right?

7:51Ryan Carson:The models were just not good enough. And no matter how good the harness was, the models just weren't there. And then fast forward to, you know, 2026, the models are definitely good enough. And when you have a good harness that is cloud-based, it's really good. So think of it as you've got your choices, right? So if you're using agents to build things, you're probably using Codex from OpenAI or using Cloud Code from Anthropic. Or you're using one of the Indie, I call them the Indie agent harnesses. You've got AMP. You've got Cursor, which is not independent anymore, but sort of, owned by Elon.

8:31Ryan Carson:And then you've got Cognition, which makes Devin. You've got Factory as another one as well. So you've got these choices, right? And so I thought, okay, I want to scale myself horizontally. Like I want to be able to run as many agents as I possibly can. And I don't want to be working on my laptop and trying to figure out which code is colliding. You know, which work tree am I on? A lot of technical stuff that really keeps you from shipping. And I know there's a lot of different folks that listen to this show. You might have solopreneurs who are newer to coding. They don't have computer science degrees.

9:14Ryan Carson:They're not super technical, but they're building things. Then you may have software engineers who are multiplying themselves 10 to 100x. They have deep understanding of software and architecture. And then you have founders who are coding but are abstracting themselves. And you have VCs who are coding, you know. But everybody is going to learn a couple skills. And what's interesting is I'll sort of zoom out for a second. I think we all thought that engineering and being technical was going away. But actually what's happening is the more you become a better agent manager, actually the more technical you become.

9:53And I think a good analogy is that you are basically an engineering manager, right?

10:01Ryan Carson:And so to be a good manager of many agents, you do need to become technical, right? There's this reality of you have to understand what Postgres is. You have to understand a production environment versus a dev environment. You have to understand migrations. But the truth is you'll learn all that by using agents, right? And so I am way more technical than I have ever been in my whole life. And so I think the idea is become more and more technical. Therefore, okay, so let's back up. Why Devin? What is it? It's a really good agent. They have their cloud environment really nailed down. So that's the first takeaway I want to talk about today is cloud.

10:39Ryan Carson:Okay. How can you run more agents, get more done? And the answer is you work in the cloud, not on your local machine. Now, for the folks that aren't super technical, sort of explain what does that mean? So when you're writing code or you're building things, you are editing code, right? That code has to be edited and then hosted somewhere, right? And usually what we used to do is we used to all work on our Macs. And that's called local development. And you have to set up your whole local development environment. You have to have your database. You have to have your authentication. you have to have everything local you have to have your dev server set up and then what happens if you want to do two things at once you have to either use get work trees which is pretty technical and hard to understand or you have to literally make another copy of your code in another directory and then so then to work on two things at once you have to have two copies of your code which is hard and then what about three or four what if you want to do five or ten things it just doesn't work So what Devin does, and I think a lot of good independent agent harnesses like Cursor is doing this now, I think Codex is starting to do this okay, is they create what's called a VM or a virtual machine in the cloud.

11:58Ryan Carson:And it's basically a little server, and it is your development environment. And you click a button, and it spins up, and all of a sudden, you can code in there. And the beautiful thing is you can create infinite numbers of these things. So I often have at least five cloud agents working at once, often 10. And there's no risk that the code is going to collide. So what it does is it takes away all of the mental overhead of trying to coordinate these things. Like I want to work in this part of the code, but what if it overlaps this part of the code? Or what if I want to do that? Have I synchronized and pulled the latest?

12:40Ryan Carson:Like, I don't know. And when I work in the cloud, I literally just open up my browser and I click new session and a new VM spins up. And I never, ever have to think about it. So I think if you are working locally, I honestly think you are a caveman. Like, I think you are holding yourself back and you are shipping 10x less than you could be. and it is not smart. No matter if you think the cool people work locally and all the smart ends, it's not true anymore. You work in the cloud. Yeah, I mean, there are a lot of people on X who are sort of like local maxis who look at people who spin up virtual machines and cloud agents as sort of amateurish.

13:30What do you say to people like that?

13:31Ryan Carson:they are not doing real work. Here's the deal, right? So as soon as we hit PMF with Untangle and we started acquiring real customers and having to ship real fixes, real features, like I had to probably 50X my output, right? And you just can't do it if you're working locally. So I think it's nice to work locally. And sometimes you have to if you're doing heavy front-end stuff. So say you're shipping a brand new feature with a lot of UI, like a lot of new UI, right? Then, yes, of course, you're probably going to spin up a local agent and you're going to do some light front end, you know, probably wireframing.

14:17Ryan Carson:But as quickly as you can, you want to move that into the cloud and then let a cloud agent take over. So I think this is happening to a lot of people. I think a lot of the traditional knowledge of how to work is out of date. That's where we're at. Okay, so if you do start using cloud agents and you do 50x output, in some ways that is a little stressful because you have so much stuff going on, right? And now all of a sudden you have to manage more. So how do you structure a system so you don't go absolutely crazy? So it's interesting. I think, first of all, you have to shift your expectation of what your work is.

15:05Ryan Carson:And what your work is now is making high stakes decisions almost all day. So in the past, we all had the luxury of making probably, at most, maybe two to three high stakes decisions per day. Right? You'd have an important meeting with your key team or with a customer and you'd make a couple important decisions and you would communicate those decisions and then they would get done over two weeks. I think you have to sort of mentally pace yourself to say probably by lunch you're going to have made 10 to 20 high stakes decisions. And so you have to learn how to do that. So which is your question?

15:48Ryan Carson:How do you do that? And I think there's a couple simple things, right? So when I'm working with my agents, I actually have to organize my threads by, and I pin the ones that are the most important work we're getting done today, right? Because there's a lot of quick bug fixes or, you know, small things you're shipping. And you kind of learn to separate into two buckets. What are the big, big important things I'm getting done today? Pin those. and and then the other things just let them rip and get back to them when you can and then set yourself this kind of almost timer right where you're like okay i'm gonna go crazy if i just click through my threads like this all day you just get completely wiped out right and so it's almost this discipline of saying i'm actually you know that the phrasing smooth is slow is smooth and smooth is fast.

16:46Yeah.

16:47Ryan Carson:I think there's a little bit of that where you realize I can only check on my agents and make high stake decisions probably every 25 minutes. And so what you might do is settle into this cadence where you check on your high stakes threads like every 25 minutes, just so you can like literally mentally rest. and then the other side of that is just understand that this is the world we live in and it is kind of exhausting um and and that is kind of where we're at and in order to survive in this new world you're going to work a lot more not a lot less and on that note i'll share my screen because there is essentially uh a world we live in now where you're going to ship probably you know so this is my prs right and we're looking these are days right and you can see what you know the average here is is sort of this 22 to 25 prs a day right and and sometimes you know 40 a day um most of these are being you know merged right a couple are being closed and what's funny is what happened this day um i went uh hiking with my son which day is that kensi or cursor oh it's uh the was the 18th um and so we went to climb mount washington and i literally couldn't access my my phone um but notice immediately the morning of right you know while he was sleeping i i shipped what is that you know eight prs um so this is kind of you know the world we live in now um And we could talk about that.

18:30Ryan Carson:But let's zoom out to the lessons here. You've got to go cloud, right? You have to understand in order to compete in this new world, you're going to be working half the time from your phone. And it is tiring, but it's like going to the gym. Okay, I have this new muscle like I never had to exercise before. And it turns out this weird back muscle, it turns out like that's the most important muscle in the world. And if you don't do it, you're not going to win. So the way to do that is in set some sort of cadence for yourself, pin your most important threads, write it down. So this, you know, what I'm going to sort of flash up on the screen here is my written to do list.

19:10Ryan Carson:Every day, you know, I use this hilarious analog system to like to remind myself what are the most important things we're going to ship today? because there's going to be 10 probably prs that we ship that are fire you know putting out fires um and and you can get distracted and exhausted so so cloud and then exercise this new weird back muscle which is making high stakes decisions all day um including on your phone right what you're basically saying is like you're gonna because of the velocity of uh of you know knowledge work and you're you just you'll have stuff going on and you will need to be there to say yes no actually this you know i'm giving feedback in real time and you know why your point is why wait until you're in front of your mac to give feedback um it makes no sense if you're trying to hit product market fit and if you're trying to forex revenue in you know in a month or whatever it is um you know why to be available gotta be available and you know i think it just it's easy to think of these things as humans right like you have a team now that is infinitely scalable they're going to do work very fast and they're going to be blocked by you making important decisions right um and you can't delegate the decisions yet i think we're still very much even if you use fable Like this is a trick is use a Fable thread as kind of the manager and then ask it to spin up children.

20:47Ryan Carson:And I do this a lot. So, you know, I'll spin up a Fable thread and I'll say, OK, you know, we want to accomplish these five things. You know, spin up five, you know, child devin sessions. Don't use Fable. I use Fusion for that, which is a cheaper model. It's cool. It's like their new model where they've got Fable as kind of the parent and they've got a sidekick. The point is, it's not viable to run Fable or any super premium model all the time. But have a sort of a parent, very intelligent model run children. You can do these things. But in the end, you're going to have to pay attention. And it's just like if you ran a big team and you were out to lunch on your boat all the time.

21:29Ryan Carson:You got to get in the office. And the office is your phone. And so get it done. So Greg, this is my homework for you. Next time we talk, I want you to be doing more than 50 % of your work on your phone. Yeah, I mean, that's why you're here, right? A lot of people ask me, how do you choose guests and why do you explore certain topics over other topics? And it's for everyone. I love arming people with the information, the tactics, the systems. But I'm also everyone. I'm a part of that, right? A part of that is I know that you're doing this and I want to understand why you're doing it. I also want to understand one of the takeaways I have is your system for dealing with tasks is not too dissimilar to a very organized person pre-AI.

22:21The idea of pinning your most important things, on one hand it's pretty obvious, but on the other hand if no one tells you Like, hey, you know, the way to do this so you stay sane is via that system. It's overwhelming.

22:39Ryan Carson:Yeah. It is funny. It's like back to the basics, right? Yeah. It's just in an agent age. And it turns out like being a very, very good engineering manager is probably one of the most valuable skills you can teach yourself. And, you know, I have the honor of sort of mentoring my nephew right now. and he's coming to me and asking me, how do I win? You know, he's just come out of college soon. How do I win in this new world, right? And, you know, I'm basically teaching him what we're talking about on the show. It's like, okay, your job is going to be managing as many agents as you possibly can. Get good at that, right?

23:16Ryan Carson:Which means, you know, you need to start to adopt these behaviors, right? Work in the cloud, prioritize what you're doing, learn how to make high stakes decisions quickly, get technical um you know it is a farce that engineering is going away like if anybody tells you engineers are going away or becoming less technical they're just not doing the work right um and it's like being a carpenter like say you had robot carpenters around if you were a real carpenter you'd be a much better manager of those robot carpenters because you understand how to make the cut, you know, where to put the nail. And so it's important.

23:56Ryan Carson:So to back up, and if you're listening to the show, like the thing I want you to walk away is cloud agents. Get your reps in, figure out how that works. Try it a couple of times, do it from your phone and get comfortable, you know, looking at your small phone interface and learning how to work that way. So I think that's thing one. It's very important. The second thing I want to dive into is automations. Okay. So say you're working in a cloud. Well done. You know, you're starting to separate yourself from the crowd. The second thing is automations. You need to sort of think about things that you would have normally had a meeting with, you know, once a week with your team, where you check on something or daily you check on something and build an automation.

24:46Ryan Carson:What does that mean? So the first thing to do is go back to your agent, start a session and say, okay, I want you to automate this task every X amount of days. And so one of them that I do, it's really simple is that I want to automate the test of signing up for Untangle, creating a case, onboarding a client, having that client go through discovery, you know, basically our user experience. but I want that to be automated in a browser. Like this is not rocket science. This is basic user testing, right? But it turns out that good agents can do this. And so the big unlock is set up your agents so that they can make themselves better instead of you trying to build these systems.

25:38Ryan Carson:And so I set that up and then in Devon, it's really cool. It's called a playbook, but the idea is it's, It's sort of like, here's how to do this thing. It's different than a skill. It's more of a list of things to do, how to do them correctly. And I have a, it's called end-to-end sign-up test. And that runs three times a week. Because it is expensive. It's probably$60 in tokens. Because it's doing a lot of browser testing. and I think that's sort of a side point I want to explain here is it's now vital that your agent can properly browser test and it really should work out of the box right this is part of the reason I use Devon is because they've been using cloud agents that can do browser testing for two years now, it's so good.

26:36Ryan Carson:It records a video, it annotates the video, it then looks at its own video and it fixes bugs that it sees, right? So you have this agentic loop that is specifically around browser testing that then is automated three times a week, right? And then what do you do with that? And then you need to trigger a triage based off of that. So say that it's like, oh, wow, the third test failed in this automation. I'm going to spin up a devon session or an agent session to then fix it. And then you have to figure out where do you know that that failed? How do you know? Did the agent just kind of silently fail and then fix itself?