I spent $20,000 on Devin in a month. Here’s what I learned | Ryan Carson (solo founder)

0:00Ryan Carson:If you're out there listening and you are doing engineering, you're doing your work locally, you really need to open your eyes. I think the future is pretty much 100 % cloud agents. I went all in. So much so I spent$5 ,000 on Devin and then I went up to$20 ,000 one month. You know, I think one of my big messages for today is that all of us have to up-level our ability to manage agents. Like that is our job. How do you effectively manage 10, 20, 100, 500 employees? This is the whole reason that organizations were built as pyramids. Like humans can't scale and have a thousand directs. What is in your stack right now?

0:38And how have you structured stuff to make you successful and build things effectively?

0:43Ryan Carson:So what I'm doing is figuring out how to manage, you know, throngs of agents. And in Devon, what you're seeing is folders. And you'll notice P0 is the big stuff. These are the main things I need to move forward as the business. New features, P1, P2. So I have priority zero. I've got bugs. Let them work. You usually give them a goal, and it's just such a crazy different world we live in now.

1:07Welcome back to How I AI. I have a super special guest, Ryan Carson, one of our earliest How I AI episodes, who is here to show us what has changed in the last 12 months in the age of coding agents. We're going to wax poetic about our favorite little AI intern, Devin, talk about all the ways you're under utilizing your coding agents, and give maybe the most anti-AI episode of How I AI, where we talk about where AI can't solve your problems, how you need to get out in front and actually talk to people, and how pen and paper still have a place in his stack. Let's get to it. This episode is brought to you by WorkOS.

1:49AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch?

2:27It's a massive lift. That's where WorkOS comes in. WorkOS gives you drop-in APIs for enterprise features, so your app can become enterprise-ready and scale up market faster. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are already using WorkOS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders at WorkOS.com. Start building today. Ryan Carson, number one episode of How I AI from our early, early days is back again. I am so happy to have you here, my friend.

3:09Ryan Carson:Oh, this is, it's so fun to reconnect. And the reason why the show is good is because of you. And just everybody listening. I'm not just kissing ass here. Everybody that I meet, I say, there's one show, if you're stressed about keeping up on AI, just listen to Claire's show. Oh, thank you. Well, I'm stress keeping up with AI. So I will take all that stress on behalf of the audience and pass on what I know. What I also am so excited about this conversation is, you know, we did we did our first episode a year ago or even a little more than a year ago. And like, it's so different now. Oh, my God.

3:44We were such babies.

3:45Ryan Carson:I know. It's it's it's kind of funny, like what we chat about now. And it'll be funny what we chat about now in a year. Right. Yes, totally. And for those who have not gone in the archives, we did a really, really popular episode about how to manage your agents with like, it was like pre-skills even. It was like three-step mark, like you invented skills from first principles. It was like three-step markdown file. We were in an IDE, which I haven't opened an IDE in months. we were in an IDE and we were really you were very prescriptive and directive of your agents very handholdy very handholdy and now you know I don't know about you I'll speak for myself what the agents are doing is none of my business no I let them let them work you know usually give them a goal and yeah it's just such a crazy different world we live in now it's so exciting Well, catch us up because there's also been a lot of changes on your side in the last year.

4:47So what are you up to and how has it impacted how you think about and work with AI these days?

4:53Ryan Carson:Yeah. So I basically have this idea to make divorce better. So I have three amazing sisters. We're like best friends. I have a sibling tattoo. It's hard to show. It's right there. You know, we talk every week. And then two of them started to go through a pretty gnarly divorce. And my parents got divorced, too. So, you know, I was like, divorce? hmm, like no one seems to be talking about divorce and AI. Why? And so I just started working on that and was, of course, empowered by Thor's hammer, which is large language models, and I started to build it. And this is when we chatted last. I was in Cursor, like you said, really hand-holdy on the agent.

5:33Ryan Carson:And since then, thankfully, it's just blown up. I basically figured out product market fit. And everyone who's listening, there's probably a ton of startup founders out there. You know, you're trying to find product market fit. You're pivoting fast. And I did what all founders do, right? You know, launched a version of Untangle, which is the product. Thought it was for consumers. Went out. Nobody, turns out nobody wants to use AI for divorce. They want lawyers, people. Just, I'm just going to say it. And then I basically thought, okay, not PMF. I'm old enough to know, like, and not hang on to things, just.

6:13Ryan Carson:pivot. And I was like, okay, I'm going to get a meeting with a lawyer and just see what they think of this thing. And she met with this amazing lawyer named Renee Bauer. And I was like, do you love this? Do you hate it? Is this like an enemy? Is it a friend? And she said, I love it. You know, there's a paralegal shortage. There's this discovery thing we do in divorce is a nightmare. I love it and I want to pay for it. And I was like, oh, how much would you pay? And she said a great number. And now we're B2B and we sell to family law firms and it's just exploding. So I've been trying to hang on to this solo founder.

6:46Ryan Carson:I'm going to do it all myself thing. And I'm completely overwhelmed. You and me both. And for folks that can't wait for the punchline, the punchline is Ryan Carson and Clairvaux, famous, famous solo founder. I can do it all myself. I need no friends other than AI. did our pre-show conversation talking about our regretful and imminent hiring of a bunch of humans of humans i'm gonna have to hire humans and if any of those humans i'm talking to are listening i love you and i'm excited about working with you now but but the ai yeah it's you get spoiled hiring ais i mean you know you give them 45 pieces of feedback an hour um you know you're Or you anyway, so yeah, going to hire some humans and grow the Untangle team.

7:35Ryan Carson:Very excited. But you haven't yet. And so, I mean, the good part of the story is you were able to get to rapid iteration, lots of prototypes, product market fit with you and our mutual intern, Mr. Devin. Mr. Devin. Mr. Devin. And so, you know, when you were starting out, again, you were like really coding, doing requirements, documents, all of that with AI, but you were very human in the loop. And as I've watched you over the past year, I've seen you slowly turn into AI bro who runs agents overnight. So tell us about what is in your stack right now? What is like very different from what we talked about a year ago?

8:24And how have you structured stuff to make you successful and build things effectively?

8:29Ryan Carson:Totally. I like to say I'm in max queue right now for those of you that are, you know, SpaceX fans. It's like the maximum pressure that I need to hire somebody, I'm doing all the work. And so, yeah, what am I actually doing day to day? So I'll walk you through it. I started off, you know, in a more specific agent like a cursor, like an AMP. and then I saw cloud happening. I'm like, okay, I'm really limited because I have to be at my machine and there's a lot of things I need to, you know, quick PRs, like here's a bug, need to sort it out or what about this piece of data? And I was like, why isn't cloud?

9:12Ryan Carson:Like, I feel like I should be working on the cloud and I thought, what about Devin? Like Devin, you know, they've been doing cloud for like years now. It was kind of crap in the beginning, right? but maybe it's good now. And I tried it and I was like, wow, this, this thing's really good. This is not, it's really good. And, you know, and I, I thought, okay, I'm going to try it. And it was just blown away by how mature the cloud agent was. And if you're out there listening and, you know, you are doing engineering, you're doing your work locally, you really need to open your eyes. Like, I think the future is pretty much 100%, you know, cloud agents, there's going to be some local dev.

9:53Ryan Carson:But so I started looking at that and was really impressed. So now, I went all in so much. So I spent at one point, I was like spending$5 ,000 on dev. And then it went up to $20 ,000 one month. And I'm like, Whoa, wait a minute, I have a big Twitter account, I'm gonna go talk to the cognition guys and tell them to give me free credits. And they did. So anyway, I get 20 grand of credits per month in Devon. Great. But the point is, I live in Devon. I live in the cloud, right? So all of my engineering almost, and it's Devon's right here on my screen. I've got probably 10, 15 threads and I bucket them in folders because I remember one of your shows, it was either you or your guest was saying, it's sort of like four or five concurrent tasks you can actually keep track of.

10:44Ryan Carson:and I agree with that. So what I use is folders. Is there any way to show it? I can show it. I'll show you one thing, which is this piece of paper. Perfect. This is the AI podcast we're doing where we say we're going to hire humans and we're just going to show you pieces of paper we scribble on.

11:07Ryan Carson:I'm doing this for a reason. Okay, so the reason why I have folders for threads and the reason why I have a piece of paper is because the skill is managing throngs of agents. Like that is what I do now. And to keep it straight in your head, you basically have to bucket these things. And so I have one bucket that I'll try to show this to you. Let's see how dangerous this is going to become, everybody. Okay, so everybody, this is Devin. This is my real setup that I'm literally working on today. And, you know, I think one of my big messages for today is that all of us have to up level our ability to manage agents like that is our job.

11:50Ryan Carson:Right. And so how do you do that? How do you effectively manage 10, 20, 100, 500 employees, basically? this is the whole reason that organizations were built as pyramids right like humans can't scale and have a thousand directs unless you're like jensen wong then you can but so what i'm doing is figuring out how to manage you know throngs of agents and in devon what what you're seeing is um is folders and you'll notice um i've got bugs right i've got p0 p1 p2 so it's sort of priority What are the things that I really need to move forward no matter how distracted I get today? And so I'm going to show you again my piece of paper.

12:31Ta-da!

12:32Ryan Carson:Okay, so I have two methods for this. Don't laugh. It works. So the way I stay focused is, number one, have a P0 folder. But then, you know, I've got a Dell 52 here. I've got eight screens concurrently open, right? And so the way I stay focused is by also having this piece of paper, which has weekly priorities on it. Okay. So it's all about weekly. What is important? And I'll cast my eye down here at this beautiful UG Monk system I have, which I'm not being paid to say that. It's a fun little to-do system. And that just keeps me grounded. Okay. So let's dig into this a little bit. Bugs is like, okay, there's some stuff going on.

13:14Ryan Carson:I want to get it pretty fast. It's not necessarily P0. you know let's look at a couple of those you know that's the one we're looking at here investigate red error retries right so devin's cranking on that um there's a couple other bugs in there p0 is the big stuff i'm not going to show you that but that's just like these are the these are the the main things i need to move forward as the business new features um usually p1 is like ah stuff that's important anyway it goes down um investors by the way tip oh my god like if you're not using an agent to help draft your investor updates and help you keep track of the business like clearly do that but here's the key build a skill to do it and this is probably obvious to everybody but another thing i love about devin is this idea has playbooks and playbooks are are sort of there's this weird mix of skills playbooks you know sort of uh thing going on but I've built an investor update skill basically, which does a lot.

14:17Ryan Carson:So this is most of where I do work. And then you'll find like, then that means I do a lot of work here. Like I'll be on my phone. And this is the other reality is like, unfortunately that means I can work like 21 hours a day now. And so you do. I do. And then, you know, and how do you deal with this? How do you deal with like the decision fatigue? Yeah. So I want to reflect. I'm going to answer that question in just a second. But I want to reflect for folks what I'm seeing here because you and I are different AI managers. And every time you come on, I'm like, no, I'm like, what an organized, what an organized individual.

15:01I am so impressed by you. You're like, here's my three steps to managing my agents. And I'm like, you don't just like float in the tokens.

15:09Ryan Carson:See what comes out the other side. And you're like, here are my like prioritized triaged folders for Devin. And I'm like, I just Slack Devin. Like he belongs to me. I just like a CenturyLink WTF question, question fix. And that works. There's nothing wrong with that. Yeah, it totally works. We're so different. But I do think, you know, something you said earlier, I want people to hear, which is like the job right now is to figure out how to manage agents. And the reason why I know both you and I are 27. So, of course, we're young. We're spring chickens. I have infinite energy. The reason why I think folks of a certain level of experience are like ready to cook right now with AI is all those management skills that you have developed over your career can be applied right here.

16:04Ryan Carson:Right now. How do you structure? How do you team? How do you prioritize? How do you delegate? When you have a team of 500, how do you still stay anchored on like the three things that are most important for you to do? Where do you micromanage? Where do you let go? Like all that stuff still. Pays off. Yeah. And it pays off. Yep. I think in multiples, like, so another practical tip I want to talk about is I have, it's a playbook, but it could be a skill if you're not using Devin. It's called Watchdog. And, you know, we're starting to pick up a lot of family law firms now, and they have a lot of cases, right?

16:42Ryan Carson:And I'm literally doing customer success and engineering and sales, right? And so it's like, whoa, how do you keep track of actually what's going on in these cases? is where are clients hitting UX bugs? Oh my God, how do you keep track of it? So Watchdog, what it does, and this is multiple iterations, everybody. So what you want is when that moment where you feel like, I need to know what's going in the business right now. I feel stressed. I don't know what's happening. This is when I run Watchdog. And what it does is it basically, it has a skill that's repeated for each firm. So think of these as customers, right?

17:24Ryan Carson:It goes into each account and says, what's going on with this account? Like, what was the activity since we last did Watchdog? Where are the errors? Where are the Sentry errors? You know, where does all this happen? But then you don't want just data. You want it to be transposed and filtered and sorted for you. So then it comes up with the top three things that are going wrong. And then either like, well, wait, have they been fixed? Because we're shipping a lot of PRs. Like I'm shipping probably 40 PRs a day now. So did it get fixed or is it currently being fixed? Or is there an open PR that never got merged, right?

18:01Ryan Carson:And it just gives me all that. And then I talk to that thread about, okay, let's dig in a little bit. And that's how I manage the chaos. Yeah, I, you know, you asked me, how do I manage the decision fatigue? And I feel like I start a business for every child I have. So like three kids, three businesses. I have a lot of decision fatigue. There's a couple things that I do. I approach it very differently, which is I think about bucketing my time more than I think about bucketing the tasks, which is like there's time in the day that I do like very, very specific tasks where I'm co-piloting with AI.

18:37There's other time where I'm just like I dedicate to spinning off everything or checking in.

18:42Ryan Carson:So I have to, as a solo founder, I have to bucket my own time on things. The other thing that I use is I use decision fatigue as a prioritization heuristic, which is functionally. If there is a feature that I'm working on that I let die on the vine, that is my. That's your symbol. That's my symbol for maybe this one is not the one I really care or believe in. And so I sometimes think that people, you know, don't tell anybody, I know I'm a token maxer and you should let your agents run 24 hours a day and ship as much code as humanly possible and software everywhere. And I really don't ship more than I think the market wants.

19:30Like I really try to like constrain my output capacity, not on quality, not on bugs, but like because I don't think I get... multiples of quality off of multiples of output.

19:44Ryan Carson:I totally agree. And so I would rather reserve that creative energy for something else, or at least keep it inside till there is something there. And so I do, I run less overnight. I run like bugs and triage and all that kind of stuff. I also work with an engineer, so we kind of ping pong overnight. But I don't do the sort of like seek out ideas, solve them, and always ship them because I just don't want it. It puts too much on me. And too, I don't think it actually adds value. I agree. I actually think we're nowhere near any frontier model having the intelligence to know what to ship. I agree.

20:21Ryan Carson:It's like a million miles. I mean. I agree. So, yeah, I think, you know, I think we all wanted these sort of like automatic improvement loops. And they just don't work with product. They're kind of a joke. And I tell people it's also because all of this intelligence has not magically created markets where there were none before other than AI markets, right? It's not like all of a sudden there are these like buyers that have shown up for solutions. Yeah, for stuff. And so I do think there's this like mismatch between the quantity of output of code and like commercializable product. Amen. So I want to double click on that.

21:03Ryan Carson:I think what is happening is people are not getting out of their chair enough and actually talking to real people. So, you know, the way I figured out PMF for Untangle was, you know, I went outbound with an email. I did a Google meet with a potential customer and I talked to them. Boom, that was step one. But then step two is like as soon as we landed them as a customer, I was like, can I come to your office? And I just want to sit and I want to talk. And this is all like obvious product stuff. But I think because we can ship so much and build so much digital product, we're not getting out of our chairs enough.

21:43Ryan Carson:I totally agree. So people go talk to humans for real. Go talk to humans, yep. And build stuff they need. This episode is brought to you by Jira by Atlassian. The teamwork graph in Jira delivers 44 % more accurate agent results with 48 % less token usage. That's a huge difference when working with AI coding agents like Quad, Cursor, Codex, or Copilot. The hardest part of shipping with AI isn't the code, it's the context. What's the right ticket? What did the specs say? What got decided in Slack? The Teamwork Graph pulls all of that from across your entire stack, from Jira and Confluence to GitHub, and feeds it directly to your agents before they write a single line.

22:26You assign the work, the agent gets everything it needs, and a PR surfaces when it's ready. No digging, no context switching, no broken flow. Same team, smarter agents. Try it free at jira.dev. That's J-I-R-A dot D-E-V. The other thing I want to touch on, because I think you breezed past it, but it has been a game changer for me in terms of using background agents, And in particular, Devin, which is I think people have still bucketed these cloud coding agents as coding agents, which is like their only job is to write code for a product or software product. And in fact, Devin runs DealDesk for me.