What are Agentic Loops?

0:00Everyone is talking about agentic loops, but the reality is most people don't know what it is or how to use them. In this ep, I brought on Professor Ras Mic to clearly explain what it is, is it hype, is it real, and how to use it. And if you stick around to the end of the episode, he shows me the most concrete use case of agentic loops that you can use starting today. Enjoy the episode.

0:35Ross Mike, welcome to the pod. By the end of this episode, what are people going to learn? You're going to understand what a loop is. You're going to understand why people are fanning out about it. And you're going to understand why it is a terrible mistake. And unless you have money to burn, that you are not to do it. I'm also going to play the other side and I'm going to show you a loop that I use. But the general consensus, I think is wrong. And we're going to talk about it. Okay, so by the end of the episode, people are going to understand what an agentic loop is, why the most well-known people in the AI industry are obsessed about it.

1:09You're going to keep it real with what we need to know about it and what we can avoid. And you're going to show a real use case, a real example of how to actually use an agentic loop. Exactly, exactly. All right, bro. All right, let's get into it. So as always, a lot of people love the diagrams. So we're just going to start with diagrams. I paid like like I think three dollars to get these stick figures. So I hope people appreciate them. This is me and you. Right. This is your average Josh Mo who does not work at Anthropic or API or OpenAI. And this is Boris and Peter and anyone else who has unlimited access to models.

1:52Now, the way me and you have been working, this is what is called a human in the loop is you and I will prompt our computer. Right. Let's say this is our computer or better yet, I'll say this is our AI agent. Right. Whether you're using cursor, Claude, codex, doesn't matter. You are prompting it yourself. Right. You're telling it, hey, build me this landing page, build this feature, X, Y, and Z. You are communicating with an AI agent, a platform of your choice via a prompt. And then a result is generated, right? A result is generated. And usually what you and I will do is we will view this result, we will test this result, and we will keep on iterating.

2:37This is the loop where it goes back to us, right? So let's say I'm working on an app and this app is a to-do list app, Greg. The first thing I'll probably want to do is I want to build up the landing page because I want to get this out to the public. So maybe they can sign up and join a wait list. So I'll prompt and build me a landing page. And let's say I like the landing page. Next, I'll work on authentication. And then once I'm happy with authentication, then I'll work on with the back end. So this is what we are used to. And this, to be sharp with it, is called human in the loop, meaning it is the agent that's building, but it is you that is directing, governing, and allowing things to happen.

3:20What everyone has been talking about, particularly Boris and Peter, they said they don't write prompts, they generate, they build loops. And essentially what they're talking about is they're building a system. And I'm just going to show you here where this is the AI agent. Right. And then this is the result. but instead of a human being in the loop the human is in the loop one time meaning it fires off said loop but then the rest of the time it's the agent checking it's the agent generating a result that result is then fed back into the agent the agent then looks at the result and continues to work now this in theory sounds cool because what essentially we're saying is hey i'm just going to have some sort of spec.md file or some prd.md or whatever.md file.

4:16And this is going to be like a to-do list, a task list. And this is going to give all of the information the AI agent needs to build this. Now, this sounds cool and this low-key might be the future, but here is where it goes terribly wrong. First and foremost, I want to paint this analogy. Let's say... Quick break to talk about today's sponsor, CodeRabbit. We talk about agendic loops in this episode and most of them are still hype. CodeRabbit is one of the few that actually works. My team literally uses it every day. You open a pull request and CodeRabbit reviews it automatically. It catches bugs, it suggests fixes, and it's one click to commit.

5:02It learns your team's standards over time so it just gets better and better. They just shipped CodeRabbit review, which takes a big, messy pull request and reorganizes it so you can actually understand what's changed and why in minutes instead of hour. Something I've wanted for years. My team at LCA and Idea Browser love it. There's a 14 day free trial. If you want to learn more about CodeRabbit, go to the link in the description. We're building a startup, right? You and me, Greg, we're building a startup. we hire a very smart developer and we tell this developer, this is the app we want to build.

5:37These are the things that it needs. And the developer goes on and builds the entire thing without consulting us. In building that entire thing, that developer is going to have to make assumptions, right? Assumptions of how the product looks, how it's going to feel, certain architectural decisions. There's a lot of assumptions that are going to be made in the nitty gritty. Now, you might think your plan document covers everything, but truth to the matter, it never does. There's always an edge case. There's always something that's missed. So what the developer is going to do is that developer is going to make a lot of assumptions.

6:08Those assumptions might not be aligned with our product vision. Now you have a developer who's come with a finished product, and now there's a bunch of things in order, but it's not the way we want it. In the same way, when you have this stacked PRD.MD file or this whatever markdown file you have, and you give it to an agent and you run it in this loop, meaning it takes the feedback, it takes the result, takes it as feedback and continues to generate code, what happens is you now have an agent that's going to make assumptions. And believe me, when you give the agent the floor to give assumptions, most of the time, it's going to get it wrong.

6:44But not only is it going to get it wrong, it's going to burn a lot of money. Now, I say this with all love, but Boris and Peter come from a place where they have no token budgets, right? They can burn unlimited tokens. If I had unlimited tokens, I'd be doing the same thing too. But this is not productive. This is great for research. And I'll actually share a loop that I use. But this idea of constructing like a meta harness where you give the agent feedback automatically, like it gets like the information, the result it's generated, and it loops on it. It is a catastrophe. And we've tried this, right?

7:24We had Ralph Loops, we had Ralph Wiggum. There's even like Slash Goal, which has been pretty popular the last couple of weeks. These are great to build prototypes. These are great to experiment with. Like, let's say you wanted to experiment with something. You wanted like some minuscule tool built out, but you didn't care about the nitty gritty details. These are great. But if you're, if you care about the details and you don't have tokens to burn, this is the worst thing to be trending right now in my humble opinion. I'll stop right here just to make sure, Craig, I'm making sense because as you can see, I'm pretty passionate about this.

8:00So Slash Goal is also trending at the moment. You know, is Slash Goal a loop? Like how should people think about Slash Goal and a loop? They're all the same thing. They have different names like Slash Goal. I know on Cursor, I think it's Slash Loop. And then on another tool, it's Slash Whatever. They're all the same thing. And basically how all of them work on a high level is you, you, you know, you type in slash goal, and then you give it some prompt, right? Like you give it some prompt, and then you can also like attach some, you know, markdown file and you tell it like, yeah, build this entire thing out.

8:34Don't stop until you're done. Don't make any mistakes. Again, these are cool. But the two issues are Number one, they burn a lot of tokens, right? And if you are not like this shouldn't even be a thought if you're not on the$200 a month plan, like not a thought. Like if you're on the$20 or I think there's$100 a month plan, you shouldn't even think about this, right? Because it's just going to burn your token usage. Number two, you think your plan is good, but it's not. Because it's impossible for you as a human to contextualize every single detail about the product that you want in one document, right?

9:13Things evolve, trends change. You know, one day liquid glass is cool. The next day we're changing how liquid we want it. Like it's very impossible for you to fill like your thoughts and exactly how you want the product to be one in one in one document. If anyone works in service, whether you run an agency or you like, for example, we develop software for other people and companies. We try all the time to get all the thoughts out of someone's head. There's always something. There's always, oh, you missed this or I wanted it like this. This is what I meant. How much more do you think an AI agent is going to understand you if we as humans have hard times understanding each other?

9:51Right. So this should only be used in the following experimentation. Experimentation, like let me do a new line. Experimentation, right? Let's say you wanted to, like I'll share with you a fun, cool little tool I built the other day, Greg. I was doing a talk and I wanted to build an Among Us simulator for AI models, right? Basically, it's this game where there's one bad guy, one imposter, and everyone's trying to figure out who it is. And I wanted to have like my own benchmark to find out which models like are good at lying. And I didn't want to, I didn't care about the details. I didn't care about how it looked.

10:26I just wanted the simulation and the benchmark to work. So I told it, I want the simulation. I want this benchmark. I want it to do this. Go and do it. It took about, I think, an hour and a half and it got it done. Now, there were a lot of details that I had in mind that it completely got wrong that I didn't specify in the initial instruction set. but guess what because i didn't care about the what it built in a sense that i didn't really care for the details it was a great thing i didn't spend a lot of time i just got slash goal to take care of everything but when you and i are trying to use ai to build something meaningful i 100 100 100 stand in the fact that the human still needs to be in the loop ai can replicate sauce It can't create sauce.

11:12So if I just have these giant loops running and then once they're done, maybe I'll go in and fix things up. Sure, you can make that argument, but I hope you have money to burn. Right. And that's the ultimate thing. This will burn money. It sounds cool, but it'll burn money. What I'm hearing you say is that the loops are going to create a slot machine. slot machine that's basically what it is now i have no doubt the boris's and the peters are building very sophisticated like loops like i can imagine like let me drag this over to here i can almost imagine they have something like again i i i'm not sure this is just me guessing but i can imagine they almost have like some sort of um test suite right where like they write uh tests for the agent to run the code against so it's a certain type of quality i'm sure they also have some sort of browser uh browser use capability so the agent can see the page live and can take screenshots like i'm sure they have an insane harness or meta harness around the agent so that this loop can be more successful than the average loop but at the end of the day the one argument I'll fight back with is this is going to burn a lot of tokens.

12:33And if you don't believe me, all you have to do is look at Peter's tweet where in one month he burnt$1.3 million worth of tokens. But I don't want to sound like a, you know, like a, I don't know what the word is, like a doomer, like, oh, like, like an old guy, like these suck. There are use cases and I'll share one, Greg, if that's okay, where my, my code review process is a loop. And I'll explain how it works. So I use cursor for the most part, not sponsored, I use cursor for the most part as my harness of choice. And with cursor, I will use GitHub as my source control, basically a place where I store code, version code and all that stuff.

13:21And every time I push a feature, like every time I build a feature, I push a feature, or whatever the case may be, I am pushing code to GitHub. And in GitHub, I have a code review agent installed. There's many kinds. My particular one that I use is Greptile, but I know people use CodeRabbit, Macroscope. They're all great. I use Greptile. And what happens is whenever I push a feature to GitHub, the code that's being pushed to GitHub is AI generated. But then I have a code review agent that reviews the AI generated code. And what's cool about Greptile is it gives me this review, right? It'll be like, oh, you missed this.

14:02There's this security thing. This is broken in this edge case. It's pretty good. But my favorite thing is it gives you a score, a score out of five, right? It could be two out of five, one out of five, five out of five, four out of whatever. It's a score out of five. And the mental model I now have is I will not push anything to production, meaning I will not allow code to go live unless the score is greater than four out of five. If the score is not greater than four out of five, this code needs to be reviewed. Now, here is where I loop. I have this skill called grep loop. And basically, again, I don't want people to think it's complicated.

14:44I just want you to understand where loops make sense. It's basically a skill that tells the agent, oh, check GitHub, read the review, and then fix the review, and then push to GitHub. So what happens is when I see a score, let's say I got a two out of a three out of five. Again, my rules is that it has to be at least four out of five and greater. So what I'm going to do is I'm going to go back to cursor, I'm going to write grep loop. And then when I write grep loop, what happens is cursor reads the review that Greptile wrote on GitHub, and then it feeds the review back into cursor. Cursor then makes the changes, pushes the changes to GitHub, and then waits for Greptile to do a new review.

15:24Every time you push to GitHub, Greptile does a new review. If the review still is a three out of five, guess what happens? The loop continues, and then more changes are made. And then let's say it's a four out of five. It doesn't give up. It takes the feedback and then pushes it back to GitHub. It won't stop unless it's taken five turns and then it'll give up or it won't stop until it gets a five out of five. Now, this is basically a loop. But if you notice this, Greg, this is a very closed off, very goal oriented loop. Essentially, I have a feedback engine, right? I have a code review agent that's giving a score.

16:03What I'm telling cursor is read the review, understand it, and get that score to a five out of five. This makes sense for code review because there's a fixed feedback loop. But when I'm building an app, again, I have no idea what I want completely in that very moment. So it's very hard for me to generate a loop on an app that I have in my mind, but I can't even fully visualize just yet. Now, Now, if you're great at visualizing, you're a master. You never miss details. You've never forgotten your auntie's birthday. You don't forget your wedding. You're just perfect. And you have a million dollars.

16:40Go ahead and build loops. But for me and myself, the only place a loop makes sense is in a very confined, constrained process with a very fixed feedback loop, a very defined feedback loop. And that's in code review. And can I be honest with you? This loop actually quite breaks at times. It's not perfect. And you know, when it breaks, anytime I push over 1000 lines of code, 1k lines of code, like if the if the code that it has to review is more than 1000 lines, I can almost never get a five out of five, because it's too much code for the agent to fully review and contextualize and understand. So even in this fixed sort of ecosystem loop that I have here, even here, there's reasons and places for it to break, right?

17:31So every time I push a change, I have to make sure it's 1K lines of code or less, or I have to tell the Asian cursor, split this into multiple PRs, multiple code pushes so Greptile can review. I say that all to say I'm not a hater, but loops just don't make sense right now, especially for building apps. They make sense for code review. They maybe make sense for like you are trying to do some SEO and you have like an SEO formula and you want like 300 pages generated and all the pages look and sound the same. Go ahead. But for anything that requires a slight bit of creativity, unless you're looking to donate money to companies that are about to go public at trillion dollar valuation, this just doesn't make sense to me.

18:19The person listening to this podcast, I mean, it's literally called the Startup Ideas podcast. They're building apps. So what they're doing is they want to create an app, a website, a startup, a SaaS, a micro SaaS, an agent-first startup that has the highest likelihood of success. And in order to do that, you have to show your app to people in order to get feedback. So what's missing from the loop is there's no sharing your app for feedback halfway through. you're just pressing or slash goal basically think of it as full self driving you're going from Miami all the way to Charleston, South Carolina and you're pressing go and you're going to go off and you see a really cute diner on the side of the road and you're going to go order a fried chicken sandwich you can't you're on this ride and whether you like it the train has left the station that's what this is and so my my belief on loops is actually a lot similar to yours by the way i loved your rant i apologize and again if any of the companies are planning on sponsoring you whatever i love you guys but this is i can't lie to the people you know what i mean like i gotta be honest like i've seen a lot of people excited about this and an idea this is cool but like i I know some people got$20 subscriptions,$100 subscriptions.

19:50This will burn through that and it's not productive at all. So yeah, I just have to keep it honest. I think where the output is binary, meaning black or white with no creativity, there is a room for loops. So that was my – when I was reading all that was going on, I was like, okay, to your point, like with CodeRabbit or Greptile, CodeReview or SEO or those pages. it's binary. Either they did the job or they didn't. So I think there's room for it. But for the people listening to this that are like, I'm going to go build a startup and I need a loop to go build that startup. Make me a million dollars, make no mistakes.

20:38That's where I think that this is a little bit misleading. That being said, to the credit of Boris and to the credit of Peter and to the credit of the people who are talking about it. I do believe that we will get to a point. 100%. At some point in the future. 100%. In the future, right? That this will be possible. Just not now. Just maybe not as of recording June 9, 2020. And again, like I don't fault them. Like I don't think they're malicious at all. But it's like if you have like I'm telling you, if I had a limited token budget as well, Greg, why the heck would I prompt like tokens don't matter to me?

21:21Right. And it makes sense. They have to experiment. They have to like they need to work on self-healing agents and all that type of stuff. So their position makes sense. My issue is everybody else who's creating content and teaching these things, saying, oh, this is creme de la creme. It's like, no, unless you want to donate to trillion dollar companies, this is not it right now. It could be it in a month and I could look like a fool. But right now, this just does not make sense. Human in the loop is the best loop. Thanks for clarifying it and keeping it real with us. Ross Mike is his YouTube name.

21:59I'll include a link where you can follow him in the show notes and in the description and on X. thank you for coming on here you were just the person I needed to come on the show to just say go off king you were a loop you were a loop in the sense that you were the loop around just explain this clearly and keep it real just don't stop well I appreciate you for having me Greg as always it's a pleasure man thank you I'll see you next time bye everybody