Building AI Agents (Clearly Explained)

0:00Ross Mic, welcome back to the pod.

0:01Ras Mic:By the end of this episode, what are people going to learn? I hope I'm going to share some wisdom on how you can use the agents better. There's a lot of information going on right now. I disagree with most of it, and that's what we're going to talk about. So at the end, whether you're building something, using an agent for some sort of work, you have the best output possible. And is this going to be a technical dive or, you know, non-technical person can... Anyone can watch this. There's going to be a lot of diagrams. That's all. you're gonna make it clear to understand the concepts right easy okay basics let's go so

0:42Ras Mic:the first thing that i want to announce previous episodes we probably disagree with this point but now what's true is the models are good the models are exceptionally good opus 4.6 is amazing gpt 5.4 is amazing. I know there's like two sets of camp where, especially when it comes to programming, people are like, oh, Opus is the better UI designer. GPT 5.4 is a better backend. Generally speaking, we've reached a point, we're not at AGI yet, where we reached a point where the models are good, but context still matters. And you have the power to steer the models in a direction where you can get quality or you can get slop.

1:18Ras Mic:And that's what I really want to talk about. But before we get into all that, and feel free to cut me off because this topic excites me, we need to learn how context works. And context is the model assembling information that it needs to execute an action. And the way the context is assembled, let's say in a coding agent, but really in any sort of agent, is there's this general system prompt, usually by the model provider. So for example, Cloud Code leaked recently. And one of the cool things that, especially as a developer, I got to do is I got to read the system prompt. So they have this general system prompt that guides the model on how to act, what to do, what not to do.

1:56Ras Mic:The system prompt is very important. And then you have a lot of people have agent.md files or claude.md files. Now, I'm just going to say off rip, 95 % of people don't need this. The reason being is, again, you have to assume that the models are already good, right? Now, imagine I told you, Greg, every time we're about to shoot a podcast, Greg, you need a microphone. You know, you need a microphone, right? You've done this plenty of times, right? So if I'm building, like, let's say a website with cloud code, and I'm telling cloud code, this code base uses React. I don't need to, because it has the code base in context, it can check the code, right?

2:34Ras Mic:So there is this disparity where a lot of people are putting a lot of onus on the harness and the context building. And I'm low key starting to strip things off. Like I'm going super, super minimal. Because again, not to sound like an anthropic or OpenAI shill. Unfortunately, I have not been acquired. None of them are paying me, but the models are really, really good. Wait, so 95 % of the time, I don't even need to bother with an agent MD file? You don't. Unless this is some sort of proprietary information. Yeah. What is the 5 % of the time I should care about it? Proprietary information that maybe is specific to your company or some methodology that is specific to you that has to be referenced in every single conversation.

3:17Ras Mic:Because the annoying part with an agent.md file is every time you go back and forth with the agent, it's added in the context, right? The cool thing about skills, and I'm going to talk about skills in a second, the way skills are designed, the skills are used in a way that's called progressive disclosure. Meaning when you have a skill file, the entire thing isn't added to context. It's just the title and the description. So the agent has the title and description in the context. And when you, let's say you have a Notion report skill, right, and you tell your agent, hey, I want you to create a Notion report.

3:51Ras Mic:It's then going to check its context and be like, oh, I have this skill. Let me check out the entire document. So it's not in the context. What's in the context is the name and the description, but that's enough for the agent to be like, oh, this is a skill I need. Let me go use it, which is fantastic. I'm a skills maxi. And I'm gonna show later in the episode, like how you craft the perfect skills. But with agent.md and claw.md files, it's context being added at every turn, right? So let's say you have like a thousand line file claw.md and let's say that's like 7 ,000 tokens. You're spending 7 ,000 tokens on every run.

4:29Ras Mic:Now, do you need to? Most likely not. It probably should be a skill. But if you have some sort of company proprietary information or like there's something specific that you do that the model needs to know at every single turn, then you use it. The thing is 95 % of people don't have that, right? So I'm not a fan unless that's the case. So, and the reason being is we're wasting tokens, right? It's in every single turn, but this is where the beauty of skills come. I'll show my screen here. Your skill, again, this is not like word for word how it looks, but a skill basically looks like this. There is a name, there is a description and then underneath is a bunch of information i'm going to put a bunch of info what when you create a skill.md file what gets added into the context is actually just the name and the description right the bunch of info doesn't get added so imagine you have two sentences versus an agent.md that has like a thousand lines that get added into the context we're talking in thousands of tokens compared to a couple hundred.

5:39Ras Mic:And the agent only gets the bunch of info when it realizes it needs this skill. So if I have, let's say, a certain way of generating a report, a certain way of structuring my code, why would I put that in the agent.md file when I can have the agent call on it progressively when it needs it, right? So this is why skills are honestly, like I'm a shill, I'm a maxi, but people do it wrong. And I'm going to share the right way on how do we create skills. So, so far we have the system prompt, the agent.md, the skills, and then we have the tools, right? So if you're using cloud code, there's already built-in tools, a read tool, a write tool.

6:16Ras Mic:Like there's many tools that it uses. This has to be added into the context because the model doesn't call the tools. Like it's the agent harness around it that allows it to call the tools. And then in this case, we also have our code base, right? Like whatever, if we're building a web app, a mobile app, I know most people here won't care for the specific framework. And honestly, we're getting to a point if you're not technical, you really shouldn't. And then we have the user conversation. So this is what the complete context window is filled with. Right. And this can total up to, let's say, like at the beginning, this could be like 20 ,000 tokens.

6:50Ras Mic:And as the conversation continues to grow, you might reach your limit of 25, 250 ,000 tokens. And that's when you see both Cloud Code and OpenAI Codex. They'll compact. Right. So beautiful so far. Right. This is how context works. Why skills are important and how you should generate skills. Let's say I have a specific workflow. For example, for my YouTube channel, we're at a point right now, Greg, where we get sponsors now. Crazy. When I first joined, when I first came to the pod, not a thing. We get sponsors now. It was just your mom sponsoring the channel. Yeah, it was just her showing love, feeding me.

7:27Ras Mic:But now we get sponsors. I get a lot of emails. and some are good, some are bad. And it's a lot of time, I'm sure you're aware, to comb through and to check. So I have an OpenClaw agent that has its own email, right? I haven't given it access to my email because there's like attack vectors and I haven't hacked before, so I'm very careful with these things, but it has its own email. And every time I get an email from like a sponsor, I forward that email to the agent. Now, the first time I told my OpenClaw agent, I'm gonna forward you emails, check every 15 minutes when you have an email. And when you check the email, do research on a sponsor and tell me if they're worth it.

8:05Ras Mic:That's all I told the agent. Every sponsor email I sent it, it was like legit, legit, legit, perfect, perfect, perfect. There was no rejection. There was no, this is bad, or these guys are a scam, or this product's not good. There was no deep research being done by it. So then I realized, huh, okay, the model needs a step-by-step guide. This is when I create a skill. But here's the problem. A lot of people will, I'll just write it down here, will identify they have a workflow, right? You have some sort of workflow, and then they'll jump to create the skill right away. This is the, let me click hide here.

8:45Ras Mic:This is the worst thing you can do. I'm just going to draw arrows to signify that this is bad. You don't do these. And the reason why you don't do this is imagine you hire an employee or you're mentoring somebody. Correct me if I'm wrong. You're probably going to tell them what to do. And if they ask you questions on how to do it, you'll help them. You would ideally like them to fail. And then you want to then tell them, no, this is how you do it. Like there needs to be some sort of experiential learning. The way I've been creating skills, Greg, and I have like 100 % hit right now when I tell my agent to do something specific, is I actually walk with it step by step on doing the workflow.

9:28Ras Mic:So in the case of my YouTube analysis, I told the agent, okay, I just sent you an email. Tell me about the company. Companies this, this, that, and that. Okay, check their Twitter. Check their YouTube. Check their Trustpilot. Check if they've raised any money. if two of these are have not if two of these don't exist are not in good standing automatic rejection it checked and it was like you're absolutely right i was using opus um these uh this is not a good company and then it would just we would we have a spreadsheet in google sheets it'd be like no contact it's so frustrating too right because you're like you give it a task and it seems like so binary like right or wrong and then when you tell it hey like why didn't you look at the trust pilot why didn't you see if they've raised money you're absolutely right yeah absolutely it's like what and and the thing is the reason why this is the case is the models actually don't think they're predictors of tokens right so when you give it english when i give it english it maps it on this vector graph and then it looks for the closest resemblance and it says this is the response right so when you say what is the capital of france it maps it again on this graph and it says, oh, Paris is pretty close by.

10:38Ras Mic:Then it gives you Paris. It has no, it doesn't think, it doesn't understand. It feels like it understands. It feels like it thinks. Heck, it even feels like it has emotion. That's because it's been trained on so much data, but it actually does not know how to think. And this is where a lot of people will be frustrated with like, why is it not understanding me? You have to walk with it. So I told it, okay, this is how you research. And it's like, okay, it researches. And guess what? This is part of the context. and we're like, okay, now that you're done researching, when it's a good company, these are the qualities you look for.

11:08Ras Mic:And then when it's really good, send me an email. And then once we had a successful run and we did it again and again, then I converted it to a skill. The reason being is a lot of people create the skills themselves, or I mean, they'll use the AI to create the skill, but it doesn't have the context on what a successful run looks like, right? Because most of the times, especially if you're using OpenClaw, it's probably gonna fail at the API call. It's probably going to call the data wrong. Like there's so many places it's going to get wrong. And I see a lot of people saying, it's just so frustrating.

11:38Ras Mic:This is terrible technology. Why doesn't it work? It's because you don't understand how an agent works, right? It will mimic you perfectly. You've given it nothing to mimic, right? So I will do the workflow myself. So the updated version is identify the workflow, go back and forth and teach it. So like I'm doing it, like I'll be like, okay, first do the research. Here's the result. and I'm like, what do you think about this? Oh, these guys are terrible. You're absolutely right. Okay, you should go to the Google sheet and mark this as bad company. I've done that. Once I've had that back and forth, then I tell the AI, review what you did and then create the skill.

12:22Ras Mic:So now it has actual context with how it worked and it's going to create the skill beautifully. I don't handwrite skills. I don't think you need to. You can use AI to do it. They even have a skill to create skills, skill exception. But you should have the context of what a successful run looks like. And this is why, by the way, Greg, I don't install skills. Like I've seen people like, oh, this notion skill, this social media skill, whatever. I'll review it. I'll check it out. I'll even give it to my AI and be like, oh, what are some things we can learn from this? But I don't download skills because your agent needs the context of a successful run, which you then turn to skills, right?

12:58Ras Mic:And this is the big thing I've seen. you see skills marketplaces, you see download this and that. First of all, it's an easy way to attack somebody. So I would be very, very careful with downloading some random person's skills. But second of all, again, it's all about context, right? It's all about, and, you know, OpenClaw has a memory layer and all these types of things. You want it to do the right thing. And the only way it can do the right thing is if you give it the proper context. And to me, the best way to create a skill is to work with it in your specific workflow once you have a successful run tell it okay review what you just did this is the skill you need to create i'll pause here i mean it makes sense right because if you hired an employee you would do the same thing yeah you wouldn't you wouldn't just be like okay go do this thing good luck yeah uh and by the way this is how you're going to go do things forever you would map out a workflow you would identify what right and wrong is you would uh do it iteratively and then once you've gotten to that point you would codify it 100 and i think like that's the thing like we should treat models and these agents like very new employees versus like these black magic boxes that like know everything right they know everything because they've been trained on a lot of data but they don't know your workflow, your steps, right?

14:22Ras Mic:So I see a lot of people who have, you know, 15, like right off the rip, they'll set up OpenClaw and 15 subagents, 30 skills, yet you haven't even set up your own workflows, right? And these things are cool right off the bat. And there's a perfect time to use subagents. I use subagents a lot. But the way you build, like, I call it scaling for productivity, not scaling for what looks cool, right? Like I've seen, like, for example, paper paperclip looks awesome cool i used it i loved it right but i think people would be more productive if they built up from scratch their own version meaning like okay you have your own like um you know like editor right content creator so you're you're asking people to do the work 100 100 and because the thing is it's like look i'm in the position where like people using like these beefed up things make a lot more sense for me.

15:18Ras Mic:And the reason being is like, I can build a product like that. Like, I know what your audience wants. I know what my audience wants. Like, you know, heck, I can spin up agents and build this thing. Right. But if I'm going to be completely honest, if you want to scale for productivity, it starts with one agent and you building up the skills. And then, okay, now you've built up some skills and now you add a sub agent and your one agent manages multiple agents. Right. Like imagine this, like imagine I start a company and off rip, I have 10 employees. Never managed a team in my life. Heck, I don't even have a really big family.

15:49Ras Mic:So like I'm alone, you know what I mean? So it's like you have to sort of, yeah, it's not sexy. And I apologize if this is not the cool thing people wanted to hear, but you sort of have to put in the work and build it up. And I personally believe you're building skills, like your personal human skills, not skill.md files, that when the models get better, when the agents get better, you will be more valuable. Because at the end of the day, as long as there's no new paradigm for models, LLMs just predict tokens. They don't understand or know the way you and I do. Right. And this is why, although like the job scene and all this stuff is scary.

16:25Ras Mic:I genuinely believe anyone who knows how these tools work and like knows how to build agents and like craft skills and like knows how to make them productive. We're in it for a good run. So you're saying that if you know how to do this, you won't join the permanent underclass. The permanent underclass. So is the permanent underclass basically like, I've seen this on Twitter a lot. Is that basically AI has replaced you? So now you're just. From what I understand, it's once AGI comes, all these white collar workers are going to lose their jobs. and if you don't know how to build skills use ai people say you're joining the permanent underclass that's that's the term it's permanent too that's scary so i just have a little bit of time left yeah by the way like it's ridiculous to call it a permanent underclass yeah because that's terrifying i can understand underclass but permanent it's like like you say there's no hope Yeah.

17:30Ras Mic:I mean, we are in like knowledge that took 20 people, 20 years to acquire is now like 20 bucks a month. Right. So there is like a huge shift. Right. People who are non-technical or I think I saw yesterday, like some guy hit like a hundred million dollars and he vibe coded the whole app. I think it was him. 1.8 billion billion yeah so you know what i mean like it is the there is a shift right and i think this idea of like i love how you were like billion you were about to just leave this podcast and just be like no you know what it is i just realized man i overthink things like i just need to drop the thing release the thing and there's like wisdom in that like there needs to be this level of delusion which i don't have like i'm trying to work on where you're like this is just gonna work out we're just gonna launch the product it's gonna succeed and if it doesn't on to the next one because 1.8 billion yeah dude like b b usd yeah we're not talking monopoly because it was canadian uh it's it's um we're not talking carny coins

18:40Ras Mic:we're talking we're talking real benjamin yeah yeah that makes sense that makes sense but yeah Like I hope this like understanding of like, again, I personally don't think you don't need an agent.md file unless you have something proprietary. Skills are valuable. Build your own, though. Build, build your own. Like, you know, like when you asked your mom when you were a kid, oh, can we have McDonald's? And she's like, we have food at home. We have food at home. Build your own skills. For coding perspective, from coding wise, a lot of the companies, model companies have realized that the agents are really good at writing code, particularly TypeScript.

19:18Ras Mic:And this is why there's been like you see this advancement with like Cloud Cowork and like even OpenClaw. Really what they're doing under the hood is they're writing code, right? They're writing code, calling APIs and all this stuff. So when it comes to building a project, you actually don't need skills or you don't need an agent MD file specific to the tech stack you use. Like I remember we used to I'm using React and, you know, Convix or I'm using Next.js and Superbase. I'm using this and I'm using that. And you put that in the agent MD file. You have like all these lines for the most part, unless again, you have a specific, specific workflow.

19:58Ras Mic:Unnecessary. And the reason being is code itself has become context now. So the more the more important thing is starting with a solid foundation. Templates used to be big back in the day. People made lots of money with templates. I believe templates are going to have a renaissance because if you have a solid like template, right, like whether it be like for a web app or mobile app, because that becomes context for the agent, it's going to build on top of that. Right. And again, I didn't need some large agent.md file. I didn't need any large cloud.md file. What I needed was, again, minimal context usage and skills.

20:35Ras Mic:So if there's anything anyone can learn from me is build your own skills, build your own skills. And there's this methodology. I don't know if I've shared this with you, recursively building skills. So let's say you've built your skill. Right. I have I'll draw a diagram because why not? let's say I have a workflow and after you like setting up my workflow with an agent I've decided you know what I'm going to turn this into a skill right so this is my skill.md now here's the thing even though you have the skill.md the agent at some point is still going to mess up because there's probably gaps in the information it has in the skill so when it messes up I'm going to work with it again?

21:21Ras Mic:How do I work with it? You messed up. Try calling the API again. Try doing this again. Or even ask it when it tells you, oh, I failed. I couldn't do this task. Believe it or not, when you tell the agent, why did you fail? When you ask it, like, what's the error that you got? It will tell you descriptively. Oh, I got a 505 error. You have insufficient credits. Like, oh, okay. So it's a credit issue. Fine. So I would tell it that. And then I would pass that failure back to the agent. So let's say it did something wrong. We identified the failure. All I did was asking it. I will give that failure back to the agent.

21:57Ras Mic:I'll be like, you failed here. This didn't work. Fix this. It's going to fix. It's going to write code. It's going to do whatever it does. Once it fixes it and it's done it right. Now you tell it with the new fix, update the skill. So this doesn't happen again. I have like for my YouTube channel, I have like a report generator. It calls Notion, Dub Analytics, YouTube Analytics, Twitter Analytics. It pulls from like eight data sources. There's no way you're going to one prompt and the agent's going to do it. But every time I tell it to do that work, it takes like 10 minutes. It executes it flawlessly.

22:33Ras Mic:Why? I went through five loops of this. Five iterations of recursively building this skill. And that skill is so good. I genuinely think if anyone's going to, if like skills marketplace is going to be a thing, there's going to be people who sell skills like really well defined like step-by-step skills because people are just creating them without having built out the workflow with the agent right so use the workflow by hand like telling it each step once it's done it completely create the skill.md file continue to use it it's going to mess up when he messes up you thank god you don't complain because a lot of people like oh it messed up i'm angry no this is a moment where you You identify the error, tell it, this is the error, fix it.

23:17Ras Mic:It'll fix it itself. And then you tell it to update the skill file so that this doesn't happen again. So that's a little bit about shifting your expectation, right? Because people just assume it's going to work in the beginning. You're saying basically it's not going to work initially. There's going to be two, three, five, six hiccups. And over time, it should be good. So this is most people's expectations. Right. Yeah. And the way I've personally experienced is it's like this. So there's like this early area of investment that you have to make that sucks that nobody will tell you, especially agent harnesses company because they wouldn't raise as much money if they did.

24:02Ras Mic:But like this, maybe I would give it two weeks because it took me two weeks. Like Open Claw, when I first set up Open Claw, I thought the same time, like, what is this garbage? much or like it doesn't understand anything it's confused and then i realized like oh like let me go lower level the models and the agents like they they don't think like you and me right i could i could tell you hey um greg we need a report on like you know the financials and notion because you're probably we're in the same business we work together you would understand based on the context you have the business what that means but imagine a new guy joins like yeah i need a report on the financials so where do i even you know what it reminds me i wonder if we can put this clip in but in the office you watch the office i am not an office watcher unfortunately there's a clip that uh there's a new boss and the new boss goes to jim one of the main characters yeah and he asked for a rundown so go go the office the office rundown oh no basically charles the whole episode is about jim trying to ask around and be like what what is a rundown like what is a rundown he's like calling his dad like what is a rundown you know what i mean he's just um he didn't have the context yeah he didn't have the context yeah and and it goes back to my initial point the models are really really good now but the context matters more than anything, right?

25:34Ras Mic:So when you see like these large agent, like companies and sub agents, and again, I'm not saying those don't work, but I'm saying probably won't work for you off rip because you haven't built it up to get to that point, right? So let's say like for me, for example, I started with one agent. Let me draw this. I started with one agent and this was like my main agent. This did everything, right? This checked my spreadsheet, this checked my sponsor's email and all these type of things. And once I had like predefined workflows, let's say for like working with sponsors, then I can actually have a sub agent.

26:10Ras Mic:What's the purpose of the sub agent? The sub agent does all the marketing stuff, right? But I'm not creating the sub agent for the sake of creating it. It's going to have skills. It's going to have context. And it actually makes sense for me to have sub agents, right? So I've built out my thing to like, now I have five sub agents i have one for marketing one uh for business one for personal and and that's it and i'm willing to bet if i want open claw to open claw with anyone my system is more productive because i didn't scale for what looks cool i scaled for productivity that was a bar that was a huge bar we gotta clip that i was just thinking that clip that's gonna rip yeah that was a bar What else do you want to leave people with?

26:53Or is this, this is the main point.

26:55Ras Mic:Yeah, like here's like the, we've got to the point where the models are good. The models are really good. The context matters plus the harness, right? So for example, there was this benchmark, although I'm not a hundred percent supporting it, that there was a difference between the quality of output that cursor generated versus cloud code versus codex, right? Right. So what that tells me is that we've reached a point where the models are really, really good. They're probably going to get better. The next iteration is probably going to get better. But the harness and the tools that you surround it, the context that you give it is going to matter even more.

27:34Ras Mic:And just like in everything in life, less is more. Right. Like building up step by step, making it productive for you first before you add the shiny new thing. like because i tried all these tools all the time like especially paperclip paperclip blew up and a lot of people have been talking about and it's fantastic but i'm willing to bet if people took two weeks to build up to the version because you can prompt open cloud to do all that stuff if they built up their own version of paperclip in two three weeks where like they're building things that they actually need their productivity level will skyrocket through the roof it's a hot take it's a hot take might get me in trouble no okay who's it gonna get you in trouble with maybe perfect clip releases a billion dollars and they don't acquire my podcast i think uh listen you're you're out there you're trying things and you're just sharing what you're learning in real time so if you're just you know things can change by the way yeah like two weeks from now it could be like no give the agent everything there's this new memory paper that google released and like now like it has the ability to index information and stuff but as it pertains to real life, less is more, simple is better, right?