AI Skills with Matt Pocock

24:15Matt Pocock:the only way that I've been able to not necessarily survive, but it took me a long time to figure out where I wanted to be in the AI space because I'm not a researcher from OpenAI. I don't have the credentials to like talk about this stuff really, especially in 2020, late 2023 was when I started looking at it. And I was initially making courses about how you put AI into applications. I thought, okay, I've been building fronted applications for a long time. It makes sense that AI is changing things a bit there, started to be clear to me that was the wrong bet to make. I wasn't sort of seeing the returns I was expecting.

24:51Matt Pocock:And I didn't, like the material was good and I feel proud of it, but I didn't think I wanted to make more of it. And around December last year, which is a date that many people cite. Oh, yes. We know, or as we call the kind of the winter break where everyone came back Yeah, exactly. The Peter break, right? The open claw break when Opus 4.5 is out, people have a lot of time off and they just start slamming it and they realize, wow, OK, things are really happening. And that happened to me too. and I realized, okay, the AI is now good enough that you can delegate to it. You can actually make structures around the agents and the agents can handle the knowledge, the syntax, the sort of, I call it the tactical stuff.

25:40Matt Pocock:Yeah. And you can handle the strategic stuff, the long-term thinking. I use that a lot. I know you had John Amsterhow on this broadcast. I've been really wanting to chat to him myself. Like he's a huge influence on me. And he talks about the difference between tactical programming and strategic programming. AI has largely eaten tactical programming, in my view, and it's up to us to handle the strategic. And I realized, OK, in the strategic layer, there's a course I can create there. I was looking at Ralph Loops at the time, sort of Jeffrey Huntley was building this really cool stuff where you can loop the agent and get it to follow these goals.

26:20Matt Pocock:and I thought, okay, there's definitely material here. I just need to find a structure within which I can put it and organize how to fiddle around with it. And I remember the Ralph Loops, you also made a video that became very popular on YouTube, everywhere, where you basically said like, all right, like here's how I created a Ralph Loop. Like here's, I have a project that has a lot of like to-dos and you did a great job and we'll link that video in the show notes below where you said like, all right, here's how usually we would try to get agents to work, like do a plan upfront and then implement each step, like the kind of traditional top-down planning.

26:55And you're saying the problem is that as you're implementing or even the agents implementing it, realizes, hang on, I need to do more stuff. And then how do you modify the plan and then enter the route loop where you gave the structures that you used at the time. There was like an MD file. It keeps it adds as there and it kind of like eats to it, but keeps adding. And it was actually a pretty eye-opener to me as like, oh, I'm a way to think about how to do these agents.

27:19Matt Pocock:Actually, the couple of years that I spent sort of trying to put agents into applications was really beneficial there. Because when you try to build an app that contains an agent, you're always thinking about data flow. You're thinking about how the data is going to get in, what shape it's going to be, what priority, whether you're going to put it in the system prompt, the user prompt. You're working at a lower level than you usually get to with the harnesses. And so when I got to working with the harnesses, it felt like, oh, this just feels very familiar. I just need to, you know, where is the state going to live?

27:49Matt Pocock:How am I going to pass the state into the agents? What shape is that going to look like? How am I going to compact it or clear it? You know, and this was, you know, still pretty early days of Claw Co. Claw Co had been out five, six months at that point. And it just felt very natural. And from there, I just got obsessed with these, I suppose we would call them loops now, but really They're just processes. They're sort of different ways of stringing agents together. It's kind of like the diagram, like if you can draw arrows from one thing to the other, that you can call that a loop, especially when there is a port that goes back or you can call it a workflow or whatever, right?

28:26Matt Pocock:I would call it a finite state machine. It felt very similar to the stuff I've been working on in X state, which is process based, which is state based, event based sometimes as well, where, you know, you have an agent at the bottom there that's calling an event back at the top. And that's I started just to see really good results from that. And I would do these experiments where I would try sort of building out my process and I would build out a feature of, you know, I have a few apps that I work on kind of to extend what I do. Like I have a custom video editor. I have a huge thing, a huge code base.

28:58Matt Pocock:I have a few open source projects as well. And I was just building these little loops and little pipelines. And I would sometimes just drop it and go back to what the default setup was. And I just noticed a huge difference. Like I just felt, wow, OK, the stuff that I'm doing here is really setting me up for success. And I started thinking, what's the best way that I can distribute that? How can I share that with other people better? And that's where I sort of started landing on skills as the distribution mechanism for this stuff. These are the skills for AI bots, harnesses, where you can typically define them.

29:36And now you can, once you install them, you can invoke them with a slash command.

29:40Matt Pocock:Skills really, they're just a folder of markdown files that can sit in your computer somewhere. And the agents can either invoke them themselves, so model invoked skills, or you can have skills that the agent doesn't know about, but you can invoke yourself. So user invoked skills. and I started seeing these skill sets pop up everywhere like superpowers and, you know, clawed code plugins that you can install. I think GStack as well. I realized, okay, maybe I can distribute what I have, this process as a set of skills and see what people think of it. And initially I just put it up and I was doing other stuff.

30:15Matt Pocock:I was working on a course and I checked back in and I realized, oh, it's got more stars than anything else I've ever done. I've not even really talked about it. It's just sat there on its own. Word of mouth, I suppose. I've done a little bit of documentation, but really not much. And it's just exploded already. So I thought, OK, maybe I should put a little bit more work into this. Maybe I should talk about them. And I did a talk at, I think, where we met last, which was AI Engineer London, about in April. That talk was entitled Software Fundamentals Still Matter. And that talk is now up to, I think, 1.2 million views or something.

30:52Matt Pocock:And I mentioned the skill set. The skill set is now at 230 ,000 stars. It is now the second most starred skills repo in the world. I think somewhere like 20th to 25th of the most starred repos of all time. Wow. You know what I mean? Like, what's going on? So there's obviously a hunger for this. So this was the second time in my career, just like when I was putting out the little TypeScript videos where I felt, wow, there's a momentum here. There's something happening. And so I felt I had to double down on that. The skills, how did you write them? Is this trying to capture your workflow, your understanding of what works with Asians?

31:30You know, not just right now, but of course, you're thinking about state. You're thinking about how you were integrating AI into applications, which, again, didn't take off all that much, but you learned. So is this kind of like Matt's workflow, Matt's way of what works for me?

31:46Matt Pocock:Yes, that's what it is. I try to think, first of all, people are going to use these skills and they're going to tinker with them. So how do I make the simplest set of skills that people can audit very easily? I'm trying to think, how do I maximize people picking these up and using them at work? So, for instance, the grill me skill, which is the most popular one. Yep. You know, I don't know if you've used it. I use it as well. Yeah. Okay. It's annoying how damn it grilled me. I just asked it. I was like, I'd like to expose an API endpoint that can can tell whoever has the authenticated token, have some basic authentication.

32:26Is this email a subscriber to my email list or not? Because I want to connect it with one of the events that I'm doing with to get priority to pay subscribers. And that's very simple. Right. And then the grill me thing, it starts to just really grill me like, OK, so what about authentication? Do you want the bear token or do you want it in JSON, which is not as safe, etc.? I'm like, OK, well, that's a decision to make. And then we go through all of these decisions and it goes really low level, including like, OK, like how do we enforce rate limits? When it comes to rate limits, you want to exactly do it when you get like a thousand per day and not allow single more, which is more complexity.

33:02And I just realized it's been a long time since I've had such an involved design discussion with with a team or an engineering team. And you typically have it when someone has deep domain knowledge. And I was both annoyed by, you know, this is just simple, like, no, don't worry about that, but also impressed that this thing, this AI, this LLM through a series of prompts is able to do all of this.

33:23Matt Pocock:It's, I mean, everyone's got a grill on my story. So I get so many of these at conferences, people say, you know, you've this, this very, very simple skill. It's really just telling the agent to interview you relentlessly about the topic. It's a very small skill. It just has this weird emergent behavior with it where the models just start thinking a little bit outside the box and they start throwing ideas at you. I think I got it originally from like a Tariq who works at Claw Code. He's saying basically get the agent to interview you and then you'll see better results. So I encode that into a little skill and I realized, wow, OK, it's just sort of 10 times better than anything I've ever used.

34:00Matt Pocock:And that grill me skill was the first one. That sort of reminded me of the discussions I would have at my first job, you know, with the guy with the sandals. It's this very senior engineer in the room really getting me to think about everything that I'd done. It was the most familiar thing to me to actually working with someone like Anderist Rake at Xstate. You know, it just felt like a really high quality developer was asking me these good questions. And I thought, wow, OK. And then I started sort of taking that and going, like, how do I mine this agent for more software fundamental stuff? How do I make it feel more like a proper developer, a real senior?

34:41Matt Pocock:How do I tickle the right latent space in order to get its behavior to change and challenge me in an interesting way? Because if you can do that, if you can increase the quality of the conversation you're having with the agent, you're going to increase the quality of the outputs. What I liked about the grill me skill is it forced me to make decisions that I know what decision to make when I think about it, but it is my decision. So unlike when I tell the when you do the slash goal command, like build this and it goes off and does this and it makes all the decisions or most key decisions. What I like about grill me is I both make the decision, but also sometimes it reminds me about things that I didn't think too much about.

35:23Or maybe it reminds me that I should do a bit of a research, for example, like it asked me, like, which authentication would I want to do bear token or over posts or even over get? And then I'm like, hang on, like, I'm going to look up like what the differences are or ask a different session to educate. So like it makes me a better professional. And I do have this belief that when you're working with AI, like as long as we're learning, I think we're fine. As long as we stop learning and outsource the learning to this thing, trouble will be brewing maybe, you know, months or years down the road.

35:55Matt Pocock:100%. There's two things there, right? I think what everybody underestimates about agents, everybody, is that there is a communication gap between you and the agents, right? There is a barrier there. You feel like because the agent is not a human and because you understand your hierarchy of values, you think that the agent will just pick up on them, right? There's this sort of feeling of, yeah, just trust the model, especially with the top tier models, you know, just trust the model. But the agent, however good it is, however smart the model is, you know, even mythos, it can't read your mind. It can't read your mind.

36:28Matt Pocock:So you have to, there has to be some process of communicating your values to the agent. Because often when you do it like a goal, when you just go, okay, just spam me out some code, give me some slop. The agent is going to produce something that's totally misaligned for me because it doesn't understand what you think is important. And so Grill Me is not only about implementation details, it's also about establishing, okay, do this. This is in scope. This is not in scope. Here's what I think is important. And so it's the agent getting to know you. Matt just described the grill me skill. When I used this skill to design an API endpoint, the first questions it asked were about what the endpoint was and wasn't allowed to do and who it was allowed to do it for.

37:07Now, in my case, I had a decent idea of what I wanted, but it's generally a terrible idea to let an agent improvise authentication and authorization as they would often do. This brings us to our season sponsor, WorkOS. How do you authorize AI agents? The problem you have is how you want to control the scope of the agent. The tricky part is how permissions are static, but the job of what the agent does is dynamic. So teams pick between two bad options. Either you read the prompt, then approve every tool invocation call by hand, and then read the prompt again, then approve by hand again, until you eventually just stop reading the prompt.

37:44Or you just run in YOLO mode, letting it rip and hoping that whatever the agent does is not irreversible. But there's a better way. Worko has just launched Airlock, intent-based access control for agents. You write the rules in plain English, for example. Read repos and comment on PRs. Anything touching auth or billing needs sign-off. Never push to main. Every call that the agent makes is judged against his task and allowed, denied, or sent to a human. Every verdict is logged by Airlock. The neat thing about Airlock is how there are no pre-granted scopes and there's no rules to assign. The task itself is what defines what the agent can do.

38:19WorkOS Airlock is in early access. Request it at workos.com slash airlock. I'd also like to mention our presenting sponsor TurboPuffer. Matt and I are discussing a fundamental question. How do you get agents to remember what's important? Here's an idea. What if instead of building a complex memory system you just let the agent search its entire history? This seems like it will be very very expensive but with TurboPuffer it isn't. TurboPuffer's object storage native architecture means that the marginal cost to source session transcript is almost nothing, making it economical to index the entire chat history.

38:53And because TurboPuffer namespace has skill virtually without limit, you can create a dedicated search index for every agent. Here's a good example of this. Entire, another season sponsor of the podcast, indexes hundreds of millions of agent session transcripts for search and then let the coding agent retrieve what it needs to recall how and why an engineering decision was made. Entire showed that their agent was more accurate, used fewer tokens, and took less time to find memories when it used TurboPuffer instead of Git history and a CLI. Agent memory is a complex and evolving use case but perhaps there's a bitter lesson here.

39:25Maybe the best solution is the simple one. Just search every transcript. With TurboPuffer this is actually possible. If agent memory is something you're trying to solve then please reach out to the TurboPover team at turbopover.com slash pagmatic.

39:38Matt Pocock:And which other skills did you create? So from there, I thought, okay, how do I take that conversation and turn it into code? And I was immediately scared because I'd been working with models, you know, just before they were good and before the December winter where, you know, things got really good. And so I was, I felt the constraints from what I'd been working with before. I knew that, for instance, the more context you put, you give to the agent, the worse it performs. I know you had Dex Hawthy on this podcast. Yeah. And Dex is a really big influence on me, especially his idea of the smart zone and the dumb zone.

40:16Matt Pocock:Smart zone and dumb zone. Yeah. So idea of that just to kind of, so you don't have to go and listen to that podcast in full, although you should. You have essentially the more context you give to the agent, every token is shouting for attention. and the more voices you put into that room, the harder it is to hear the important ones. And so the model starts losing the connections between things and making mistakes because of that. And you can think of that as a slow decline, but there is a portion of the context window where it's better and where it's worse. And so you have the smart zone, which is currently, I would say, about the first 150 ,000 tokens of Frontier models.

40:56Of a 1 million token window. Yeah, of any size token window. Of any size token window.

41:00Matt Pocock:It doesn't matter the context window size. It's all about raw amounts of tokens, raw amount of attention relationships. And then the rest of it will slowly degrade more and more and more. And so I started thinking, how do I take work that's bigger than 150K tokens, which is not very large, and portion it out over multiple context windows, multiple sessions? And this took me a lot of tries, a lot of different fiddling around with different approaches. The Ralph loops was one version of that. Ralph loops are designed to make the most of the smart zone because they essentially just give the Ralph loop a goal and they say, do the smallest possible change that will get us further towards that goal and then clear your context.

41:39Matt Pocock:And then clear the context, start from fresh. Exactly. And you're not technically starting from fresh because you've got the code base, right? There's a little bit of state saved in the file system and in the environment, but not in the model, essentially. So that's the idea. So I started thinking, how do I take that Ralph loop idea, but make it a little bit more stable and turn that into skills. And so what I realized I needed was two different types of documents. You need a document for where you're going, which is the destination document. I used to call that a product requirements document or a spec is what I call it now.

42:10Matt Pocock:So that's the specification that declares when you've reached the end. And then you need to break that spec down into individual tickets, one ticket per session. And so I have a very simple skill just to spec and then to tickets. And so you take that grilling session that you've had and you turn it into a spec. Now that spec can work over, you know, 30, 40 tickets, let's say. You can have really massive, great big chunks of work that are all tied into that spec. And so that's the main idea. You just grill, you turn that grilling into a spec, and then you just run some kind of implementer loop over those tickets until you've got a huge chunk of work.

42:46After grilling, do you get user input as well or throughout this process or it depends?

42:51Matt Pocock:I was mostly designing this to be run for the user, like to be away from keyboard totally, because there's this idea of like the day shift and the night shift. Have you heard of this? No, no, no. It's great. Basically, the optimal way to work with agents is to plan during the day shift and then get the agents to work during the night shift. Right. And so hopefully you wake up in the morning and you've got beautiful, clean code to look at. And that's what I was trying to optimize my process around because I was really sick of what I and what I still do to an extent of just switching between terminals, context switching all the time, just going boom, boom, boom, boom, boom, boom.

43:28Matt Pocock:What I wanted and what I'm trying to optimize for is to just get a good chunk of planning done and then let the agent work for a couple of hours. And then I can do other work, decent chunks of time, 15 minute chunks, working on one thing, planning on stuff. And then I can review the code and do that. So that's what I was trying to optimize for all the time when I was doing Ralph Loops. And that was the big thing that I found in December is these guys are good enough to delegate to. And so I can run them AFK. And then you have a different skill as well, which is a bit more ambitious called the Wayfinder skill.

44:00Can we talk about that? Absolutely.

44:02Matt Pocock:So in exactly the same way that implementation, I noticed, needed to be split out over multiple sessions. Sometimes you're grilling something and you're hitting the limits. You're going to grill something, you know, build me a Stripe clone or something, right? You are going to hit the limits there. There's no way you can plan that in 150K tokens. And so I thought, how do I break that up so that I can run grilling sessions that can be infinite length, right? How do I split up grilling so that it can work like that? And so I came up with this idea. Again, I'm thinking about the flow of information.

44:38Matt Pocock:Essentially, like, what does it need to perform well in a grilling session? it probably needs to understand exactly what it's what the purpose of that grilling session is but it also needs to understand what's been decided so far it needs to understand what other grilling sessions might be happening at that moment and i came up with this idea of a map and the map would be the sort of center point of everything that was needed for all the decisions that you were coming up with and once you've got a map you realize okay there are certain things i can like as I'm trying to find my way to a destination, there are certain things I know I need to decide, certain points that are kind of like milestones on the map.

45:16Matt Pocock:And there's a fog of war. And that lovely metaphor just sort of carried me through designing the rest of the skill, right? Because you've got your map, you've got your fog of war, you vaguely know where you're going. And every time you have a grilling session, it opens out more points on the map until you sort of figure out where you're going. And so this is kind of like a directed acyclic graph where where you're walking down until you reach your final destination. And so you've got the map and then each individual session in there are tickets on that map. And I realized, OK, grilling is good, but what if you need to prototype?

45:48Matt Pocock:What if you need to do research? What if you need to do like an arbitrary task, like provisions and infrastructure or something? Well, those are different types of tickets on the map. And Wayfinder basically just guides you through this process. I've had maps that have, you know, 50, 100 tickets or something until I finally reach my destination. I've actually been using it for course planning as well. So non-technical stuff, which is really great. I've been using it to build a garden office in my garden, right? You know, a lot of these skills, we say, okay, these are great for engineering. Then you realize, okay, engineering is just a discipline that what are we doing here?

46:24Matt Pocock:We're just discussing something. We're doing things in real life, like clicking around websites and stuff. You realize how easily that can map onto other domains. So that's kind of maybe we can touch on that a bit later, which is I am thinking how transposable this stuff is into different disciplines and into different areas of life. So Wayfinder has been great. Yeah, but if we think one interesting thing about engineering and software engineering, when Hillel Wayne was on the podcast, he interviewed engineers like who he thought are real engineers, chemical engineers, mechanical engineers, civil engineers, and to try to find out is software engineering real engineering.

47:00And in the end, he found that it probably is. But he said that one interesting thing with software that is very different to every other engineering profession is the materials that we work with. In every single place, mechanical engineering, civil engineering, even chemical engineering, you have a material that has a threshold of things. You don't know exactly what it's like. You know that it'll be like it can take about this much load, etc. But in software, the material is software, which is it just works like a program. I mean, take out the non-deterministic, which maybe brings us to more engineering.

47:33But software, a code, you run it a thousand times and it does the same thing a thousand times, whereas in other fields it doesn't. And he said that that he sees a big difference. But now I guess with LLMs, maybe we have this thing where we have a thing where you run it a thousand times and it will have these variants that most of engineering has. So who knows if both what works with LLMs will be useful at other engineering where, again, they already had this variance or we can take some approaches from other engineering professions that will maybe work nicely with working with this material called AI.

48:07Matt Pocock:I totally agree. What I think is interesting about software engineering and the way the reason agents are good with it is it's all the inputs and all of the outputs are text based. Everything. So the inputs, code, documentation, instructions for the agent on what to do, all text-based. And the output is more code, is test suites, is type checking results, linting. All that stuff is text-based. The thing that agents really struggle with is anything that's non-text-based. You know, you see these amazing demos of people one-shotting a perfect UI first time. Well, what about if you have an interaction problem in that UI?

48:46Matt Pocock:What if you're like hovering over something and the animation doesn't look right? How are you going to get that to the agent? I mean, you can record a video, I suppose, and it sort of pauses on certain frames, let's say, but it's actually not that good in terms of vision just yet. And so anything that's non-text based is just garbage from the agent. It just can't handle it. And so I think in those sorts of professions, if you can turn, I assume they're doing simulations, right? I assume they're doing some kind of, you know, I don't know if you have like a linter that It can work on architectural diagram.

49:19Matt Pocock:I'm sure you have some variety of that, right? Some simulation. If you can make that text-based, if you can take the interactions that you have in your day-to-day life and turn them into text, which mostly they are anyway, then the agents are going to do a pretty good job. That's something I'm trying to do currently is take all of the services that I use and plug them into agents, right? Make them available to the agents. But yeah, the more we can make our work agent-friendly, the better results we're going to get. And circling back to AI as a whole and then what has changed, like it has changed so many things.

49:53But one thing that comes up with AI is often, especially researchers and people working in AI companies, is no priors. With AI, you should let go of everything that we know before because this thing is different. Start from scratch. The approaches might not work. In fact, let's assume they don't work and come up with new approaches. having been a developer before ai and actually like you're like a like you you were really interested in building quality great software how much do you think ai has changed of of everything

50:21Matt Pocock:including the fundamentals this is something that i thought too i thought right ai has changed everything i'm gonna throw the baby out with the bath water right i think we just need to look at everything in a new way i started doing that a lot i was especially looking at like spec driven development, you know, which is, I have sort of mixed feelings towards, I think it's a strange term, it encompasses too much. And I thought, okay, right, maybe English is the hot new programming language, right? Which went viral at some women under Caprari Posted. Exactly. Like, maybe I can just write a spec and that specification is going to be persistent, it's going to be something I can edit and just get the agent to change it as it goes.

51:03Matt Pocock:And as I experimented with it, I tried it a lot and I was just getting worse results than if I'd coded it by hand. And it wasn't getting better as well. And I noticed that every time I would sort of run this loop of change the spec, see the code change, the code would get worse. He's not supposed to look at the code, of course, but I looked at the code and it was garbage. And I thought, how is the agent going to perform well in here? How is it going to work? Because the feedback loops are so important to the agent. If you have a bad test suite, the agent is going to get bad signal from it, just like a human would.

51:34Matt Pocock:And I thought, how do I improve the test suite? How do I get this setup not like churning out garbage every time? And I just, I opened a book that I had on my shelf that I think was still wrapped in plastic the first time I took it out, which was The Pragmatic Programmer, which is everyone told me to read it. Everyone said, yeah, this is the best book ever. You just got it. And I bought it and I didn't read it for some reason. and I opened it and it had a whole chapter, a whole section on software entropy. And software entropy is the concept that, you know, entropy is the idea that things go towards a more disordered state, that that is more likely than them going into an ordered state.

52:12Matt Pocock:And I realized, okay, software entropy is inevitable. What I'm seeing here is that agents are producing software entropy at a higher rate than ever. And I started looking more into that book and almost every line I read, I thought, wow, this feels like it was written for today. You know, you should go back to that book, these ideas of like, don't outrun your headlights, always work within your feedback loops, programming by coincidence, tracer bullets, so many smart ideas. And I realized this book has been out for 25 years, right? This is probably in the agents' priors. Maybe if I just mentioned some of these concepts, especially the ones that are really pithy, like tracer bullets, for instance, which is the idea that you should always get feedback really quickly on the work that you're doing.

52:53I guess the idea of the tracer bullet is, right, do you like a tracer that leaves a mark, you implement a path that works like an important piece of a software instead of like building a database layer and the application layer and the, I don't know, whatever layer, like building all three and then putting them together, like just build one part of each, but they should work together.

53:13Matt Pocock:That was the problem I was seeing with agents. They would, you would get it to build a piece of software, even with Ralph loops, and you would, it would build the entire database and then it would build the entire application layer on top of that. Then it would build the entire React component library only at the end would it start actually plugging things together and getting feedback on what it was doing and it was maddening because things in the database like will affect what you show on the front end you know you only really know whether things are actually making sense when you see it crossing those integration layers and so another concept is vertical slices right instead of these horizontal slices across these different deployable units you have a vertical slice where it gets feedback on what it's doing straight away and builds out from there and so i just started using these phrases in my prompts when I was talking to the agent.

53:58Matt Pocock:And I started noticing that it was saying those phrases back to me. It was repeating them back to me. It was saying, okay, I'll turn this into a traceable. Because this is a traceable, I'll do this. It was using the words that I was using in its own reasoning traces. And so this is what I call a leading word, a light word, let's say, which is a sort of fancy literary term, where you lead the agent just with a simple phrase that you repeat a couple of times in the skill or the prompt to change its behavior. And so Tracer Bullets was a fantastic one. And I just started diving into different books, all the books I could find to try to mine them for leading words.

54:35Matt Pocock:And another one was John Osterhout's book, Philosophy of Software Design, where I picked up tons of great stuff like deep modules, which is a massive one for me. It's interesting to consider if these agents have obviously been trained on those books, just still available for print. And of course, there's arguments of like what they're doing with those books and whatnot. But if it's in their training data and the agents, as they're trained, they connect all these different concepts. And yeah, these, I guess, leading words could invoke those concepts. And I wonder if this is much different to when on a topic you talk with a professional and you're trying to describe as an amateur what you want in a professional that says a word that does that and a fellow professional gets it.

55:18And this is jargon, right? And jargon, on one hand, it's not very inviting when you join a company and there's jargon, but we use it because it makes things faster, easier, fewer misunderstandings.

55:30Matt Pocock:Definitely. That was something, that idea led me to, because obviously you've got these leading words that are in the agent's priors, right? Like tracer bullets, all that stuff. What about describing my application? What about describing my code? How do I get the agent to, because the agents are just awfully verbose, right? And especially Opus 5, for some reason, people really go after that model for being verbose. And it really is. And I thought, how do I get it to be less verbose? How do we start talking a common language between me and the agents, this communication barrier again? And it led me to DDD, domain driven.

56:05Matt Pocock:Domain driven design. Eric Evans' incredible book where he talks about ubiquitous language, a language, again, really deep in the agent's priors. It understands it really well. I sort of started toying with the idea of maybe changing grill me a little bit because grill me is very simple skill but what if we while we were ideating while we were thinking about the application we were going to build what if we were also building a domain language what if we were also deciding on the right terms to use and this turned into a skill called grill with docs which is terribly named skill but it essentially creates this domain language as you go and if you get the agent to use the domain language, the difference is night and day.

56:47Matt Pocock:Because suddenly you're speaking the same language, you're able to describe the things you want to change in so many fewer words. Like I had this app that I sort of work on. There's this complicated interaction where there are ghost lessons and real lessons. And what happens when you turn a ghost lesson that's inside a ghost section, inside a ghost course, into a real lesson? That means the ghost section needs to become real and the ghost course needs to become real. How do you explain that? Well, well, that's the materialization cascade, right? And you came up with these terms. Yeah, with the agent, right?

57:18Matt Pocock:The agent is actually really good at coming up with these terms. And so I have a domain modeling skill. And we talk about jargon, but really it's domain language. And if you can integrate that and integrate that not only with the way you talk about the app, but the app code itself, then you've got a stew going, right? Like it's very, very exciting. And it means that the agent can navigate your code base a lot easier. It can find the functions that mention that specific domain terminology. you know, just with a simple grep. It's just gorgeous. So that's something I've been really integrating with every part of my setup is DDD.

57:50But this is so interesting because in an effort to make these agents work more efficiently or do workflows that just mean that you can produce better software with fewer mistakes and these things, you start to go back in time, found this book that is now, I think, what is it, like 20, 30, 40 years old? And you're even still going back and finding gems from us. I'm sure at some point you'll get to the Mythical Man Month.

58:15Matt Pocock:Yeah, I've got it already. Absolutely. Which is now more than 50 years. And you're trying to find the right words to describe things, which is very curious because when I talk with Kent Beck on how they used to program with Ward Cunningham as they were coming up with the concept of actually just domain design patterns, they had a thesaurus with them. And they would look through trying to find the right word that has the right meaning. And they had it on their desk. Right now, this feels we're going back to the fundamentals, how that people have been asking themselves. And every now and then people write it in books and it kind of spreads as wisdom.

58:51And now we're back to where we started, which is what you're trying to teach is the wisdom part.

58:56Matt Pocock:It's wild, right? Like because AI is so different to humans, you need to optimize it. Imagine you essentially had a human who wakes up every morning and cannot remember who they are, right? The guy from Memento, you know, this is Memento driven development, right? We are trying to optimize our code bases for new starters. So we're trying to have the most healthy code base that we've ever had. Because if you, a human can work around a bad code base, they just develop memory. They just slam their head against the wall again and again and again until they've got there. But an agent can't do that.

59:33Matt Pocock:It starts fresh every single session. And so you need to optimize your code base for that person. That leads you down into really interesting paths. And it turns out that software fundamentals have been saying we've been trying to do that for the entire time, right? I am fully like, I don't know, Eric Evans pilled. I'm fully like software fundamentals pilled. We are sort of changing the rules a little bit, but maybe we're just emphasizing rules that we knew we were supposed to do, but maybe we didn't. And I find that really fascinating and it's definitely a lot of fun. Now, OK, like I think it's easy enough to follow with this train of thought why fundamentals matter.

1:00:10But which fundamentals? And if I'm an engineer, especially maybe someone who has been just kind of like heads down coding, more tactical coding. How do I go about and find those fundamentals that matter and go back to what do you find work?

1:00:23Matt Pocock:This is a really tough question, right? It's really tough because strategic programming has always been really hard to learn. The reason for that is that the feedback loop on it is really long. You would often find like people who quit their jobs after six months, their strategic mistakes never catch up with them. Maybe that strategic mistake takes nine months to come back at you. I think of learning strategic programming is kind of like you've got a huge mixing desk in front of you with loads of these different sliders. Maybe one of those sliders is like the amount of deployable units that you have.

1:00:56Matt Pocock:You turn it up, you've got more microservices, right? You turn it down, you've got a monolith. How do you make that decision? Where do you put that slider? Because it's kind of like you're mastering something, you're mixing some music, but you can't hear what's wrong until nine months later, right? Until the mistakes come and get you. So I think the only thing that can make that feedback loop faster is moving faster. AI now lets you move faster, right? And so your strategic mistakes will come back at you quicker. They will come back at you quicker because AI is just able to produce so much code.

1:01:26Matt Pocock:And so what you need to be thinking about is that your code is the environment the agent operates in. And you should always be thinking about improving that environment, thinking about how to do it better. And obviously that requires a bit of tactical knowledge, right? You need to understand what code is and how it fits together and what the memory constraints are and all that stuff. But in order to get better at strategic programming, you just need to be thinking on that level all the time. And I would say reading these books as well, because just having the language to explain that and understanding the difference between applying strategic techniques and not is the whole game.

1:02:01I mean, up to, you know, pre-AI for senior developers, for senior engineers, staff engineers, they were the people who often you didn't see a senior engineer under five years of experience because you typically needed, even in a fast-paced environment, you needed that much time to get the feedback loops, to make the mistakes, make your own mistakes. And by the time people got to staff engineer, oftentimes around 10 plus years of experience, And some people did it earlier, but they often just had paddle skulls all over them. And they would, you know, someone started a new project and they would go in and they would just make a tweak.

1:02:34And it wouldn't be clear why. And they were like, trust me on this. We're avoiding disaster in production or on-call or whatnot. But all of this came through lived experience. Now, AI speeds things up. It also makes it easier to fix mistakes. So I'm wondering how this might change. Like on one end, I can see how it could just speed up experience. Like you can in a year, some people, some teams will ship more projects than they have in four years or about the same as in, let's say, three or four years before. So you get a lot more experience. But I wonder if sometimes the mistakes that you make are just not as serious because you can fix them quickly.

1:03:11And now I wonder if the learning is not as strong. Because, again, like some of these battle scars, these war stories are it was just really bad outage. We lost a lot of money because we didn't have an item potent Kia. Now, of course, now you know what Idempotency is. It's not an easy concept, but it's important if you've been heard by it and so on.

1:03:27Matt Pocock:If you're a company right now and you want to train the next junior developer. Like, because this strategic programming knowledge is so valuable now, because you can use it as such higher leverage, are you really going to employ someone without it? Like, why would you? Like, I was asking, I did an interview with Uncle Bob the other day, and his recommendation was, okay, you just hire someone and you treat them as an agent for a while. You just delegate to them, you keep them in that tactical mindset for a while until their mistakes start coming up at you. But that's such an enormous waste of money for people, right?

1:04:04Matt Pocock:Like when software engineering, when the tactical stuff has gone below minimum wage in a lot of countries. So I don't know, is the answer. I only know that the strategic stuff, the understanding of the code, the understanding of the long view has gotten more valuable than it's ever been, right? Because you can just get so much leverage out of it. I asked about interesting things I'd like to know from you. and this is very related to this, this person asked, like, how do you convince non-engineering stakeholders that investing in software fundamentals are important, even if they might reduce the speed and productivity on paper?

1:04:39I think the question here is if some people advocate, like, look, we do want to get the fundamentals right, which means we want to take it a bit slower, think about their decisions, maybe educate ourselves as well, as opposed to just like churning it out. I mean, you could have asked

1:04:54Matt Pocock:the same question 10 years ago, right? And like, it would have still been relevant. You know what I mean? Except with sort of top of elements, we will have asked about paying off tech debt. Exactly. And it's the same thing, right? Like we have been having the same conversation, which is quite satisfying to me because, I mean, you need some sort of metric for like figuring this out. And it's a little easier to figure this out because agents allow you to move faster. And the first step to this is getting observability in your organization over every single agent on what it's doing and what its success and failure rate is.

1:05:25Matt Pocock:we've never been able to have that with like developers before you know it's kind of invasive for developers but for agents it's like it should be it's okay it's okay right we are paying for this service right we need to understand how well we're optimizing for it the first step there is actually getting a harness or observability around your agents the entire organization to work out what's working and not and you probably need someone whose whose job it is or part of their job is to look at that data and figure out what we're doing maybe some repos in your organization have better success rates than others.

1:05:56Matt Pocock:And so you take the lessons that are in there and you pass them out. I also think that most organizations need to gather around a common set of skills. You need a common software workflow process so that everyone can contribute back to it, so that you can experiment with things, you can A-B test things. You know, you can have one team doing one set of stuff and one team doing another set of stuff, and then you ask them afterwards. And so everyone working with agents in any kind of organization needs this experimental mindset. You need to be thinking, how do we get more juice out of these tokens that we're spending?

1:06:29Matt Pocock:And observability is the first step there. Yeah. And I also wonder if there's a human feedback loop in the sense that, I mean, just talk to your colleagues. Like, you know, we do have rituals, team meetings, company-wide meetings for a reason. like their share here's what's working for me here's where it didn't work here's what i'm learning like in in the end we are in charge of setting up the rules deciding how we use them where we use them where we don't use them and where we say like no this this needs to be humans need to take 100 like we're not even getting ai involved which again will be different everywhere and it's not only that like a lot of this stuff now you don't need to be human in the loop for right you don't actually need to delegate that much time in order to build up a better code base I have loops that essentially every morning it will run my improved codebase architecture skill and give me a proposal for the something that I could improve in the codebase.

1:07:23Matt Pocock:And then I can just press a button. I can say, OK, turn that into tickets and then let's ship that. That is pretty easy to do and it's pretty easy to stream that in with other work. And so I think that I don't know whether you need like 20 percent of your time focusing on the factory that builds your software as well as the software, because I feel like that's a massive, incredible investments into your future leverage and not only your leverage with your work, but also your team's leverage and understanding and getting better at those skills. But of course, you need results and you might need to hide that work for a bit before you actually reveal it to this is what we've been doing.

1:07:58Well, and this is down to your environment, but yeah, no one's going to be mad at you if you come back saying, oh, by the way, guys, I also did this. Yeah, exactly. I want to ask you about your specific kind of how you use tools. First one is coding agents, local or in the cloud? And you recently posted a pretty provocative tweet, which I'll quote you. I'm moving away from my local dev setup. Makes zero sense to me now.

1:08:22Matt Pocock:A lot of people ask me, how do you make your skills collaborative? How do you have a collaborative grilling session? And the answer to that is that you need more than just your terminal and you, right? We're in a phase now where every dev has like 100 terminals available to them. And that seems crazy. It feels like you need those hundred terminals available to your entire organization. You need to be able to collaborate in a shared space. You need to be able to ask someone, tag someone in to your grilling session and say, OK, do this. And so it makes a lot of sense for me to have a lot of those interactions in the place where you already work in Slack or in Discord or in Teams, whatever, or linear.

1:09:02Matt Pocock:And that is really the thing that's driving me to explore this. I don't work with a team particularly, but I understand the value of that. and I've been trying to build that into my flows. So on the train over here, I'm in Discord chatting to my Hetzner box, you know, building stuff for my course or fixing bugs that students are coming across. So I can see less value now in just doing things locally when I have this setup that I can port forward into, let's say, and, you know, and like see the dev server as it's making changes. And I don't know, it just feels like it makes way more sense to me than having a very, very expensive laptop that can do this stuff.

1:09:39Matt Pocock:It feels like wasted compute. And especially because on that remote box, I can set up schedules. I know the box is always going to be on. I have like a morning stand up with my agent where I get it. It schedules my day for me and it understands all of my Discord chats and all that. Yeah, having that remote feels like it makes just so much more sense for me. And the only thing I do locally now is debugging issues with the remote bot. Yeah, I think I wonder if there's a question of how easiest to replicate some pretty complicated local setups in the cloud. But once that becomes possible, it's probably a matter of when, not an if.

1:10:13Matt Pocock:Yeah. And if anything, people are having this similar issue with local setups, right? With just a thousand Git work trees just spamming their hard drive. And with how do I have a work tree that I've got to run like five Docker containers in order to get my local dev setup? Well, that's often a little bit easier in the cloud because you can just provision the resources that you need on demand. And by the way, we're seeing that companies like Ramp, Stripe, Uber, that have platform teams that manage to take a local devs full setup and put it into the cloud on a cloud machine that you can now invoke with an ad Slack or a website.

1:10:47site, they're seeing people use these agents far more except for front end work, which you still want to have that feedback loop. There are a few exceptions where you really want to have that local dev setup for latency or whatnot, but they're also seeing like 70, 80 % of devs are just voluntarily going for the cloud.

1:11:08Matt Pocock:Yeah. I mean, I think you can just tunnel through and just get the, if it's running a dev server and you just have that appearing on your local machine, how is that different from having it locally, right? I don't know. I think I've not experimented with that, but that's when I talked about that and said, oh, maybe frontend is a good exception. That was the immediate response that I got. And it makes sense to me. I want to ask you about planning and requirements. You're a big believer in grill me and planning up front or getting the plan and then having the agent work. But there's a devil's advocate here.

1:11:38Agents are so fast at implementing. You can actually even have like several, So the few agents implement different architectures. What about the approach of like, well, they're fast at implementing, so I might not need to do as much upfront planning. I can just course correct as I go.

1:11:52Matt Pocock:It depends what type of work you're doing, right? Because I believe that you shouldn't be using Grill Me for everything. Essentially, you need Grill Me for pieces of work where the actual thing being done is going to be quite large and hard to row back from. If you feel like, OK, this feature, maybe it's a whole new page, maybe it's a big feature, this code, you think if the agent gets it wrong, then the wrong code is going to be in its context window influencing everything that comes afterwards. And actually going back and editing the stuff afterwards and doing the alignment after the fact is going to be expensive.

1:12:27Matt Pocock:Whereas for those cases, it makes sense to align first to answer all of the tricky questions like your JSON cookie or whatever, your authentication token first and then do it. But for some cases, like simple bug fixes or just like move this button three pixels to the left, it's obvious that you don't need to align before that. You can see the thing if it's just like a five line change or something. You can align afterwards. And so that's how I think of it is that where you can, you should shift right as much as possible. And actually, there is actually certain features that I have a little in my video editor.

1:13:03Matt Pocock:I have a button that I can send feedback to it. and I often use this for very simple tasks where I send the feedback, it goes into a GitHub issue. This immediately gets picked up by an implementer agent, gets just worked on immediately. Then a code review agent comes in and reviews the code. And then at the end, I get to see this actual thing being fixed and I can do my alignment then. And that's worked really well for things that are very easy to specify, things that I don't need to grill on. So those are the choices you've got. Is it a small enough thing that I can align afterwards? Then don't use Grill Me.

1:13:35Matt Pocock:Does it fit into a single session? Then use GrillMe. Does it span multiple sessions? I need to align over the entire thing. Then use Wayfinder. Interesting, because this is not all that different to where some tech companies landed years before, which is on the PRD, the product reference document. If it's something trivial, just build it. If it requires the team, like it's a team level scope. I mean, write a PRD, send it out to the team, maybe CC some other teams, but it's not a blocker. And if it's something bigger, then it's a block or like we need to wait for feedback. Basically, the way we would say it is like, look, if it's like a one month project, like spend two days.

1:14:14Like it's not a bad thing to spend like one or two days planning it because we're going to save time on it. But if it's a if it's a one day project, like forget about it. If it's a one year project, I mean, what are we doing? Like it should be smaller one.

1:14:25Matt Pocock:Totally. And I want to like there's a bit of sort of criticism I hear just from outside the room when you say that, which is that doesn't this sound like waterfall what we're doing? When I'm talking about Wayfinder and when I'm doing any kind of like building up any kind of spec, I do a lot of upfront aggressive prototyping before we get there. That's something that comes up again and again and again. It's like this is just waterfall. What are we doing going back to the 70s? But agents give you this ability of just churning out slop, right? And sometimes you can use that to your advantage because a prototype, right, just getting a sense for what it should look like.

1:14:59Matt Pocock:you can build out three or four different versions and just choose your favorite and iterate on it and just keep churning, churning, churning. That can be a really powerful setup that we've not really had before, right? It was always expensive to produce prototypes. Now it's the cheapest that it's ever been. And that's an essential part of writing specs to me, is actually producing these prototypes. Yeah, but also like with the Waterfall criticism, I think Grady Booch might've told me this as well. It's like, don't forget, like we should not criticize Waterfall because for example, a lot of big tech, the largest companies from like Amazon, Microsoft, Google, Meta, you name it, they are kind of doing mini waterfall, like pre-AI, they've been doing pretty mini waterfall, which is let's do a plan.

1:15:40Let's agree on it. Let's build it. Let's ship it. And this is all done in like two weeks, a month, two months, three months. Three months is kind of the extreme. But CradyBooch was saying the problem was never this with waterfall. The problem with waterfall was the planning was literally taking like a year, like one year, and then the implementation taking three years. and by the time it was ready four years later it's not what we wanted and that was the problem he was like the problem is not like having like a one or two month project or a one week project with a waterfall the problem was always this we're talking years and he said that the industry has not seen waterfalls for for decades now and so here we're using this term which which is a bit like we're criticizing or many waterfalls were criticizing that one it's actually that's i thought

1:16:22Matt Pocock:not a bad thing necessarily you see what i mean scarecrow that we're punching or something Yeah, it's a piñata which stopped existing. It might exist in some crazy enterprise project that none of us know about in regulated industries, but I feel even there it's probably caught out of style. I think it's like if we're hitting the piñata, I think it's actually a useful thing to have up there. It's like a useful ghost or useful cautionary tale, right? Because which one fits the agentic setup more closely? It's going to be agile, right? Because the cost of labor has gone down so much. We can just make changes very, very quickly.

1:17:00Matt Pocock:I don't know. That feels like the right metaphor to me. So I don't mind hitting on waterfall, even though no one really does it anymore. Well, one other thing that just went out of style, we didn't hate it, but test-driven development, TDD. What is your take on using them for AgendaX? When I talked with Ken Beck, we talked about how this could be a great fit for many reasons, but I still don't see people really using it. I see people writing tests. The agents also like write tests after the fact, which is how most people work. But I think you've been an advocate for TDD, right? Yeah. So I have a TDD skill, which I recommend using.

1:17:33Matt Pocock:and this is quite timely because I have been thinking about it, but I haven't really posted about it yet. TDD optimizes for having a very small working memory, right? You write one test and that test is supposed to fail. And it means that even if you get distracted, you go for a coffee or something, you go for a long walk. When you come back, the test is still failing, reminding you of where you are in the implementation and guiding you to the next thing. Agents don't need that. Agents, the thing that's great about agents is that they have a much larger working memory than humans. right? They can actually hold a lot more in their heads than humans can currently, which is very useful.

1:18:10Matt Pocock:But they don't have an infinite working memory. And TDD, it's sort of aiming at the wrong problem, I think. But the thing that agents really do need is that they need to have feedback loops. So they need to see what they're doing and how it's interacting with the environment of the code. They need to probe it all of the time. And having an agent that builds it, builds the failure first, it's also very hard for an agent to cheat that. So not only are you forcing the agent to build its own feedback loops, the agent is providing proof to you that the thing is actually working as it goes. And even if I'm not using TDD directly, where it writes the failure test first, then fixes it and refactors, I will often say provide proof that your change does the thing it's purported to do.

1:19:01Matt Pocock:Give me TDD evidence, right? That it would fail without this change. And that's been really good for just improving the feedback loops, essentially, because another thing with TDD that agents get wrong is they will often just write crap tests. They'll often just write, especially tautological tests, where the test is just asserting the implementation itself. It's just like a duplicate of it. You know, it writes a constant, and then it says expect this constant to be this value. I mean, what's the point in that test? You know, it's just asserting implementation. So yeah, I have a mixed relationship with TDD.

1:19:38Matt Pocock:I do still recommend it just because it gives you so much more confidence in what you're building from a human perspective. But yeah, I'm starting to see the counter arguments. Let's talk about tech depth. Jared Friedman at Y Combinator wrote a tweet that I'll quote from him. technical depth used to be something you just had to live with with a sufficiently large code base no longer and to which you replied yes now you can live with it even in a tiny code base that's good it read out loud actually you really gave the uh the sense to that one um yeah it's just so easy for agents to produce rubbish right on even really smart powerful agents because they're unable to think strategically, they're just focused on what they're doing right now, it's very easy for them to produce tech debt.

1:20:29Matt Pocock:What is tech debt, right? Tech debt is anything that makes the code base harder to make modifications to over time. A good code base is one that's easy to change, easy to make a change in that doesn't result in cascading failures, right? So a code base with a solid test coverage and a good test suite is a code base that's easy to change. But it's so easy for agents to just make a code base worse over time. And it's a really hard problem. And it's one that you need a strategic mindset to think about, because one thing that I found works really well is automated review. So you have one implementer agent to do the thing, and then you have another automated review agent that sort of imposes your coding standards, that looks for these tautological tests, improves the quality of the test suite over time.

1:21:17Matt Pocock:But then how do you know if the automated review agent is doing a good job, you know? And so even in tiny code bases, even in one line changes, the agent can produce crap, you know? And so I think it's just something we need to live with and something we need to be in a constant battle against. No, it's also not a bad thing. We bring a bunch of value when you understand what good code looks like, when you can recognize what tech depth is. And it's also a problem that we've always had. You know what I mean? It hasn't gone away. Hasn't gone away. You know, this is just, this is what I feel like.

1:21:51Matt Pocock:We're just having the same conversations we've had for 20 years. It's just there's this new elephant in the room. I want to ask you about living in the UK and AI. This is a question that also came from one of the readers. Now that you're based in the UK and outside of London, but you're now educating about AI. Is being further away from Silicon Valley and the HQ of the labs making things easier or harder for you? I'm really just trying to plow my own furrow, really. Like what I realized quite early on is that I have no power to predict the future, right? Because I'm so far away from things, I'm just a person in the field working with this stuff.

1:22:29Matt Pocock:I have no way of knowing what's coming, right? I don't know whether the model's going to improve. I don't have privileged access to stuff. And so I'm just trying to focus on what's working right now. And because of that, I think that's narrowed my scope a little bit. That means I can just try to get my stuff working. And it's sort of quite surprising to me that it's working as well as it is, you know, because I don't have this privileged access. I'm just trying to make this one approach work. So I think, yeah, you're probably right. I probably would be able to do this stuff if I lived in San Francisco, but then I'd have to live in San Francisco.

1:23:05Matt Pocock:I don't want to do that. that's miserable you know i've got a great setup here my parents are just down the road you know i've got my son growing up in the countryside so it is what it is and um yeah now you're an educator at at heart how have you seen the business of of teaching or educating software engineers change and also how people want to learn if if you've observed any trends from before like already when you started you i feel you were on at the time where online courses and learning over video became a lot more popular as opposed to let's say a decade ago where it was maybe tutorials and then before that it was books obviously they still exist but they're just different preferences yeah it was around covid time that sort of video tutorials really took off i think you wanted a much richer learning experience and i was kind of just uh after that wave i suppose i think that people's people's way they've learned doesn't hasn't changed that much.

1:24:02Matt Pocock:Right. And their desire for certain types of materials hasn't changed. I think it's very sexy, the idea that, you know, an agent can just come in and teach you everything. And that sort of works in some contexts. But really, what you want is curation. Right. You want a human to have come in, understand the flow of the information. I always think of information as kind of like a graph, right? You have a piece of information that's dependent on another piece of information, dependent on another piece of information. And turning that graph into a linear path is how I think of my job, right? I'm just trying to teach you, like, find Dijkstra's algorithm through the graph so that you can learn it in the most sensible way.

1:24:47Matt Pocock:And that level of curation is just not something that, again, that's strategic, right? That's not something that AI is particularly good at. So, I mean, I've obviously made this huge pivot from TypeScript, from tactical stuff, really, to this strategic layer. And it's working okay for me. I really can't speak for other folks doing this work. And I know that lots of people are not having this level of success, I suppose. So I think what it shows is that agents have just changed the game in terms of what people value and what people prioritize and the industry has shifted in seven months faster than it's i think ever done you know this is a huge shift doesn't mean we need to throw away our working practices but it does mean that what we need to focus on is different and i feel like i've been able to move with that quite well whereas i think others just haven't because they're focused on different things and i i wonder if in in your case it's also with total typescript and even before with with typescript some of the things you shared you were helping people use the very popular tool at the time.

1:25:52TypeScript was gaining market share. There were migrations happening from JavaScript to TypeScript, from Python to TypeScript, and so on. And so developers wanted to get really good, a lot of them, or the top 10 % or top 20%, you name it, wanted to get really, really good with TypeScript, and they were looking for efficient ways of doing it. Now, AI is here is changing how we work as software engineers, And I think it's pretty clear that building software is valuable. But there's a question of how do I use these tools more efficiently, which is more pressing right now than how do I write TypeScript efficiently, especially with the agent.

1:26:25So I wonder if you've kind of just a little bit how you pivoted from voice acting to what you couldn't do from outside of London to a thing that you could do outside of London, which was still teaching. You've just pivoted to teaching a different area, which right now is, again, it's on so many people's minds.

1:26:43Matt Pocock:I think I've just been lucky, basically, of choosing the right thing at the right time. It would have been very easy for me to, and I actually took quite a fair bit of convincing to move into AI. Like back a couple of years ago, it was Joel, my business partner, who was pushing me to actually go, you've really got to try this. It's actually pretty good and you can use it for all sorts of stuff. And it took about three months of me actually trying it and failing and trying it and failing before I realised, OK, this is great. I just feel quite fortunate that I've landed in the right place at the right time.

1:27:13Matt Pocock:and I try not to narrativize it. I try not to think, oh, well done, Matt, you've been so smart, you know, making the right play at the right time because I've made several mistakes as well and I could have easily found myself in a different zone and I mean, that's no bad thing. I would just go back to being an engineer. That's what I love too. Putting yourself back into the shoes of when you were someone just starting out in the industry today for people starting out in the industry, early career, junior folks, what would you recommend them for tactical things to do? Like they will know, Like, look, I want to get that experience.

1:27:44I want to get that judgment, that taste, those fundamentals. You'll need to get repetitions. And if you found yourself in those shoes, how would you approach? Like, I want to be a builder, a software engineer with all these AI tools, whatnot, which is now confusing because now there's a mix of do I use these AI tools just to do stuff for me? Do I get in the fundamentals, which is slow and so on?

1:28:05Matt Pocock:Yeah, I mean, I would love to be a junior right now. I would love to be in the exact position I was in like 2014, where I was building these tools for my students. Right. I actually got really nostalgic for it on the other day. I thought I'd love to get back and do some singing teaching because just the ability to like I could finish a lesson and then just prompt the agent. OK, this tool didn't quite work in that way. I could maybe modify it a little bit and, you know, see it working. I just think the right thing to do is to use these agents as much as possible, because that's how people are going to be working now.

1:28:36Matt Pocock:And I think the thing that I find valuable about my skill set is you're constantly in touch with the changes that are happening. Grill Me not only you're having a discussion with a senior developer, right? That's beneficial for the developer, but it's also beneficial for you. Keeps you thinking about these deeper ideas. And the absolute rubbish that I was churning out, you know, with my spectrogram analysis tool, that would have been so much better if I had an agent to work with. it ran like a pig you know like it was performance was absolutely terrible if i'd have been able to say okay this frame rate has dropped to 10 frames per second how do i fix that it would have seen the six nested for loops and gone okay maybe you should do something different there so i think that there's never been a more empowering time to work on this stuff as long as you're interested in not only the code you're producing but also the process of creating the code there's never been a better time to be a kind of navel-gazing programmer, just constantly thinking about your own processes and being introspective.

1:29:39So it sounds like if you're motivated, you should be able to learn really fast compared to even before.

1:29:43Matt Pocock:Absolutely. It's just about being curious about being adaptable. And that's the people that I see who are thriving in this new environment are the same people who were thriving 10 years ago, because they're just interested in this work, interested in making better software and interest in their own process. And how interesting in making better software is, I wanted to ask you about gardening. A software engineer on XLauren posted, I'll quote her, every team needs a gardener, someone quietly watching the stream of PRs flowing into your code base, noticing the smells, the lens suppressions creeping like ivy across your careful plant garden, a steady hand intending the weeds that would otherwise engulf the garden.

1:30:20And to which you replied, I'd argue the only thing your team needs are gardeners.

1:30:25Matt Pocock:You probably do need a couple of other people as well. Yeah, but more specifically, I want to ask about this concept of gardening. I actually really love like how Lauren described the weeds taking over the garden and getting them out. I think I made a tweet a while ago that we are, this was when I was sort of thinking about Ralph and sort of the agents sort of looping over stuff. We are essentially just Ralph's platform team, right? That's what we are now. And we are our agent's platform team. We are trying to build the environment for them to succeed. That's exactly how you should be thinking about it.

1:30:56Matt Pocock:Again, it's strategic. and that gardener metaphor is nice because you know um it's very easy for the garden to itself just to suffer entropy right to gather weeds and to do all that stuff so understanding and diagnosing that stuff before it becomes a problem in your own code base is an essential skill and might be the essential skill right as long as you can queue up work for agents as long as you can build these loops now that we're starting to see these processes where agents improve the code base based on bug reports and feedbacks. That feels to me like really cool work and noble, interesting work as well.

1:31:33We talked about some great standout software engineers that you learned from, you got inspiration from. Today, what skill sets, experience, approach do you think makes a great software engineer?

1:31:47Matt Pocock:I'll use an example, which is Lars Grammel who works at Vercel on the AI SDK who I had a chat with the other day. and he is building an entire software factory for his extremely popular open source library that gets a ton of issues. We're talking about plumbing again. We're talking about gardening. We're like thinking about the processes of software development. And I suppose if I had to put it in a word, it would be introspection. It would be looking at yourself and the ability to take what you do and put that into something the AI can work with. you're essentially trying to put your process into words and that's what I've been doing with the skills that's what I've been trying to do with the automations I've been creating as well it's I just look at what I'm doing and think how could I do this better and also how could I encode this into this strange animal that I have in front of me how can I make it work like I want to and that attitude has been really really helpful for me it's something that I value in in Lars and I of value in all the people that I work with when they approach agents.

1:32:54And then as closing, what are, what is a book that you would recommend or multiple books?

1:32:59Matt Pocock:I'll go with Pragmatic Programmer, Philosophy of Software Design by John Osterhout, and I'd say the first like three chapters of DDD, the Eric Evans book, the ubiquitous language one. That one in particular, it's really great for the ubiquitous language concepts, the domain modeling, the actual sort of encoding it into code I'm not such a huge fan of, but those three are the big three. Awesome, Matt. Well, thank you. This was really interesting and really fun. Great to finally be on the podcast. Yeah. Meet the famous guy himself. It's great. We've met before, obviously, but it's great to be here.

1:33:32It was so nice to sit down with Matt. And I have to say, knowing that he was a voice coach and actor makes me understand how he talks so smooth and how he's so pleasant to listen to. Probably the most amusing part from this conversation was how, as Matt was searching for how to work better with AI, it wasn't modern approaches that he found really useful. Instead, he went back to classic software engineering books. The Pragmatic Programmer, A Philosophy of Software Design, and Domain-Driven Design. There's some irony as to how the best practices documented 20 plus years ago, like tactical versus strategic programming in this book, not only do they still work, but they become more important when writing code with AI agents.

1:34:09A related point I want to emphasize is the importance of leading words with AI. When Matt started to use terms like tracer bullet or vertical slices, the model started to follow his ideas better in planning. And if you think about it, this makes sense because software engineering literature is part of LLM training. So these terms are also part of the model's priors. Just as interestingly, using the right words for describing your problem is not a new concept. For example, when I had Ken Beck on the podcast, he talked about how 30 or 35 years back when him and Ward Cunningham had a thesis on their desks, they used it to try to find the best words for the specific thing they were describing.

1:34:47This was just another full circle moment on how words do matter. Finally, I appreciated Matt's push on how you should want a clean code base, not just because it's easier for a human to navigate, although I think you really want to do it for that as well, but also conveniently, agents do not have a long-term memory, and they will look at your code base for the first time on every new run. And it's much easier to get around inside a well structured code base than one that is really messy. Check out the show notes below for an interview with John Osterhout, the author of A Philosophy of Software Design, a book I really love, and related deep dives for AI engineering and context engineering.

1:35:22If you liked this episode, please make sure to be subscribed in your podcast player and submitting a rating is always appreciated. Thanks and see you in the next one.