Try harder. Ultrathink! (Friends)

32:16Well, friends, I'm here with a new friend of mine, Harjot Gill, co-founder and CEO of CodeRabbit, where they're cutting code review time in half with their AI code review platform. So Harjot, in this new world of AI-generated code, we are at the perils of code review, getting good code into our code bases, reviewed, and getting it into production. Help me understand the state of code review in this new AI era. The success of AI in code generation has been just mind-blowing, like how fast some of the companies like Cursor and GitHub Copilot itself have grown. The developers are picking up these tools and running with it pretty much.

32:55I mean, there's a lot more code being written. And in that world, the bottleneck shapes the code review becomes like even more important than it was in the past. Even in the past, like companies cared about code quality, had all this pull request model for code reviews and a lot of checks. But post-gen AI, now we are looking at, first of all, a lot more code being written. And interestingly, a lot of this code being written is not perfect, right? So the bottleneck and the importance of code review is even more so than it was in the past. You have to really understand this code in order to ship it.

33:25You can't just wipe code and ship. You have to first understand what the AI did. That's where CodeRabbit comes in. It's kind of like, think of it as a second order effect where the first order effect has been Gen AI and code generation. Rapid success there now as a second order effect. There's a massive need in the market for tools like CodeRabbit to exist and solve that bottleneck. and a lot of the companies we know have been struggling to run, especially the newer AI agents. If you look at the code generation AI, the first generation of the tools were just tab completion, which you can review in real time.

33:55And if you don't like it, don't accept it. If you like it, just press tab, right? But those systems have now evolved into more agentic workflows, where now you're starting with a prompt and you get changes performed on like multiple files and multiple equations in the code. And that's where the bottleneck has now become code review bottleneck. Every developer is now evolving into a code reviewer. A lot of the code being written by AI. That's where the need for CodeRabbit started. And that's being seen in the market. CodeRabbit has been non-linearly growing, I would say. It's a relatively young company, but it's being trusted by 100 ,000 plus developers around the world.

34:29Okay, friends. Well, good. Next step is to go to CodeRabbit.ai. That's C-O-D-E-R-A-B-B-I-T.ai. Use the most advanced AI platform for code reviews to cut code review time in half, bugs in half, all that stuff instantly. You got a 14 day free trial to easy no credit card required, and they are free for open source. Learn more at coderebbits.ai.

34:58Can I share my screen and indulge all with my initial clod code? And I'm going to go from the very top to the very bottom. So the very top obviously is where I began. The very bottom is where I ended up. there you go that's how things work when you're in the terminal so obviously it's npm install and then i had an issue because i guess well i guess you actually installed and then it told me there's a new major version of npm available so i installed that like a good boy and then i tried like an idiot to reinstall cloud code but it already installed i didn't realize that and then so i naturally changed directory to this very cool repo called adamstukowiak.com which is like anybody's like my own little blog here is where I play around with.

35:38And obviously I'm in this tailwind branch that has dirty stuff in it. Dang you tailwind. Why are you sitting with your repo? Oh man, but whatever. Okay. So I can tell you the last time I was, I don't mess with this code too often, maybe once or twice, a couple of times a year. Every once in a while I think I'm going to blog again, but I just never get to. Yeah. I think I got something to say and I'm like, oh man, I just never said it. Anyways, the last time I'd played with this repo, I got this wild hair up my butt where I didn't want to run Jekyll and run Ruby and run all the gems and all that stuff on my actual machine.

36:12I wanted to run this thing in Docker. I wanted to be completely isolated from any Ruby versions, anything like that. Because I only use Ruby really in Jekyll. So I was like, I don't want to mess with Ruby at all. I wanted to all be in Docker. And so the last time... I don't think about Ruby. Say again? That's how you feel about Ruby as well. So the last time I did this, I knew the, that I had some thoughts of like, okay, I don't want to mess with, you know, my Mac. I wanted to keep it pristine. Everything's in Docker. And so obviously here is the now we're now in the present and that's where we're at.

36:45I'm like, I forget how to use this thing. Right. And so then I told it to do, what did I tell it to do? It said, do you trust? Yes. Confirm here we are. And then I said this right here. So that's my little prompt there. I said, I've run this Listen, Docker, can you remind me how this thing works? This is what I love about how cool this tech is because I'm now in this repo. I forget how it works. And it says this is how it works. It goes and reads the Docker file, the readme file, et cetera. And then it's like, hey, this is how you run the project. DocCompose, up, dash, dash, build. And it reminds me of all the things.

37:23Here's how you can do a post in Jekyll, et cetera, through Docker exec. I always forget Docker exec dash dash or dash IT and what all this mumbo jumbo is. I always forget that. So that's nice. Man, how cool is this? Like, this is how you run it, how you create a new draft, how you build the site, how you access the container for more commands, and then how a lot of reload works. Like, to me, that's just, that's beautiful. That's the way it should be. It is beautiful. You've seen the light. I've seen the light before, but I'm seeing the light literally. Every time he sees a light. moment impressive you know i get it sometimes i'm blown away as well but sometimes i'm definitely not oh yeah like it's it can be real dumb and like i've had i had an issue node builds right i wanted to build a project for esm and common js i think that would be like an easy thing to do in 2025 it was not and i kept trying to like get claude to do it and it would it just went in circles for like over an hour i probably spent 50 in tokens like just going around and around i took the initial prompt and gave it to chat gpt 03 pro and it was like oh this is how you do it and it was like and and to prove that this is how you do it this is exactly what i'm proposing is exactly what tan stack does and exactly what zod does and it like showed examples from each of them And I like, I just copied that and pasted it back to cloud code.

38:54And I was like, oh yeah. And it fixed it immediately. Take this. And so that got me like playing with these, like, you know, how can we do this? And I found this cool tool. It's like a paid tool. And I don't, I don't know how to use it because it's very, very, very confusing. But it's cool. It's kind of cool. It's called repo prompt. And you like drag a folder in from your repo and you can drag multiple repos in. And then you can give it a prompt like, I want to do this and this and this and this. And it will run like a really cheap model to determine what files that you've given it actually matter to the context.

39:32And then it'll include those. And then it has a copy button so that you can copy a prompt that you made. And you can add in all of these prompts like, oh, I want you to be an engineer or I want you to be an architect and do all of this. And then you tell it exactly what you want to do. It knows what files to include in the context. And then you can take that and go give it to an O3 Pro or like a very high expensive model to do the reasoning on it. And it's going to give step-by-step guides. And it includes like instructions on like when you tell me to update code, give it to me in this XML format so that I know exactly what to do.

40:05This XML diff format. And it will do that. You literally just take whatever Claude or O3 or whatever gave you, paste it back into this tool, and then it can use a super cheap model to go actually implement it. like a Gemini, whatever. This is like mimicking real life, isn't it? You got your solutions architect, right? Your system architect and they're expensive. And so their time is precious. And then you have your like entry level software engineer. That's just like, you know, grunt work. Go ahead, grunt. Work on this until you work your way up to architect and they're going to be cheaper, right?

40:37They have more disposable time. And so you're doing like a, you're distributing the load based on price and quality. That's pretty cool. it's fascinating and like you can get wildly different results and like it just shows you that that when a model is failing and you can tell pretty early when claw it's just going to spin its wheels right yeah go to something else give it to something else and like make them work together is is like the the suggestion i suppose because they will come to a solution better and faster than than i would you just said it was not very smart at the beginning of all this then you're saying it'll come up with a solution when i'm giving it more power give it what it means yeah use use the prompts use the tokens the expensive tokens for what they're good for and don't waste them on on the mundane like application work right can we go back to this uh agent experience you had jerry where you fixed a bug and you didn't feel like you were working like how can you do that at scale is that like a is there a scaled version of like this agents in the background and just sort of like tidying things up.

41:41I think about it like a Roomba or a iRobot or something like that where you're like, I kind of just want clean floors. They don't need to be immaculate. But you know what? Come on, robots. Clean the floors a couple times a day. Is it like that? Can you describe some versions of just like project maintenance with this agent stuff? I mean, I don't do project maintenance because. Isn't that what that was though? Where you were like, remove this file, drop these feeds, SQL query to do it. That's kind of like project maintenance. Yeah, that would be like data management. But in that case, I think, you know, I'm fixing a bug.

42:10So it's maintenance and the fact of like, you know, it's not working right. Let's make it work right. But if you're talking about code quality, adding tests, et cetera, refactorings, you can certainly use it for that kind of stuff. I haven't done that myself. I know that when you first launch clod code, for instance, it's like, it gives you a few ideas of things to do. And one's like, suggest improvements to this file or this project or something. And we'll go through and say, here's some low hanging fruit things you could do or help me secure this. Like that's kind of stuff that you could certainly set one off to do a security audit against your entire code base and then come back to you when it's done.

42:47But I'm more curious how Nick does it when he's working. Like, could you, I mean, you code for hours, right? Mm-hmm. Okay. Theoretically. Depends on who's listening. No. Well, because I have other stuff I'm doing, right? So like for me, I'm writing a newsletter and I'm recording a podcast. And so I'm having to do stuff while I'm not working on software. And I wonder what I would be doing if I was like, if my job that day was to fix that bug, would I be on Reddit? Would I be bored? Would I be off doing something else? Like when you're using these things throughout a day, are you also coding in a different area of the code base while it works?

43:26Or how do you manage? Because there's a back and forth where you're not really doing anything. Oh yeah. Sitting and waiting is, that's like the amateur, right? Like there's a process that you go through and like, you know, getting to these tools is like, you know, the good you're, you know, you're in the fourth dimension now, but you're just like twiddling your thumbs in the fourth dimension. Now you have to go to the fifth dimension where you're like going through, you've got multiple of these agents. I think that's something that I really took away from, from that podcast with Steve was like, just set these agents off to do a bunch of different things all at once.

43:58And like, you're no longer involved at that. You're a manager of agents. And maybe you have an agent that manages sub agents and goes from there. And like, yeah, I posted a link in the, in the things it's a blog post I have called how I use get work trees. And I only call this out because it's by far the most popular post on my website even still. And it gets like regular hits. And I think it's because of AI. And I think it's because work trees are just a fantastic way to work with these tools because you can set off Claude in one work tree to do one task and then have another work tree where it's doing another task.

44:33And these are effectively just like different checkouts of the repo, right? But they're all connected. And so then you can just like hop between all of these and do like five things at once in the same repo and then come out with five different pull requests. And that's really easy and really fun to do. And like what I'll often do is like, just because I want to feel like I'm still hands on keyboard a little bit is like, I'll have it off going and doing stuff. And then I will work like more in depth on one of the problems while I'm like kind of traversing between team accessions to keep the prompts going for other ones.

45:08But the main thing that I do is like I make it I make it write a plan and stick to it. So I'll spend hours at the beginning going through a markdown plan. And it's just like step by step, phase by phase. It likes to do things like phase one and then there's like steps one through five of phase one. And then that way, like I can focus it on doing that. Like I know exactly what it's going to do from start to finish. And that's written to a markdown file. So it's like in there, it's something tangible. I can, you know, kill this instance of Claude and come back and just give it that file and continue going.

45:40And then I tell it like, all right, let's work on implementing phase one. And it'll only do that. And then it will like, you know, do the changes and it will stop and think and ask questions based on that. I have like a whole bunch of prompts that I, you know, tell it to ask me questions or wait on things. And then it will, um, it'll stop right there. And then once we're done and I like the phases, I usually try and keep them to something that I can like manually test to make sure, yes, this is actually working correctly. Then I'll do a code review and fix things or, you know, clean things up.

46:11It likes to be very verbose. It's not very dry. Not that it has to be very dry, but like it does things that I probably wouldn't do. So, um, I try and make it do things that I would do and then um and then you can do like a slash compact or a slash clear i usually just do compact and that will like compact the the session down and give it a summary of what we've talked about so far so that you have a new context window full of you know fresh memory and then now we'll start on phase two and it kind of has like hints of what it did in phase one and it can go back and check you know obviously but uh then it kind of starts fresh with phase two and goes from there and that's how i like i really like to do it so like i set it off to do phase one i go work on something else and then i come back and see where it's at and you know monitor it but i'm usually doing like two or three things across two or three repos usually like just like all over the place i maintain a lot of sdk so i'm just kind of constantly jumping around so you're doing it man you're uh you're ai babysitter yeah now ask me if i'm more productive I was going to ask that.

47:11I was like, now, honestly, is it faster this way?

47:17I feel like it's like a learning thing, right? I'm still learning about how to properly use these tools. And I was explaining this to a colleague yesterday. I think that like, it all comes down to who I think has more context going into something. If I have more context about it, I should just do it because I'm going to waste way more time explaining to Claude how exactly I want it done. But if I don't have a lot of context, like I'm working on right now a plugin for BetterAuth, which is like an open source authentication library for doing that in JavaScript. And I'm working on a plugin for it.

47:59I've never used BetterAuth. And I've never really fully set up SSO from the start. And this is an SSO plugin. and so like i don't have a lot of context of either of these and so i just you know i literally spent an entire day just going through a plan of what i think i want and not just what i want but like i also need an example app to like prove that this works and prove to myself that this works but also as like a demo and that was day one coming up with a plan day two was just guiding claude through that plan and by day three i had it working and that's not something that i would have done on my You know, it would have been weeks and weeks of me just like toiling for a while.

48:36Do you think it would have been weeks or it definitely would have been weeks? It would have been weeks. Yeah. Certainly got to be a DevRel's dream, right? DevRel's always making demo apps and trying to show off things and prove the technology to themselves first so they can become enchanted with it and excited about it and all the things so they can go then tell the rest of the developers, hey, this is how it works. And here's a demo app. And here's how I made it work. And you could do it too. i mean three days to to dev rel perfection is better than weeks procrastination now of course i haven't shipped that plug in yet like the code it's it wrote a lot of code and i'm scouring it but like that was more of a project where i was like i'm just gonna you know i'm gonna give you auto accept edits and you're just gonna go do it and i will review it later and like clean things up so i'm kind of like in the process of doing the cleanup right now and understanding it from top to bottom, which is much easier than me, like having to do that from the start.

49:31So lots and lots of code reviewing going on right now. But, but to your point, Adam, I did hear something, I think yesterday where like some, it was some company talking about how they have like a soft rule now on like ideas. Don't come to me with ideas, come to me with vibe coded proof of concepts. and that's like for everyone in the company from engineers up to like go-to-market folks or like everyone in between like just come to me with with fleshed out proof of concept and it doesn't have to be perfect code you don't even have to have looked at the code but you are showing me a real working thing and that is something that's very achievable with the tools right now so as we talk productivity we just had abinoda on the show last week he brought some survey based and some quantitative as well analysis of actual productivity inside of enterprises and they found that people that developers are generally 10 percent more productive which for me was lower than i expected but there are a lot of a lot of details that conversation i guess you can go back and listen to that one however since then we've had this new paper that came out which is what we've been alluding to, but haven't actually talked about yet on this show, which is this is a research study that happened recently and a paper that was published by Meter, M-E-T-R, which seems like it's actually a pretty well-run deal.

50:59Now this is synthetic, like they, and so far as it's set up, it's not just real world and then see how you're actually productive. It's like, here's what you're going to do. They go and do it. And from what I've read, people who are in the know about these things, they did a pretty good job running this particular study. And what it found, and this is on open source, this is called the Measuring the Impact of Early 2025 AI on Experienced Open Source Developer Productivity. So there's your context, experienced open source developer productivity. That's a small slice of the software engineering world, but an important one.

51:39And what they found is it says when developers are allowed to use AI tools, they take 19 % longer to complete issues. A significant slowdown that goes against developer beliefs and expert forecasts. This gap between perception and reality is striking. Developers expected AI to speed them up by 24%. And even after experiencing the slowdown, they still believed AI had sped them up by 20%. That's one of their major findings. And I think we can talk about some of the reasons behind that. But there is this perceived speed and actual speed thing. That being said, Nick, you seem pretty confident that your perceived speed is actual speed.

52:29and curious your reaction to that number 19 slowdown seems like a bad deal if uh compared to what everybody has been telling us yeah no i i believe it and i also think that i'm more productive because of ai um it's it's because well i'm a team of one right now so like i rely on conversations with ai like top to bottom it could be having a do code like write code it could be just having an architectural level discussion to flesh out my ideas before i go bother a real human you know i've knocked out all the low-hanging fruit i'm going to sound super smart because i know everything because i've already had this conversation right and now i'm validating whether it was just you know being telling me what i want to hear which they love to do and i hate it so much um but like we're all the the study like i'll be honest i didn't read the paper so i don't know exactly like the parameters and things that went into it but if it was early 2025 that's going to be different than june 2025 for sure definitely uh but also like we're all going through this right now like figuring things out it's like um we're just you know throwing spaghetti at the wall and seeing what sticks what works what gets us there and sometimes the thing that worked one time doesn't always work because it's like it's like a slot machine you know and it's it's difficult to like hone in on what's going to get you to where you need to go.

53:57And you might like change your process slightly and then waste a day because of that. And that like obviously slows you down. And so there's a lot of experimentation and it's going to be interesting to see where it goes from this because like, are we all going to become super personalized to where like these prompts work for me and the way I interact with it, but they're never going to work for you, Jared, or you, Adam. Like we're all going to have our own bespoke things, just like we all have our bespoke timelines right now with like the social algorithms and all right like nobody is looking at the same things now we're not even like coding the same way uh and that could be super interesting and i have this like this is a tangent but like i have this future casting thought uh going way down like what if in a year from now these llms are just so optimized and they under like i just i don't write code anymore i just talk to the llm and the llm maintains code for me and what it's actually maintaining is just like wasm byte code and when i need to step in and help it debug it's going to give it to me in typescript and jared's going to come in and be like oh jared can you help me with this and it's going to give it to you in ruby because that's more tailored for you and it's literally the same code like over and over like we're just gonna like not even speak the same languages anymore right it could be interesting that's a cool idea i'll never have to see typescript again zing that is that really is cool though because like that's essentially what we do now like it is it kind of all goes to some degree to bytecode right you know it's there's a layer the layer i speak you know the layer you speak we're speaking different languages in some cases different protocols different access levels to the file system or whatever you might have that's already kind of happening now you're it's kind of personal preference i suppose in that case there's something to this this phenomenon happening where we're working differently like the perception of effectiveness the perception of speed or what's the the true terminology here like improvement efficiency what was the terminology we're trying to like really capture here in terms of like goodness you know is it really better or worse productivity is what they called it thank you okay because like you said nick in three days you did something that you would have probably done in weeks or multiple weeks and potentially didn't even tackle because it was just too challenging for you without some support whether it's human or not and that to me is pretty wild.

56:30You know, you feel for the moment, like you're enjoying the work more. So you feel more effective. You feel more productive, even though your output is roughly the same. You're spinning your wheels, but you're spinning them in ways that you're getting traction. Even if you're incrementally moving forward, it feels like, you know, like this perception from the humankind, it's, it's like, yeah, I feel better about the work I'm doing. And I'm, I'm enjoying it. And Like you said, Jerry, where Steve kind of helped you have permission to play with this and then have fun with it. It's like it opened up this new human layer, which is enjoyment.

57:06Kind of necessary. For sure. And like to that, I feel that there is a reckoning coming with that. We'll speak of it. I've. Scary. Warn us. I have. Get the reckoning. i've i've been more on the fearful side for most of this ai hype right like i don't want it to i don't want it to take my job and and uh i don't want you know it to do everything i don't want i don't like the feeling that like oh congrats you solved this hard problem eh claude solved it i kind of just put my name on it you know i don't like that but like at the same time i'm like this is kind of the new reality and i can move faster because of this and my i've always thought of myself as like a craftsman for a better lack of a better term like sure you know i really like the code and solving the the problems in there and i'm like focused on like the the micro problems and less focus not less focused but like that's where i really get my enjoyment and my like dopamine hit is from the the little micro problems that i solve in the code and then you have like the macro like the the app you know you've solved a you've created a feature in the app or you've done fix this major bug or something like that that's like you know more macro and i think that if you were as we're going through this transition like every engineer is going to have to like this is the reckoning like they're going to have to figure out can they transition to solving the higher level problems and letting the ai handle the lower level things and of course reviewing it and you know you keep it up like that but are you going to get your dopamine hit from that and are you going to be able to transition your workflow to potentially like managing fleets of these agents as an engineer and that's like what a software engineer is now and if that's if you can that's great if if that's not interesting to you then maybe people start self-selecting out of this field and that's where like the the AI is taking our jobs yeah right I don't know if that's how it's going to go but like I could see it going that way I could tend to agree with that.

59:08I think that I have both perspectives. And so I think probably why I was mostly trepidatious and now I'm just more excited because I also have been focused on the quality and the craftsmanship and really the maintainability of software my entire career. Slow down and go faster is one of my mottos. And I think that motto might not matter anymore because... as the quality of these tools improves which so far it has i mean clot what i'm using today versus last year versus two years ago is way better in terms of the actual output and its ability to write code and to solve problems and if that trajectory continues not even at a you know parabolic pace but just even like at a linear pace we won't have to maintain software like we have in the past, especially as the price of compute, you know, trends as close to zero as it can, which is what all the investment money is going to.

1:00:10It's like, let's get some more free energy and let's come out with ways of doing this. It doesn't tax the grid so much, et cetera, et cetera. Those are all just like time and money and innovation will solve those problems to where it becomes very, very cheap to have very capable coding agents working for you. So along those two lines, why maintain software like we used to like why care so much about the craftsmanship do you actually have to slow down to go faster and maybe you just go faster to go faster and replace when necessary build modularly and replace parts that need replacing and at the end of the day the reason why i cared about the maintainability was because i wanted to be able to continue to go fast over the long time period, right?

1:01:00Like everybody starts off fast. That's why they call them sprints. And that's why we have spikes and proof of concepts and startups that, you know, just ship, ship, ship, ship. That same startup that was just shipping like crazy 18 months ago, they start to crawl because of the maintenance problem. Well, if that problem gets solved, and this is assuming that that will get solved, then at the end of the day i just wanted to go fast and get stuff done and have the productivity like that's what i was after i just realized pretty young in my career that you couldn't just accumulate technical debt non-stop in order to go fast like it just wasn't smart to do and so i would often sell that to people like hey if you want to build a thing with me let's build it but we're going to build it right that way you can continue at a pace that's sustainable right at the end of the day i was after the the end not the means and so what i really like is to be productive and stay productive and at least for now that's what i'm enjoying especially as a busy person it's like i don't have eight hours of day to write software you know i might not write any software but with cloud code i might write some software today even though i'm not gonna be writing it for me like that's super exciting and so i can see a world where that's what really matters is to move up get more abstract less hands-on but more productive and more software at the end of the day so i've been of two minds but i'm leaning more towards that mind of like okay let's just embrace this and see what we can get done or as i said in news yesterday let's get cooking you know all that excites me every like like i literally when i look at my arm i'm like i got like chills i'm so excited about that future right i'm just like i'm down for it except uh oh except this is goes back to that monkey theory i shared i think less eloquently let me try again on a previous podcast you have a room where monkeys are locked in it right and there's a ladder and it's fixed in place vertical can't fall over and it's got bananas monkeys love bananas right they're at the top of this ladder right what do monkeys do they crawl they're pretty easy crawling right they're pretty strong pretty good at this stuff so these monkeys they're brand new to the room brand new to the ladder they love bananas they know all this this is their instincts right they're gonna crawl that ladder to get the bananas but they start getting sprayed fiercely hard like it hurts bad some of them may get a banana and they're excited about it most of them I'm like, nah, man, that hurts too bad.

1:03:39I ain't trying again. Like, I'll try one more time. And they get blasted. They try one more time. They get blasted. I'm elongating the story, just for emphasis. So these monkeys learn, this initial group of monkeys learn that banana good, ladder gets me there, water hurts, don't go do it because the water hurts. And so they get trained, don't go and, you know, get these bananas. And then they incrementally swap one of those monkeys out with a new monkey that knows the old instinct, instinct, doesn't know about the water splashing and the spraying and the hurt. They just know banana again, right?

1:04:13But they come into this group with a previous knowledge and the old group's like, no, no, no, no. You can't get those bananas. And he's like, why? Bananas are great. Ladders there, crawl, banana. Water, spray, hurt, not good. Anyways, I'm entertaining myself as I'm telling this story. Are these cavemen or? They're not very smart, but they're very smart. I get it. If something become cavemen. Long story short, But over time, the monkeys teach the new monkeys coming in. Don't go for them bananas. Don't do that anymore. And so I juxtapose that against this idea that you want this world. And I agree with that world, Jared.

1:04:46I think it's great that we transcend the need that my abilities to write software well is predicated on my ability, my personal ability to maintain that software. So this aha moment you came to was essentially like, OK, this new world doesn't have to be constrained by me. it can be construed by my brain but my ability was what you're coming to is like i don't have to be the person maintaining all this code anymore or and you said this and clarify if i'm wrong is that you said that you don't even need to understand the code anymore did you say that i don't know what i said i said a whole bunch of crap okay did you hear that nick that he didn't really need to understand the code anymore he was okay with that at which point like today at some point at some point along this journey it could be a year from now at some point you're okay with letting loose, not having to understand all the code.

1:05:35Oh yeah. Just let loose and just vibe it up. Okay. So the long monkey story, all this detail, okay. To get to this point, I don't know if I want to live in a world where we no longer understand the code we write. Like what happens when we can't understand this code anymore? And the only way to knowledge of what it's doing is through an AI that we think is for us. Yeah, I mean, like, we think it's for us, and we're designing it, and so we're sort of designing it for us, but what if we get to a point where these monkeys that says banana good, but the old monkey's like, nah, banana bad. Oh, we've solved that.

1:06:13We've solved that. We have. We just don't have new monkeys coming in. That's right. Not new monkeys. To be clear, that's not a good thing, but, like, that's what it feels like. We are the last generation of code monkeys. I get what you're saying, though, for sure. Like, like you're right now we're focused on the artifact, right? Like the algorithm, the file, the, the module, whatever. Like we're focused on that as the outcome. The thinking's the valuable part. And the thinking is what the LLMs just can't do very good right now. And maybe ever, I don't know. And that's where we'll always be able to, to guide it.

1:06:49And so we have to, that's like the reckoning, right? You have to shift. Am I more focused on the, on the artifact or did I, do I feel fulfilled because I used my thinking brain right i don't know if the thinking is actually the differentiator i would i would slightly augment that and say our context plus taste is what makes us difference whereas of course we can say well what does thinking really mean are they actually thinking well they're just brute forcing a bunch of word autocompletions fine but it approximates our thinking and a lot of times they actually have better ideas than us because they're like i wouldn't have thought of that great idea or i'm like i wouldn't have thought of that terrible idea that's judgment that's context and that's taste i think that's what makes us different moving forward uh more so than the thinking i'm just using different words to describe the thing the same thing that you were describing so i'm not necessarily disagreeing with you but i don't know adam i feel like maybe you're anthropomorphizing these things too much like you're assuming well you say like we think they're on our side or something and you're getting very much in the t2 territory here i don't think i mean i was decompressing a longer version i mean i was saving us from one more monkey story okay okay well thank you save us from as many monkeys as as need be so i don't i don't think that at a certain level of software quality that can be objectively tested as best of our ability produced on our behalf by ai models and agents into the future.

1:08:25I don't think not having to look at the code or know what the code does, whatever I said, is all that big of an issue. How is it different than assembly code, which I've never looked at in my entire life? Like at the end of the day, you could say, well, you never look at the assembly. You don't really know what it's doing because you don't look at it. It's like, no, I don't, but I have it translated into Ruby for me at the last second so I can read it and then put it back into whatever form they wanted it, you know? You know, maybe I'll take a step back then. So this is a great exercise of a volley.

1:08:59And so what you just said basically is that we can understand the code because it can now be spoken to us in a language that we understand. So just because we don't understand bytecode or assembly anymore doesn't mean it can't explain it to us in a different language, which is actually a programming language, or potentially in English, like a reverse prompt. Like, here's your code. here's a non-prompted response essentially this is how it works and so you want to get back to this non-deterministic reproducibility if they get so good at coding brand new things this brand new module and maintainability goes out the window well now all you have to use is the the language i know which is english to get the software i want which is something i do not understand i don't know i just that computes to me but i just wonder will we be motivated as a human race to have a certain amount of people call them scientists call them whatevers that just have to maintain this intellectual knowledge to the point they always know like there's a subset of group that's always going to know we have those today they're called gray beards the gray beards you know the ones who still know assembly they still write assembly on 8 p.m on a saturday you know when they have their glass of spindrift yeah i don't know obviously we're getting hypothetical at this point i think that you do want to have some people who are still in touch with the underpinnings of our society yes i would say that's probably wise to keep some people there um but how many generations do we go down you know that's when it starts to get disconnected because if you're talking are kids fine are kids kids maybe are kids kids kids i don't know it's gonna get pretty weird well even think about that like kids kids kids you can almost um you can not quite but you can almost run that test now so how many generations can you back go back to like your great great grandfather did you meet your great great grandfather no or grandmother i didn't either nick the jew no okay so how many back is that great great that's three my dad's dad my dad's dad's dad all right that's three back yeah pretty easy to get to that's not even six no three is not six no that's not even six it's not seven either just so you're clear i mean like three not six monkey understand on us okay i'm stuck in my monkey speak like banana Yeah.

1:11:26Monkey, ladder, banana, good. Water, bad. Water, real bad. I feel like there's some sort of like PTSD going on where maybe Adam was experimented on and he's still just like acting it out later. Maybe so. The point I'm getting to is like my grandfather's dad, I can't even relate to the challenge of that person's life. Right. Completely different lives. Except for the human struggle. that's the that's the constant is like we will always struggle in humanity yeah whether relationally interpersonally uh you know inside yourself all the way you know psychologically yes there's always that struggle there but like i can't understand that person's struggle and our lives are so dramatically different that i almost wonder if like he wouldn't care maybe that's what you're saying just like you just don't care because you can't understand contextually the empathy required or the care required beyond being human for the for the task challenge you know not the life challenge but the in the moment challenge it's just too far-fetched well we'll probably get stuck right where we are but our kids kids kids they'll have a completely different context you know and we've likened it in the past to driving a stick shift versus an automatic and how there's a generation of people now it's more just like preference and hobby of course you You can still drive stick shifts.

1:12:53You can find them. You can buy them. You can build them. And there's people that only drive a stick shift. They're very rare. But 99 % of humanity does not know how a transmission works, right? And how the difference, like what a stick shift was. You had to actually shift the gears. Wait, wait, wait. What is a stick shift? I don't know. I'm just making stuff up here. Hopefully you guys don't notice. I drove a stick shift. Today? No. Oh, you drove one to drive. No, at one point in my life. I drove one once. Well, more than once on a few occasions, my friend had one. And so I had to drive his. And then my, one of my early jobs had a stick shift and little, little arrogant.

1:13:30Jared was too egotistical to say, when someone says, can you drive a stick? I, my ego was so big. I just said, yes. And I had never done it before. I had watched people do it. And so I had to just figure it out on the road, which was a terrible day. but that's uh arrogance and youth are eternal companions anyways i do think that that with how um how fast you're able to turn out code now um and you're you're you know potentially looking at it less uh you're definitely looking at parts of it less like just trusting yeah it just works right that is like it's making me realize that for the most part like the code i write the code i get paid to write it's pretty boring the assembly line right this is not handcrafted custom furniture this is an assembly line and i got to get it out the door right and it's helping me to like come to terms with that it's been there all along but it's just like yeah that's true and i think that's what we're going to be automating is the assembly line of software and the assembly line workers whereas like we're not going to replace why the lucky he's stiff, for instance.

1:14:41I was gonna say RIP, I don't think he's dead, but he left the internet. And like that guy used code as art. And he wasn't just a craftsman, he was an artist. Probably still is, I'm still using past tense because I haven't seen him for years. Just for instance, or somebody like Ken Thompson, who's just like building the foundational bits and is just amazing, savant of software. not replaced still necessary probably for a while like the people who design certain protocols you know like whoever is out there doing http 3 if that's a 4 i should say 4 h3 is out there right quick like those those things so like code is art low-level bits of like hardcore engineering all i've heard some hardcore engineers confess certain areas of their code were just completely vibe coded uh because it's like take this thing and you know because you can kind of approximate it with it but the point stands i'm not saying they never go away i mean i don't think code is art ever goes away but um i have a counterpoint go ahead you what i'm never a counterpoint i I haven't been one yet.

1:16:00How so? I heard you on a podcast. I don't know. I don't remember which one. But as you've been going along this journey, you at some point talked about using this to like the ROI on like developing tools for yourself is too high in most cases. So you just don't do it. But now you can and you have and you've made like little scripts and tools. That's like the artisanal coding. Right. And you totally automated that away. so it's not like you're not like the furniture person working at the factory who comes home and hand creates the chair that they're sitting on so i guess i kind of agree depends on what you mean by automate away like i'm doing more of that stuff now yeah i'm just not physically typing the keys but i actually have more scripts today than i would have had without these tools you know i'm saying so i'm actually doing more of that uh-huh not less so you're saying i'm automating it away I guess, because I'm not writing it.

1:16:55I mean, I guess I was taking it in terms of like still being the, you know, the hand, the craftsman working on. Oh, I see what you're saying. Making your own tools. When you have the time, you're not going to sit down and be like, I'm not going to use Claude. Oh no, I'm never going to just write, sit down and write a nice function anymore. Right. You know, I'm not. Oh no, no, no. Unless I'm going to submit it to the hot or not, you know, pretty soon you're going to become dependent. you're gonna be like oh i wrote this but i hang on i gotta just check with claude to make sure lose all my confidence as a young man i would have just deployed this but today i'm gonna check with my overlords yeah possibly but i am excited also about more artisanal software this is why the headliner in this week's news which wasn't really news was all about a an app can be a home-cooked meal and this idea of like we can build one-off personalized apps and scripts and websites that we would never have had the time or the ability to in the past.

1:17:58And if it didn't scale, you weren't gonna build it. You know, like that's the kind of mindset I would have to have is like, if I can scale this to 15 people, then it's worth it. But if it's just for me, it's just way too much work to build that thing. Or if it's just for my family, not worth it. So like that calculus completely changes. And now I'm just like, why not even just give it a try while I'm doing something else? this thing can be vibe coding it. And maybe it's too hard right now. I mean, there's things that I've tried to build. I'm like, nah, they can't quite do that yet. And there's other things where I'm like, yep, easy, cool.

1:18:32And I think that explodes. So we're going to have way more of that kind of software, at least in the short term. But you're shifting away from the artifact, right? And more towards the outcome. And the outcomes are great and they're wonderful and they wouldn't exist without this. But what do we actually want? Do we want artifacts or outcomes? I don't know. Are you a craftsman? I told you I have two minds on the matter. Like I, yeah. At the end of the day, I think I always was more about the ends. I always was. I was like, I want the website. I want the plugin. I want the superpowers. I want the, and I realized at that time, 2020 or 2004, 2005 through, you know, up until recently the actual craft of the software mattered so much to get that thing in a way that it wasn't going to crumble after I ship it to the world.

1:19:23And so that's why I cared about the craft and I can totally nerd out about the details and get into the flow state of writing the best quick sort function you've ever seen. Like I can totally understand the dopamine hits that that provides because I've done that for years. And so that was why I was on two minds. But like what really matters, the dopamine or like the longstanding value that it brings to myself and other humans? your fulfillment. My, my fulfillment's what matters the most. But it can also afford you that, right. It can be off doing like 90 % of the work and you're sitting there writing one function and being like, I contributed and you got that dopamine hit.

1:20:02Meanwhile, it did, you know, it created an entire truck. Totally. But look at that decal I put on the bumper. That's all me. Maybe that bumper sticker. I came up with that. Yeah. I will say like, like one other thing that's really cool. is like with this and like with Claude, I'm sure other ones are like this too, but Claude's the only one I have like the most intimate experience with. Same. Is that like, you have all of these like secret words that you can use. You can tell it to use sub agents and it will actually like split off into sub agents and do things faster. If it doesn't do things right, you can tell it to try hard and it will reason better.

1:20:40And you can go try harder and you can say ultra think and that is the ultimate amount of reasoning that it will do. and so like we're finally to this point where we literally have incantations that get us to where we want to go and like you if you know these incantations you are casting these spells you're an engineer prompt engineer yeah that's crazy man you like that i actually i hate that makes me mad i just wanted to try hard every time like i just shouldn't have to tell you like now i'm talking to a human again it's like come on i shouldn't have to motivate you yeah you know you can do this Claude go and do it try harder it's like a programming language that isn't a programming language like you have to just know these keywords and right and do them and you like you can read them through the docs probably but like you learn them through trial and error well trial and error and just like legend that people someone else telling you something yeah yeah that's a that's a truth that i think goes is trending downward over time and i hope that it trends towards nothing.

1:21:41I should not have to tell it to use sub agents. I should not have to tell it to try harder. I should not have to tell it. What other things you're telling it? Ultra think. I should tell myself that like when I can't figure someone out. Come on, Jared. It's like go, go gadget arm, you know? Yes. That's what I was thinking. I'm going to start using that. Come on, Jared. Ultra think. oh it's interesting times isn't it it it may it does like it does kind of make you question the value there's like different layers of value to software in an enterprise it's clear what the value is right but in personhood in person life like you're suggesting like just write this one off piece of software that satisfies my life like you mentioned like with this home cooked meal i'm so down with that i'm so down with that because software is cool and the whole reason why we're even sitting here is because software is cool it's not because necessarily we all want to be and it may be fun to be a software craftsman or craftsperson but we kind of got locked into that mindset because of our limitation of us pushing that ball maintaining the software so we're that monkey don't climb the ladder right right and now we can climb the ladder with no water and the water hits us we got we got different skin now we've we've evolved yes but software like it is such a powerful thing obviously and to give so many more people almost the entire human race the ability to create software as a home-cooked meal i think is a is a world i want to live in where people can make and create software that benefits their individual life and maybe it can go to a scale thing if it needs to and maybe there's a maybe commerce doesn't even matter after that like maybe that it's successful or financially viable doesn't even matter because things will evolved to the point where it's just software it just runs everywhere it doesn't matter you know it's just really ephemeral or just really blase about how it was created or what it was written in or doesn't even doesn't even matter does the ends work does it do its job almost like uh almost like tron you know what do they say um like tron in tron it was like uh they were going down the line like looking at each program and saying you know rectified or deleted remember that part in the latest trial when he said that it's like you're a bad program you're not doing your thing you got to be rectified and you get rectified and you come back out new software that's doing the job versus look how maintainable this thing is like that's a good thing for a human it's not a good thing for anything else than that because that's all you care about is like humans care about time we only have the window of time we have you know in this on this earth so our finite resource is time so we care about things that really benefit our moment and doesn't take away from future moments speaking of time i think that there is a this is the perfect time to cash in on uh cleaning up all of these ai slop-coded apps because there is a lucrative business there for sure is there in a time window oh yeah in a time window right in the in the current window of time yes um ai hero what do you call it i don't know slop cleanup aisle nine whatever your company is called if you can offer that to clean up ai yep for sure there's an opportunity there for now until their code is way better than ours yeah and um and we'll see if that actually happens but we didn't really rebut this 19 slowdown very much i mean we're over here talking about how amazing everything is but like cold hard facts are saying nope now i think there's a few factors that play into that in fact we'll link to this page this blog post called ai ai slows down open source developers peter nauer can teach us why um because it goes into details on why this particular set of people in this particular place,

1:25:54some good reasons why they might have been slowed down, whereas in other areas and other contexts, you actually are sped up like Nick confesses to me. So we'll just leave that in the show notes for people to read. But the short of it is high context people know their software in and out already. value, get less value out of having to provide that context to AI versus being able to just do it themselves when they have the entire domain in their heads and they've been working on it forever, perhaps slower with having to wait on a robot to do things when they could just do them themselves. I think that's a generalization of what that argument is.

1:26:41So, which certainly played a part, which is why I said that's a, it's a slice of, developer land, but it's not the whole thing. Yeah. In terms of the rebuttal, I think that sometimes when you do a change that creates efficiency, sometimes there's inefficiency while the system gets recalibrated to become efficient. Sure. So I wonder if the perceived advancement in speed to the person and the truth and the data is not accurate. Maybe that's just temporarily accurate you know like it's it's actually they're actually less efficient or as efficient if that's what the data is saying to be determined if they'll become more efficient i wonder if that more efficient will actually just come because over time you will recalibrate to this new inevitability of how we work which is ai is here for everybody better use it and get replaced or use it and replace somebody who doesn't use it i think that's kind of an interesting take really is that maybe the efficiencies come shortly after not right away well after nick gives you all the magic incantations you have to know then you're really good at it ultra think ultra think all right anything else or should we say goodbye well that's it for now that sound you hear is the sound of inevitability get them bananas bye friends bye friends next prediction of a coming reckoning was a lot but there's more he shares a second reckoning in our changelog plus plus bonus segment that's coming up right after this unless of course you're not directly supporting our work with your hard-earned cash get in on the action at changelog.com slash plus plus and enjoy an additional 13 minutes of us discussing the browser company, ARC, DIA, OpenAI's new browser, and more ultra thinking than you can shake a stick at.

1:28:33Thanks again to our partners at Fly.io and to our sponsors, Auth0, the identity infrastructure for the age of AI. Start building at Auth0.com slash AI. And to CodeRabbit, AI native code reviews. Learn more at Coderabbit.ai. We are officially one week away from our Change Log Live show in Denver and I'm officially revealing our mystery guest. We'll be interviewing the one and only Nora Jones on stage at the Oriental Theater. You may remember Nora from Jelly, her incident response platform that got acquired by PagerDuty where she now works. Join us and Nora in Denver, head to changelog.com slash live and get tickets.

1:29:12Have a great weekend, share the changelog with some humans who might dig it, and let's talk again real soon.

1:29:38Maybe it's a commentary on our current times that we had Nick with us and we forgot about browsers. Well, it's a bright new dia. Oh, that's what I want to know about. Okay.