AI Challenges in Software Development

0:00David Heinemeier Hansson:Welcome to Rework, a podcast by 37signals about the better way to work and run your business. I'm Kimberly Rhodes, joined as always by the co-founders of 37signals, Jason Fried and David Heinemeier Hansson. We have talked about AI a couple times on the podcast, but with the launch of our newest version of Face Camp 5, AI was involved a lot in that development. So I thought we'd talk a little bit about it. I know I saw a few messages poking around on our tech team about AI and things we needed to do to rein it in. And so, David, I'm going to let you kind of kick it off and talk to us a little bit about Basecamp 5 and how much was AI involved in this?

0:36Basecamp 5 was the first fully AI accelerated development process that we've had. The past product that we launched before Basecamp 5 was Fizzy. And Fizzy had very little direct AI generated code in it. It had a little bit around the edges, but the vast majority of that code was written by hand for someone to put that product out. And it's interesting because Fizzi only just launched back in December. But now here we are having launched Basecamp 5 just a few months later. And yet it could not have been more different, certainly towards the end in terms of the role that AI played. It really went from AI is this pair programmer, you'll ask for questions, and it's an easier way to look up APIs and docs and get things explained, to AI is where most features start.

1:28That most things we are now doing with Basecamp and all sorts of other developments starts with a prompt. Not always. Sometimes there are specific features where the developer has a really clear eye on what needs to change, and that doesn't start with AI. But it's less and less, and I'd actually say at this point, it's certainly the majority of fixes and feature upgrades that start with a prompt. And that's just a big change. But more than the change for the developer in terms of their workflow and whether they start with a prompt or whether they start with an editor is the fact that AI allowed our design team to start developing much more of the implementation by themselves as part of the design process, which as a baseline is amazing.

2:17It is absolutely wonderful that AI is enabling designers to see their ideas and their designs literally come to life and therefore being able to interact with them and therefore being able to gauge whether their ideas are any good or not. The problem with traditional design is traditional application design is when you divorce it from how does it flow? How does it work? What does it look like with real data in a production situation? It's just really difficult to land the spot. It's really difficult to get something that's great. But if you allow the designer to get the whole thing working, and not just working, but deployed, and not just deployed, but usable on a beta server, working directly with our real data, you just move so much faster towards, yes, that's it.

3:12That's what I want, which is an enormous part of the whole design process. There's the part that's just I know exactly what I want and I'm just going to design it. That's usually the end stage of things or certainly the rare situation. A lot of the times we start with I want to solve this problem. There's a million different ways I could do it. But to figure out which one is right, I have to start building. I have to start developing and then I'll tweak it and so forth. But what was the norm prior to AI was that the designer would reach a barrier quite early on in the design process where they just couldn't get further.

3:50They just couldn't get this thing actually working, actually usable. So then they'd have to pair up with the programmer. And that process, in some cases, either the designer had to wait, there just wasn't a programmer available right now, or the project that they were working on was actually quite intricate. So it might take a week or two weeks or in some cases even three weeks before they're fully able to evaluate the vision and design that they had. And then in some cases, they figure out that what the developer had just spent several weeks working on was not right and was not what they actually wanted.

4:23Because no one knows what they want until they have it in their hands and can tinker with it and go like, oh, yeah, no, no, no, that's not – that grip is wrong. And I couldn't envision that. I can only feel it. So that process really changed how we started developing features. But it also has some limitations at the moment. And one of those limitations is that AI will literally give you anything you ask for. It's kind of like that monkey hand story where you get three wishes and you ask for a lot of money. And the way you get a lot of money is like a death in the family with someone who had a big insurance policy.

4:59We're like, that's not what I meant. I didn't want a granny to die for me to get a million dollars. I just wanted a million dollars. Some of AI development is a little bit like that. There's a little bit of a monkey hand where you ask it to do something for you, and it'll just do it. It won't say no. And sometimes the correct answer is no. And if you'd had a human developer with you, they would have said no. I would have said no. Jason would have asked for something in a particular way. I'd say like, well, I wouldn't say no. I'm more polite than that. I'd say like, that's probably not the best way to do things.

5:33If we do it this way instead, you can get 90 % of what you want for 10 % of the complexity. And in most cases, it's like, yeah, totally fine. Great. I didn't even care about that the last 10 % anyway. AI is not going to ask you that question. It's not going to push back. It's not going to be offended by having to come up with 700 lines of code when 35 could do. The problem is, if you just keep doing that, if you keep asking the monkey paw for more wishes, you're going to get a distorted code base. At least right now, I mean, we should timestamp this episode because it might very well change by the end of the summer as new, more powerful models come out here.

6:15But as of, what are we, July now, 2026, this is still the state of the art. The state of the art is that the models are incredibly powerful, and they've even gotten quite a lot more powerful just in those last six months that we've been rushing to get to the end of Basecamp 5 and then doing the follow-up work afterwards. But they're still not at a place where if you just have a designer prompt their way through, let's say, 20 features, that you're going to end up with something that has a coherent architecture, that's easy to continue to develop on, that's a pleasant place for humans to work alongside it.

6:51So we had to pull back a little bit a couple of times where the eventual rule became, you should start like this. As a designer or a product manager or anyone with a good idea for a new feature, make it work. Whatever you need to do, just make it work. Run it on a branch. Get it on a beta. Show it to Jason. Show it to Brian. Try it with a larger share of the company. And then if you like what you have, we should not think that that's something you can just hit merge on. You cannot just take that output and then shove it straight into the code base and then expect that all was going to be well.

7:28We did that a couple of times, and we ended up with some technical debt, which is the polite word for a crappy code that needs to be swept up after the fact. And it's always harder to do it after the fact than it is to do it before the fact. So the final rule was if you're making changes to the Ruby code or to the JavaScript code and you're on the design side or a junior programmer, you should just have someone look it over. You should have someone more senior look it over. And when they do look it over, you kind of have to be able to stomach that sometimes the answer is I can't use any of this.

8:02What the AI has produced works, but it would destroy performance. It would puncture the architecture. literature, it would have security issues or any of the other considerations that a skilled programmer might identify with a piece of code is present here. And so what? That's fine. We are no longer attached to this piece of code that has been produced by prompts because, I mean, we didn't write it. There's not a human on the other side. If you tell the AI to, well, you don't tell the AI, you just start a new session, right? And boom, amnesia. They've completely forgotten that they They spend a lot of expensive tokens producing this stuff.

8:40And they're not offended. In fact, if anything, Anthropic and OpenAI, they're going to be like, great, you want to start over? That's another batch of tokens for us to sell you. So when that's the case, when the residual code has very little value, you don't feel as bad about throwing it out. And therefore, when a senior programmer on our team looks at something produced by prompt that isn't right, they can just go like, I'm just going to put it all into trash. And we're going to take the idea you arrived at, the concept, the design, probably the HTML, probably the CSS, and we're going to wrap it around a new implementation.

9:19And that's going to be fine. We've still got to the final destination much faster. And that's the other thing. I mean, this really is a lot faster. We were able to do a lot of things, especially in the last sprint, where I would look at the amount of outstanding work and went like, do you know what? Pre-AI, this just wasn't going to happen. Now, there was a slice of that that was whack-a-mole. There was a slice of that, especially in the phase where we were still trying to figure out how far we can push the prompt. And some designers pushed the prompt very far. I mean, awesome. You got to touch the electric fence to know whether it's actually on.

9:59And it was on, so it gave us a little bit of a shock. but totally fine. They pushed it really far and then we kind of had to pull back a little bit because otherwise you can end up in sort of the prompt loop where you're like, make this happen. Oh, you broke two things. Fix one thing or you broke another three things. And that's just not a pleasant place to be. So even if code is really cheap to produce, quality code still takes a fair bit of effort and it takes some dedicated prompting, some understanding of how the architecture already is and what fits in well and what you can just let the models run wild with and which other parts you kind of have to rein them in a bit.

10:39But I'd still say as a whole, Basecamp 5 is a lot better because we were able to use AI both to figure out what we wanted and then also to produce what we wanted. And then finally, to fix a lot of issues, AI assisted. I mean, this is one of the things we saw after we launched. We talked in the previous episode about some of the performance issues, some of the pathological cases where some customers just have very rare data shapes where like they have thousands of entry in a part of the product where we don't expect that because we very rarely see it and we don't have it in our own account. And you take those kinds of cases and AI is truly incredible, not just at fixing the issue, but at diagnosing it and going through logs and figure out where these outliers and trying to go through some loops where it's rewriting queries or whatever the optimization is and then producing something quantifiable.

11:35And it goes like, here's the thing you had. Here's a new thing I made. And it's now 72 % faster. And it's kind of that labor intensive, almost scientific work that AI just excels at. And then still on, I want you to write me a new feature that requires a fair amount of Ruby and a fair amount of JavaScript. The reins have to be held a little tighter and it's still possible for it to get off into the weeds. Again, as I said, we weren't even there. Like for the initial part of Basecamp 5's development, this started, Jason, when was the first dig? Like spring of last year, I think?

12:13David Heinemeier Hansson:We talked about it at the previous meetup. Yeah, probably about over a year ago. Yeah, so about a year ago is how long we were working at Basecamp 5. That initial phase didn't have any of this. It didn't have any of this AI acceleration. That really only started to hit around December-ish. That was when the affliction point was. And then we had maybe half of development phase that was AI accelerated. So it was a really interesting moment where you could see both of these things. And I'd say finally, there are qualities with the slower handwritten approach that we not just appreciate but actually miss.

12:50There's something about the interaction when a developer and a designer is working closely together for a longer period of time on a feature where occasionally you end up with something that's better. And that's just the reality of things. If you take three weeks to slow cook something and spend all your attention on it, it's not all mechanical. Some of it is breakthroughs in concept that you're not going to find if you only spend two days on something. So that's the tradeoff. I still think it's very worth the trade-off. You just got to know that there is a trade-off. And then you also got to know that this acceleration, the ability to just go through long lists of things that are nice to have is also kind of dangerous.

13:38And I don't know if the industry at large have fully appreciated the devil's bargain. It is to be able to get whatever you want, however much you want. because a lot of products don't always improve. I think that's a polite way of saying it when they add two times as many features, three times as many features. So what happens to product management when suddenly there's not this great constraint of I only have so many programmers, they can only work so many hours and therefore they can only produce so many features. What if you suddenly double that, triple that, 10X that? Do you actually trust yourself and your product managers to say like, no, we're not going to do these things for a greater order.

14:21I think even here at 30 Timeless Inglets, where we've prided ourselves on less software and fewer features and so forth, it's a real challenge of figuring out what to say no to and not let your appetite sort of just gorge itself on endless streams of features until your product comes out completely bloated.

14:42David Heinemeier Hansson:Speaking of that, really quick, just to add, I was just looking at our card table in Basecamp five. We have a card table called cycle three, which is the current cycle of work we're on. And there's about 20 things that we have listed as things we might be working on over the next six weeks. That would normally be maybe six things like a year ago. And now there's like up to 20 things. Now, some of these are much smaller and quicker and not a big deal, but still we wouldn't have had this much stuff on the menu to choose from a year ago. So to David's point, like, I think at this stage, we're making a lot of quick, subtle changes because of feedback we're getting and things we're seeing ourselves.

15:21David Heinemeier Hansson:So there's going to be probably more on the plate right now than there will be in six months. But still, it is easy to just keep doing more and more stuff. And, you know, if you think about the ground has just shifted, something I'm conscious of, and I was talking to Brian about it, it's like, we've already upset enough people, like, with change. And I think many of those people will come around. I'm not, like, worried about permanently upsetting them. But like at the moment, we've upset a lot of people. Let's make sure we don't keep shifting heavy bits of ground under their feet. Like we can make some subtle changes.

15:51David Heinemeier Hansson:We've already made a couple, given them a big change in the home screen and the activity screen. Like that's actually enough big change right now. So let's shift into smaller change for a while. Stuff they may not even see, but might appreciate because things are getting a bit more dialed in. So it's not just about quantifying the amount of change. It's about qualifying the size of change, the scope of these changes, and understanding when to roll those out. And you don't want to keep rolling out massive continental shifts basically all the time. And I think that requirement of discipline is going to be very difficult going forward.

16:25And I think there's some opportunity here that when it becomes easier and easier to make a bigger and bigger product, I'd like to believe that the distilled product, the carefully curated product, the product that doesn't do everything for everyone in all different shapes and sizes is going to gain more value. And it's not exactly easy to figure that out because there's also pressure on the low end that if your product is really simple, even if it's well-made, AI is going to be able to eat that quite quickly, right? There's some space in between the gigantuous behemoth that does everything and the small little feature that could be created by AI in a snap of a finger where there's a magic space that exists.

17:11And I think that's the space that remains our big challenge as a company, as someone who's been making Basecamp for a very long time. How do we make sure that we distill the essence of project collaboration to such a form that it remains humanly understandable, humanly learnable, humanly adoptable without requiring big manuals, without requiring big tours? As Jason mentioned in another episode, sometimes it's hard to even get people to spend five minutes to watch an introduction video to a new version of the product. Like, good luck trying to get them to sit through an hour or read 100 pages. Like, that's just not going to happen.

17:55So the natural constraint on how big can your product be in these kinds of consumer-adjacent spaces is going to be the human capacity for picking something up. And that's, I mean, historically, always been one of our fortes, that Basecamp was easier to learn, easier to adopt than other softwares in our industry because it was just right. Less to some degree, but also just more approachable. And we have to be careful not to lose that as we gorge ourselves on all these new productivity gains.

18:32David Heinemeier Hansson:And on the front end of that, we've been very careful about that on the marketing side as well. I invite everybody to go to Basecamp.com and open a few different browser tabs. Let's talk about other products in our industry. You can go to ClickUp.com, Monday.com, Notion.com, Asana.com. If you look at all of those products, how they're presented today, it's AI first. It's AI is everything. It's all there is is AI. And I will tell you, having just interacted with literally thousands of customers over the past handful of weeks here, a lot of people don't want that. They're not interested in that right now.

19:11David Heinemeier Hansson:It's not that AI isn't useful in their life, but to lead with that, to be the primary touch point, to be the front door for everything is not what a lot of people want. Now, there might be plenty of people online who want that, but there's a lot of people in the world who do not want that. And it's extremely intimidating for them. And I'm not saying that they're behind either. Like they just don't, their work is not that. That's not what they need out of a tool. And so we're very careful about making sure that we're an option that is modern and you can bring your agents to Basecamp if you want.

19:45David Heinemeier Hansson:We have agents working in our Basecamp and you can do all those things. But we're also saying like, this is straightforward and easy to use. These are words that have been lost. This used to be how everyone actually talked about their product, easy to use. Like we still have that in our homepage. I actually don't think I've seen that phrase kind of anywhere else in the last few years. It's sort of been forgotten. And I think for a lot of people, it's still preeminently the most important thing for them. Can I use this thing? Can I get other people to use this thing with me? Am I going to understand this thing?

20:14David Heinemeier Hansson:If I have to explain this to someone else who's not a technical person, are they going to get it? This is very, very important. And I would encourage people who are listening to not just jump on the AI bandwagon and going, every customer is looking for AI this, AI that. Like there are plenty of people who are not and big, huge, huge, massive groups of customers who do not want that as the lead feature. It's not all that everywhere. And if you talk to people, you'll find that they are not all that everywhere all the time. So keep that in mind as well. I think this is what's so hard about our industry, because if you are a developer, a designer, anyone who's making software, you will have heard almost about nothing but AI for the past two years.

20:57And therefore, it's very difficult not to let that soak into every decision you make about everything, including how you position your product and how you build for your product. That AI is such a tidal wave that it seemingly just washes all these other considerations away, including like, can I get humans to understand this product that we're making and selling? And yeah, it's not easy because we can't just dismiss this as it's just a fad, right? Like the AI revolution that's just coming underway now. I mean, that's the other thing that's important to remember. Literally, when we were just talking about how AI was being used to make Basecamp, we're talking about what happened the last six months, the last nine months, the last year at the very most.

21:45This is extremely recent stuff to be able to use it for these kind of productivity gains. And first of all, even when you have things that are getting adopted very quickly, like AI, it doesn't get adopted. it universally within six months or 12 months. In some cases, it doesn't even fit at all. But it is still the most important tidal wave that has come from our industry, quite possibly bigger than the internet. And that was the tidal wave that we have been riding for the past quarter of a century with Basecamp. It's this weird in-between place where on the one hand, I think if you are a software developer and you've decided that AI is just not for you at all, you don't care about it, it's all just a fad, it's all just hype, I don't think that's a good place to be.

22:31And I'm saying that with as much empathy as I can because, I mean, I occasionally also read just the 900th breathless opinion piece on the latest model and just go like, Jesus, would you shut up for just a second about this stuff? I mean, first of all, you were raving and ranting about just this other thing three days ago. Like my brain simply can't take that amount of just breathless boosterism. And I then put that in a little jar and it goes over there. And then there's also just the magnificent daily encounters with these models where I just go like, I cannot believe we're making computers do this.

23:12It is so exciting. It is so novel that we can just talk to the computer and it produces amazing software. Not always, but increasingly persistently good software. And you have to be able to hold those things in your head at the same time that there's a tremendous amount of hype. And some of the hype is ahead of where the market is right now and certainly ahead of where lots of customers are. And then there's also the other part of the equation that is this is absolutely real. This is not a fad. This is as strong of a signal as I've gotten from our industry literally since the internet. And I remember the internet very fondly and very vividly.

23:58And I also remember in like 95, 96, 97, how many people were just determined to tune out all the hype there was for the internet in those days. and go like, this is nothing more than an advanced fax machine. It's not going to have a major impact on our society or our economy or whatever. Just tune it out. And clearly that was just a category error, fundamental misunderstanding of technology. So I think I'm trying to sit there. We're trying to sit there as a company, trying to embrace the enthusiasm, embrace the wonder of computers doing these things. I think if you are a person who's a designer of software, a programmer of software, You should be excited about computers.

24:40Like computers, fundamentally, the idea that you can make machines do these things should be appealing, should be exciting, should be just invigorating. So soak that in, sit and enjoy that, and then also go like, yeah. And then now I'm also going to make software that's just not just for me, my industry, my super early adopters and that part of the curve. We're also making people or a bunch of software for totally normal people whose day and life and work has not just been completely rewritten overnight by AI.

25:12David Heinemeier Hansson:Okay, last question before we wrap it up. David, you mentioned tokens earlier. And I've heard stories about companies unknowingly spending millions of dollars on AI development. Tell me how we're managing that. It might be none of my business, but I'm curious. Like, are we looking at our token spend? Is there a budget for tokens? Like, how does that work in this small company who is framed on always staying profitable? Yeah, so it's actually interesting. We're in a really good spot on that as of yet. Now, these plans may change and so forth, but we exist just far enough below the enterprise tier where we're not actually buying individual tokens.

25:50We're buying these subscription plans that the providers of the frontier models are offering. And they can be expensive, as in like hundreds of dollars per person per month, but they're not to the tune of some of the spends that I've seen or heard about from large enterprises that just buy tokens. I was going to say in bulk, but that's not actually what they do. They buy them a piece. And those tokens really can add up. I've heard cases where individual developers have been able to spend as much as a quarter of a million over a one-year run rate. And obviously, well, I don't know, obviously, maybe not obviously.