Generative AI: Disrupting Software Development

0:00Season 3 of the Built Right Podcast is right around the corner, but we've got one big change coming your way. The Built Right Podcast is now the Talking AI Podcast, and we got a lot to talk about in AI. In the Talking AI Podcast, we'll be having in-depth conversations with both AI experts and early adopters of AI. That way you can understand how the technology works and how early adopters are beginning to implement and more importantly, get value from AI. Our guests range from AI research scientists to founders of AI products to industry leaders putting AI to work in their business. While you're waiting for season three, go ahead and subscribe on your favorite podcast platform so you don't miss an episode.

0:41And make sure to leave us a comment about the AI topics that you want to hear about. So get ready to talk some AI in the new Talking AI Podcast, coming your way August 6th.

0:53Welcome to Built Right, a podcast by Hatchworks where we help you learn how to build the right digital product the right way. In this season, we're going all in on generative AI with guests ranging from international AI speakers, founders of Gen AI products, experts in specific domains of Gen AI, and leaders across industries. We're here to help you figure out how to take advantage of this new emerging technology so you can win in the market. So whether you're an AI techie or just AI curious, we got you covered. Let's get into it. Hey, Bill Wright listeners, we have a special edition of the Bill Wright podcast for you today.

1:29In this episode, you'll get the unique opportunity to hear from Hatchworks CEO, Brandon Powell, and Hatchworks new executive chairman and former CEO of Kenan Carta, Jay Schwant. And they're going to be discussing the future of software development in the age of generative AI. I've listened to this episode and I promise you're in for a treat. These are the type of conversations and insights you just really won't get anywhere else.

1:58Welcome to a special edition of the Built Right Podcast. I'm Brandon Powell, the CEO of Hatchworks, stepping in as your host today. Joining me today is our new executive chairman of Hatchworks, Jay Schwan. Jay has over 25 years of founding and scaling global digital consultancies, including Solstice and Kenoncarta, and now runs the Second Mountain, an early-stage investment fund focused on technology services companies. He's an engineer by trade and absolutely one of the best strategic thinkers I've ever seen. Together, we're going to dig into the massive impact of generative AI on software development, exploring both the challenges and the incredible opportunities it presents.

2:38Welcome to the show, Jay. Thanks, Brandon. Good to talk with you, as always. Great. So I'm really looking forward into digging into today's topic. Hatchworks has been really going all in since generative AI came out. And we would love to understand from you what you're seeing and what is it about generative AI coming from an engineer's perspective that you see as such a disruption to the market? Look, by no means am I the most astute technical expert on this topic, but I'll kind of break it down into, I guess, the layman's way that I think about it. I started my career kind of as the kind of the internet cycle was peaking and a lot of the infrastructure was getting built out and platforms were emerging.

3:29And I kind of started really career on some of those early web 1.0 kind of technology stacks and, you know, just saw how transformative that was. And then kind of midway through building Solstice, you know the mobile wave hit and it followed a very similar platform you know you have kind of hardware infrastructure that got built out and in that case it was really the mobile networks was the hardware infrastructure that got built out for mobile or that was needed and then these platforms kind of merged on top of them you know things like the app store for you know for the iPhone for example and then apps all of a sudden started to get created on top of those platforms And then like, I think what I would consider be like the true creative resources started to be developed, meaning like mobile native applications or applications that really only could exist on that platform, you know, because they were whatever location aware, contextually aware.

4:25They used all the capabilities of these of these new platforms that, you know, you couldn't traditionally do on the Internet. So if you think about what's going on today, it's following a very similar cycle. And we are in the hardware kind of infrastructure building phase. That's why you see like NVIDIA and all these chip makers. And it's like, you know, they're making tons of money because everyone's buying, building out their data centers and building out their AI capabilities within the data centers and the hyperscalers building them out in the cloud. So it's like foundational. Like this is just the beginning.

4:58And I think that's what's really exciting. We haven't really seen platforms emerge yet. I mean, there's some that are kind of emerging out there, but most of them are kind of these closed wall APIs within some of these different models. But true standardization and open platforms that everyone can say, yeah, this is how you build generative apps. We're still not even there yet. And so super early. And then after that, we'll get apps building both on the platforms. And then we'll get those creative, you know, true like generative AI native use cases that frankly, many of us haven't even considered or thought of yet.

5:39And so that all is to come. And that's to me is what's so exciting about this. I mean, it's just, it's, it's so similar to what, you know, what we saw on the internet and mobile days that, you know, and that's why there's just so much, I think, um, fanaticism and kind of franticness right now, because everyone knows it's coming, but, uh, no one quite knows what it's going to look like yet. Quick break in the pod. If you're listening to this podcast, chances are you've been thinking about how to actually use AI inside your business. And that's exactly why we built the AI Opportunity Finder. It's a free tool that helps you uncover high impact, tailored AI use cases based on your business, your goals, your pain points, and your industry.

6:21No fluff, no generic use cases, just real ideas that fit your business and the ranked by ROI potential. It takes about three minutes to run and it's like having your own personal AI strategist for free. If you want to try it for free, check out the link in the show notes or go to hatchworks.com backslash AI dash opportunity dash finder. Yeah, and maybe this is the million dollar question, but I think you're seeing the parallels between how you navigated Solstice into mobile, super successful at scaling there. And you're saying, right now we're more in that hardware slash infrastructure, everything's there.

6:58So the ingredients are all there. Now our next step is the platform phase and the applications on top of that. Do you have any predictions or some early winners that you see there in that platform stage now that we're kind of starting to move into that area? In terms of emerging platforms, I think it's super early. I think, you know, I think the I think open source is going to win. So I think the models, the closed models are going to struggle to keep up with with the open source frameworks. and Meta's kind of committed to open sourcing the models that they're creating

7:44and them demonstrating that they can kind of keep pace with these closed models coming out of like OpenAI and Anthropic, for example, kind of shows that the models themselves are going to get commoditized. So the platforms in which those models run, I think those are just getting built out now. Like who's going to win there? it's hard to say. Every hyperspace is going to have their own solution to that. I think there's going to be an open source platform that most likely emerges. You're seeing things like Pugging Base and Langchain offer different capabilities and marketplaces that are a bit more open.

8:24I think those are definitely things to watch. But where are we going to be building these kind of generative native apps? I honestly don't think that the final resting place has been determined. The final standard has been determined yet. I don't think anyone's actually solved that. I think we'll probably see some early signs of it in 20, in later this year, like, and it'll start to become a little bit clearer with how fast everything's moving. But I, you know, I don't, I don't think anyone can claim to be winning there quite yet. No, I completely agree. I thought the Meadow one was just the most interesting because if you think about Microsoft and Amazon and their partnerships, their investment, they're all about driving usage on their infrastructure layer.

9:13That's essentially like, I want you in my infrastructure and drive uses. And many of these models are very expensive that came out there, that came out to market first. I thought the meta was just really interesting. When you come to the game with a completely free open source model, it really kind of puts things a little bit in check. And also you think about Meta, they're not an infrastructure company. So what's driving them in that space? Data, input. It's the ability to get more data into their platform and product. And I just think it's really compelling right now, the different strategies.

9:45But I'm with you that I think the LLMs all end up being a kind of a commodity, like choosing your infrastructure type. But they've all got these different strategies around why they're doing it. And it'll be really interesting to see who wins. But shifting gears a little bit, as we look at software engineer, right? So we've been in the software development space for over seven years now. And we've seen some decent pivots, right? We went from on-prem to cloud. And we started putting DevOps and automation, all these pieces into software. But coming from that engineering background, people are always saying that this is the area that's going to have a huge amount of impact in terms of developers and actually creating code.

10:27Why do you think that is? I think it's getting a lot of attention right now because as software engineers, most of what we do is text-based. And so, you know, the generative, the most capable models right now are outputting text. They're taking in a lot of text and they're outputting text. Um, and so fields that, um, you know, are input, you know, take a lot of text as input professional fields or careers that take a lot of text as input and then produce a lot of text as output. I mean, those are the ones that are, are, um, going to be able to take advantage of this technology first. So things like writing and journalism, things like legal, you know, and contracts, you know, things like finance and, and software.

11:16So I, you know, I think it's more about, um, greatly increasing the capabilities of one individual with this technology. I don't think writers are going to go away. I don't think lawyers are going to go away, you know, fortunately, no kidding. Uh, finance people aren't going to go away. Software engineers not going to go away. Um, yeah, I, I, I mean, I remember dating myself a bit, but like my first project out of college, I was working at a mutual fund company and I was coding in VI, which is a green screen based text editor. That's how we were programmed, um, for this particular project. And, um, like IDEs, like integrated development environments, which is, you know, kind of like the windows based applications that people use to write code and introduce debuggers and things like that those were just kind of starting to get um get traction and as they were coming out with these idees people were like oh we're gonna need we're not gonna need as many software engineers because now these idees are gonna make it so much easier to find bugs and to write code quickly and it's got all this autocomplete capabilities and blah blah blah and you know i look back to the 90s versus today and there's probably a thousand times as many software engineers as there was back then um so you know what i think's probably going to happen here is we're just going to greatly increase the capacity or productivity of these different kind of text-based disciplines including software engineers which is going to make it much cheaper to write software which is going to allow a lot more software to be written.

12:59Things that were not financially viable, you know, even three or four years ago to do are now going to be financially viable from a project perspective or a capital allocation perspective. And that's incredibly interesting. Yeah. So in your, and this is the kind of a, there's, there's two sides to this, right? Is once some people believe, okay, oh, software development, you know, that's going to kind of go away. There's other people that's like, yeah, I'm in the same quarter as you that yet, I think it's going to drive better productivity, better efficiency. You still have to understand how to code, how the application is built, but it's going to draw more people into the pool.

13:36The pool being more people that are willing to spend on building software and building applications because you're essentially, from my perspective, at least lowering the barrier to entry. And now you've got access where it isn't going to cost you the same to build or maintain or run that application. I get asked that all the time because I feel like there's a lot of noise in the market. You know, no, don't let your kids go be a developer. They need to go through this these days. And so I think speaking directly to kind of that developer that's out there in that community, do you believe that Gen.AI will replace the role of the software engineer at some point?

14:12Yeah, I don't. I think the role is going to change dramatically, just as it has changed dramatically over the last 70 years. Um, but you know, I don't, I don't think it's going to eliminate it because at the end of the day, what software engineers do is, you know, they, they kind of sit there, like the human interface that exists between a business problem and the machine that solves it. And, um, I, I think there's a real value, like value exchange that happens there. There's a lot of considerations in terms of like how things get done. So I think like a lot of the lower level tasks are going to get automated.

14:56They're going to become easier just as they have been getting automated and getting easier over the last, you know, 20 years of IDEs and all these other capabilities. but the foundational element of having an engineer, a true problem solver, someone that knows how to take the science and apply it to a specific world or business problem, that role is still going to be necessary. And I also think there's got to be new roles. Just as back in the mainframe days, there was one role. It was a mainframe of all. That was it. Right. And, you know, and as like architecture has got more complicated and, you know, you started getting tiers of capabilities like data tiers and server side or middleware tiers and front end tiers.

15:47And you started to get these specializations at these different tiers. So I do think new specializations are going to emerge coming out of this, you know, new type of software system, new generative software system. I don't think it's going to be prompt engineers. I think that's, it reminds me of like when, when Google started getting popular, you can go buy a book on like how to Google, like, like it was, you know, like you could use these like symbols for ands and ors and stuff. And people, I mean, it just reminds me of that. I think I, I, I don't see that as a discipline as much as, um, hard to seeing kind of the data that exists behind these firewalls, putting them in a model that's efficient, that's purpose built, that's inexpensive, that's fast, you know, that is going to be a discipline and a skill set in and of itself.

16:42And others like it that will emerge that did not exist three or four years ago, in addition to like what we consider to be like a traditional full stack software engineer. Any predictions on what some of those new roles might be? One that I've been, I think I have the wrong name for, but I am so aligned with the fact that engineers solve business problems and the how will shift. I loved how you said it. The how will shift and it's shifted even over the last couple of decades. this idea of like an AI, I know this is the wrong word, like an orchestrator, like kind of like there's all these tools out there.

17:20There's all these different sets that they can make, but they're still got this business problem and they've got to figure out from this toolkit, what are the things that I'm going to use to solve this problem? And it might be a lot less of writing just a bunch of code versus like, I have this I can pull from that guy can pull from, I mean, we have libraries today, but just like what's a role or an example maybe that you see popping up out of this new era? Yeah, I mean, I think it's not too dissimilar. Like, you know, software engineer is going to they access different frameworks and libraries and APIs today to get stuff done.

17:58So add a class of capabilities called agents to that mix. and agents are, you know, are going to be able to do multi-step processes versus like what we consider of just like a method where you put notes, you know, instantiate a class, call a method, put data in, get data out. You know, they may have, agents may have the capabilities of doing multiple things with multiple different types of input over time. But I do think it's going to be similar in that, like, there'll just be a new class of tools available to a software engineer to use for specific types of applications. And so that's why I don't see the role going away.

18:45I just see it getting more powerful because the tools available to it are going to be much more vast and are going to increase the productivity and capabilities tremendously. Yeah. Let's double click on the agent piece of this because that's something I give a lot of thought about it it's somewhat difficult for me coming from more of the product side but like the idea you mentioned that maybe some of these junior these more junior roles go away or these lower level tasks that maybe you're doing offshore doing somewhere differently um you know we saw devon that launch right kind of that idea as a software agent um mixed mixed reviews on the success of dev in there yeah but you like i've spent a lot of time thinking about like if we added you know a team right in today in that of this agile world you put your pot out there and most of those roles you know you've got defined roles but i feel like in the in the future like this idea of having this agent that's part of your team like this is our hatchy this is actually he's part of this team and he or she does these tasks but any perspective or view on the on the role of agents?

19:54Because I'm a big, I believe the agents are going to continue to get built. We're going to get to more of this agents talking to agents and go from there. It's just in this space of software development where it is quite complicated, there is a lot of things that can be automated. But do you have a vision or perspective on the role of agents specifically within software development? Yeah. So I don't think there's going to be like this one super agent that we go to to to to you know ask to do about everything i i you know i'm i'm much more of the mindset that it's going to be more componentized like like we think about frameworks today or libraries that we would include in a software project you're going to have these agents that have a specific set of capabilities that are trained on a specific set of data they understand a specific set of processes and and you know can make decisions and and perform those things so So, and some of these may be publicly available that everybody uses, you know, in the, you know, hyperscalers clouds.

20:57And, and most of them I think are going to be very specific to a company or organization. So, and I think it'll be more modeled after like how you think about roles in an organization today. So, you know, you could have a agent for workday that can do certain HR processes, but more likely you're going to have an HR agent that's very specific to a company and how they enact their HR policies and processes. Cause every company is going to be a little bit different. So you would have these agents that would be performing different roles within an organization. Um, and this is where you're going to get hyper productivity gains because it's not just going to be embedded in, you know, it's like, it'll be everywhere.

21:43And so you'll have all these agents and all these different departments that are very specific to how a company wants to run its business. And the developer will be able to tap into these different agents for different things, depending on what it is that they're trying to do. Onboard a new customer, run some particular types of reporting or fulfill an order today. So not dramatically different to how we think about APIs, but just much more powerful, much more intelligent in terms of their ability to make decisions and much more encapsulated in terms of like running overarching processes and not just, you know, individual kind of calls.

22:22So I think like your point about the engineer as an orchestrator, I would argue they already are orchestrators. They're just now going to be that much more powerful with this new class of tools called agents on top of these other tools that they have had, like libraries and frameworks and APIs, and will continue to use. And I was thinking too that these large hyperscalers, right? You'd pick them. Even like a Salesforce, for example, they're very incentivized to build agents that can build software to build, basically build capacity on their platform, right? The faster you can build an application, the faster you're going to consume.

23:06And so that's one of the thoughts is like, yeah, there'll be these open agents that you can use on an AWS to help you build the application quickly. And so it got me to thinking, it's like, how do you think about organizations? How do they differentiate? Do you think that companies like Hatchworks and other even big organizations that are in a space, do you think that that will build IP into their agents and that'll be one of the ways that they're able to differentiate from their competitors yeah i i don't think it's going to be a whole lot different than it is today in that vein so you know for example like um zappos is known for having phenomenal customer service you know i would argue it's not as great as it used to be i mean it used to be amazing before and not that amazon screwed it up but it's like, you know, but it's a great example though of a company that was just like hyper, hyper, hyper focused on it.

24:05Um, and there, you know, and, and that goes through like everything from, you know, order fulfillment and returns and, you know, and your ability to shop and all these things are recommendations. And, and I think that like, you know, the companies that are very, very focused on, um, providing a fantastic customer experience that goes above and beyond whatever the industry standard is, will invest accordingly. And their agents are going to be very specific to them and much more capable and much more intelligent and offer a source of differentiation for that end customer experience. And so this is why I was talking with the CIO the other day who was kind of just coming up to speed on some of these generative capabilities.

24:57And they were like, you know, we're partners with Salesforce and Microsoft. And I'm just thinking like, they'll give us, they'll give us whatever we need to do, you know, to do this, you know, to use this stuff. Like, you know, I don't need to go build something. I'm fine with just waiting to take whatever they give me. And I won't say what the company was, but looking at the company, I'm like, that makes sense for you. because the experience that you're offering today is not necessarily unique. It's just enough to meet customers' expectations in terms of what they need. And that's not the source of differentiation you're looking to build.

25:42And so some companies, it's going to be fine. Like, hey, whatever the stock capabilities of the year piece system are, that's enough for me. There's going to be other companies that are going to say, you know what? I see an opportunity to differentiate utilizing these next set of technologies, just as they had perhaps with the Internet and mobile and other things. And I am going to build some differentiating capabilities to allow me to really create a distinct, whether it's experience or hyper-efficiency in the back office or whatever their strategic goals or points of differentiation are. And they'll invest accordingly.

26:22I don't think it'll be much different than, you know, you have those digital leaders today. They'll use this as the next platform to do it on. And the ones that are laggards are happy to be laggards or are okay being laggards will continue to. And they'll use, you know, whatever the standard kind of baseline tooling is. No, that's great perspective. What do you think, you know, with AI, right, been around a long time, but what do you think, you know, we're going to have, AI is going to struggle with versus what it's going to, you know, do really well at. I mean, I think, you know, there's a lot of hype out there, obviously, and, but there really is some great, you know, we've seen some really big gains at Hatchworks and some of the things that we've been doing.

27:00We've also seen some things that don't work really well. Um, but, you know, let's think about the longer term, um, game here. Where do you think AI is going to have the most success and where do you think it may never solve the problem that we're looking at? Um, I don't know. I mean, when you ask that question, what are some things that come to your mind in terms of limitations that you're seeing today? Yeah. So I think in general today, there's a cost factor, period. Right. I mean, that's going to come down, but there is definitely a it's a pay to play type of platform. Right. And when you start looking at significant volume on these platforms, it's it gets very expensive.

27:40Um, you also have issues with hallucinations. You've seen stuff in the media out there where, you know, some of these, you know, I don't believe that personally, I don't believe that the, the, the large LLMs are going to be the answer. I think there's going to be, you know, cause small language models, you know, models specific to certain things. Like, it's like, we're, we're getting a Ferrari right now. And that's all we need is to take the bus from one stop to the year. And there's, you know, there's going to be more options out there, but they're all very, very expensive right now in terms of what you actually need to do the job.

28:15And so I think that over time that you're going to get very specific models. You mentioned the Zappos example earlier, there's going to be a model, you know, specifically for retail, and it's a set of things. And then from there, Zappos can go and bring their data to the table and they can customize and train that model to be in some companies or even who have the money and the scale, maybe even go build their own model because they see that as a key differentiator and a special, special sauce for them. But I, you know, I think in general, I think AI can solve a lot of problems. I think coming from my side, more from the products perspective, I'm concerned that engineers will think that they don't actually have to learn how to build software.

29:04So it's not necessarily a technology issue as much as it is like, hey, I actually never knew how to debug software. I've always just depended on AI to do it my whole life. And then you get hallucination and stuff like you get today. You can't take everything as like the answer, though it's 100 % correct. You still need a human to kind of look at it, make sure that makes sense. And I think I saw some data around how long it takes to debug things now, that it's taking longer to debug because I think you have people using AI to build things that don't actually know how to do it. And so they're back and it's taking them longer to figure out the problem.