20Product: Is an AI Winter Approaching | The Future of AI Software Development: What is Real & What is BS | OpenAI: WTF is Going on & How Far Into Application Layer Do They Go | The Future Role of Software Engineers with Guy Podjarny

0:00I think that's bullshit. I don't think that's true for a variety of reasons. First, SaaS businesses are far more than just the software that they create. In fact, if you have a SaaS business and your only differentiation is I've written all this code and nobody else can do, then you're probably your days are numbered. Expectation of when would the GPT -5 scale models arrive? I now think that those are further away. You've seen Sam Altman and others change their tunes to talk about reasoning and things like that as the progress. and my sense is there are actually some architectural problems so it's possible there's a year or two in which these things are stuck.

0:33This is 20 product with me Harry Steppings. Now stay we are joined by one of the best product leaders. He founded and scaled Sneak to being a 7 .4 billion dollar company. Now he's back with his new company Tessel which he's already raised 125 million for at a 750 million dollar valuation from the likes of GV, Index and Boldstart. But before we dive into the show today, how do you get your users to do what you want them to do? Well that's where Pendo comes in. Pendo is the only all -in -one product experience platform for any type of application. Honestly, Pendo's differentiation is in its platform.

1:09Every capability from analytics to in -up, guidance to session replay, mobile feedback management and road mapping are all purpose -built to work together. But don't take my word for it. 10 ,000 companies use Pando and Pando also does mine the product. The world's largest community of product management professionals. So check out their amazing free product today on Pando .io forward slash 20 product that's Pando .io forward slash 20 product hyphen podcast. And talking about AI, Glean is the leading work AI platform that transforms how teams operate. So this is how it does it. AI has the potential to help everyone at work with nearly everything they do.

1:48But only if it deeply understands the data, the people, the processes, and the context specific to your work. Well Gleene is the leading work AI platform that does exactly that. So Gleene's powerful AI assistant helps you find answers, generate content and automate work. You can also build no code and no code custom generative AI apps and agents that scale the power of your company's knowledge anywhere it's needed. That's why hundreds of enterprises, including some of the world's leading technology, telecom, retail manufacturing banking businesses, put AI to work with Gleene. Empowering engineers to ship code faster, sales reps to better understand their customers, support agents to resolve cases more accurately, even IT teams to deflect questions and solve issues automatically.

2:33Gleene is work AI for all. Visit clean .com -4 -20 -20 to request a demo today. You have now arrived at your destination. Guy, I am so excited for this, And I always learn so much from our conversation. So thank you so much for joining me today. Oh, thanks for having me back on. Clearly, I said a few interesting things last time if you were to do it. I always learn from ours. Now, I've got a little bit spicier. And I want to start with like a quick fire on what people have said and how we think about it. Now, Massa -Sun said, Invidia is undervalued today. Agreed disagree, and why? I think there are really three questions in one in that one.

3:10The first one is the market and VDAN is going to continue to grow. I think that is absolute yes. The semiconductors for AI know that it's going to grow. There are going to be a lot more players in that space. I think the second question is how much would an VDA capture of that market? I think they are going to continue to dominate. I don't know percentage wise, but I think they're doing some brilliant things like taking an advantage of their momentary lead. Not that momentary. They have quite a substantial hard to capture lead, but they know that the clouds, for instance, have distribution advantages.

3:45So they're building a cloud. They're using their semiconductor advantage. So I think they will continue to be like the market leader by a margin for a long time. I think the third question is, what do you think about, what is it now? 35X multiple on these revenues. That was a bit trickier because it really isn't a question of would they grow? The question is, should you put that money into other stock and would they grow faster? That was a bit harder for me to answer. The only one I have a question on there is actually the market itself, which I think everyone would actually uniformly be surprised at me questioning.

4:14Everyone seems to also acknowledge that actually we're going to go through this trough of disillusionment, that companies will realize that actually the RO and a lot of these AI tools hasn't proven out in this first batch. And actually there's going to be this kind of depression, so to speak, in the AI world, which will lead to a reduction in demand for Nvidia chips potentially in the next year. Well, I think, first of all, they sold a lot of commitments already. And so I think a good amount of their revenue is pretty guaranteed. I also think that the core technology here compounds. It keeps evolving and becoming better and better.

4:46So it's quite likely that they're also the ones best able to produce things in the cheapest fashion or that achieve output with least compute or other types of savings. It's not just about being able to deal with the biggest and biggest models, which is a part of it as well. So I think those are just between the manufacturing, the IP, the processes to be able to deal with that, Kuda and the whole development environment on top of it, all of those are pretty durable advantages. Do you think we will go through this trough of disillusionment with regards to enterprises thinking that actually it hasn't delivered the value that it wants to claim to you?

5:19I think so, but it's not because AI is not going to be as promising as it is before, it's just because the numbers are a little bonkers at the moment. The numbers of bonkers or the timing with which the value should be created as bonkers? Well, I think a lot of AI budgets at the moment are non -resilient, right? They're coming in, people are spending money, they're trying things out, and they expect a lot from them. And I think in the long run, that would be correct. But it's hard to think that people will adapt the processes, that companies, that, you know, the way that would work, will adapt quickly enough to really return something in the periods of time.

5:54And so there will be some amazing winners, But yeah, I think a lot will go sideways. I think the biggest one also is all of these tiny startups where there's like a thousand I think in some cases there might be even 10 ,000 companies doing the exact same thing So it almost doesn't matter how good they are and how good the use case is a lot of that money is just that redundant You know a lot of these companies are building the same thing that would surely you know a lot of that is waste I cannot tell you how many note takers for wealth managers for dentists for door I'm literally like, oh fuck, say not again.

6:24Okay, staying on mass, he said the cumulative cost to achieve AGI is 9 trillion in capets, but the benefit would be a shift in GDP to 9 trillion per year. Agree or disagree? Yeah, the investment would be substantial, but I think people underestimate the non -technology parts of adopting AI. I think AI is a transformative or a disruptive technology, not a sustaining one, and it requires people to change, it requires accountability to change, insurance and things like that. And so there will be a lot of delays on how the society accept these things. I mean, the holdback for adopting an AI lawyer wouldn't be the tech.

7:04It would be the law, right? Or insurance about what happens if that lawyer kind of gave you the wrong advice. Who do you sue? Same we're seeing this with self -driving cars, right? It takes forever. Yeah, the tech is a part of it, but really a lot of it is society. And so I think there's a lot of investment, But I think there's also a time horizon. So I think the volume is probably about right? I know it's hard to assess. Is it nine trillion or 15 trillion or three trillion? It's like very large amounts of money. I think the end result would be very much worth it. The time horizon is much harder to assess, not because of the tech evolution.

7:35Larry Allison said, to enter the front -end model race, you need a hundred billion dollars. Do you agree? Do you think we're seeing the democratization of cost and that race? Yeah, I don't think we're seeing democratization of it. Of course, when it comes to coming to the models, generally, I agree. I think you need vast quantities or vast amounts of capital to be able to properly compete in the foundation models. There are kind of two competing theories, though. One is that scaling laws will continue to show themselves. As a result of that, you'd need more and more money to make progress. The general models, the generic global models, they will continue to grow when they will eat everybody's breakfast.

8:12The counter argument is that you would actually really benefit from specialized models. And so if the big guns are really trying to get the massive models and they're trying to scale and they fail Which there are also some rumors around how the latest sort of GPT -5 and equivalent in Gemini and Anthropic know that are failing with some test But if you if that is correct and they spend a lot of money and time on that then the ones that are building You know, maybe code specific or robotic specific models of it They may have a shot at being able to train with relatively small amounts I still think that's a short live though if you if you take a five -year a 10 -year time horizon Those are just like maybe you get some gain for a few years, but over time, capital plays a huge role here.

8:51So you would fundamentally say in the short term we'll have specialized models which are better for efficiency and accuracy, but it will lead to longer -term, more generalized models. I think I am more willing to bet that the specialized models will prove out in the sort of call it three -year timeline than I did before, just because I think the big generic models are apparently, and this is again a bit of rumors, they are running into some limitations on it. Did you contemplate building your model with Tassel? Not very long, I think fundamentally, you also want to tap into the innovation of the market.

9:22You see now on Throbic and OpenAI, some of them might be better at reasoning, which software development might be important when you design the piece of software. On Throbic is currently perceived as better at the actual code generation piece of it. Other models might be better at visuals. And so why choose? I definitely don't want to compete with all of those, but I also don't even want to pick a mid -stem. I want to be able to use the best technology available to me. at any time. If you draw the cloud analogy, I want to be the best user of the cloud as opposed to actually trying to compete with the cloud players.

9:50I want to move into that kind of sub -statement beneath it. There is a huge amount of AI dev tools. What I mean, we spoke about kind of note takers. There's a lot of dev tools. Benioff came out fighting against co -pilot. He said, it just doesn't work. And it doesn't deliver any level of accuracy. Gartner says it's spilling data everywhere and customers are cleaning up the mess. We've got Benioff coming on in a couple of weeks, so that'll be fun. Is Benny of justified in his criticism of Copilot guy? Well, I think GitHub Copilot and rather coding assistance as a whole cursor and GitHub Copilot, it's hard to say that they are not providing value when they are in a place in which a lot of developers say, this is now the way I develop software.

10:30I don't want to go back to not having them. I do think that it develops software of questionable quality in many places because you're just kind of reviewing the code that get generated as opposed to actually writing it. and so just naturally they might have thought you put into it is far lesser. And as a result of that, you don't notice anything. And I think like LLMS as a whole, they kind of average everything, right? Like they generate things that unless you give them very specific instructions, the code to get generated is quite average. And most code is totally fine being average, but it is a bit of a concern that it's duplicating a lot of code that is average.

11:05Why does everyone love cursor? When you think about AI solutions as a whole, if I think kind of broader mental framework, the solutions that are easy to adopt right now are the ones that really don't require any change in how you work today and they just provide you with some magic and they don't really need you to trust the result they just need to work often enough and so coding assistants and cursor specifically are really really nice and cursor has taking that further in Hey, I'm gonna make changes in various places in the code and things like that in the multiple files and they just make it very easy very easy to eyeball to say this is correct, this is not correct.

11:39You're not worried about them doing something incorrect. It's in your flow. You're just sort of coding away. We mentioned kind of the proliferation of AI dev tools. And just how many they are. Are they any good? I think they are providing value in very specific places. And so where they've been helpful is reducing toil. Is when you are trying to do a toil is this notion of just repeated work that you have to do to decorate and sort of describe your code of our time. So creating the commutation, creating tests, they help because they kind of like a template for your email, right, like you, instead of needing to start from scratch, they give you something to start with.

12:16And those have been quite helpful. The code completion piece because of that sort of low cost of verification, they've also been helpful. Beyond that, they haven't been dramatic yet, mostly because they are still unreliable. Our founding engineer calls it the jagged edge of AI. You know, it is amazing at some point and then it bombs it like a complete disaster the second and so it's very hard to Rear arms predictable. They're not there's sort of two types of mistakes that might happen There's the ones that humans might make and that AI sometimes makes so maybe you never will but humans are are kind of quite forgiving on those And then there's mistakes that AI will make the humans will never make and that makes people mad that makes them like viscerally annoyed if there's a leaf in the middle of the road and the car thinks that that's you know whatever some creature and it's not willing to move people get upset they get annoyed by that and so I think in code you see this as well like when there's sort of small bugs, small mistakes that they can imagine themselves doing it and that's like finding I'm gonna fix it but then sometimes you're gonna come across as like what is this and what is a lot of people today say to me oh we can actually replicate the majority of SaaS companies and actually service -tight and it's not that complex a product we can and just build it in seconds with creation tools and spin it up.

13:30I think that's bullshit. I don't think that's true for a variety of reasons. First, SaaS businesses are far more than just the software that they create. In fact, if you have a SaaS business and you're only differentiation is I've written all this code and nobody else can do, then you're probably, your day's a numbered. So you might have data, you might have distribution, you might have specific switching costs as you've built in, that's when you're already successful. as part of relationships, customer relationships. I think there's just a lot more to the machine that is a SaaS company and specifically a SaaS service over time, which is a lot more than just a code to think that you can just replicate a SaaS business because you can tell an agent what it would do, I think, is silly.

14:10What is a genetic development? How does that differ from AI dev tools? So it depends on at what level do you sort of delegate a task to the machine? Let's say you want whatever, and a shoe e -commerce shop, right? Do you just basically tell that to the LLM and you trust the LLM to go do the product research, explore it, figure out what are the types of shoes to handle in there, build the application, verify that the application behaves correctly, do all of that process, and you're just this slightly less sophisticated customer interacting with the system on it. That's the extreme of an agentic system.

14:46There are a lot of decisions there that you don't even know what are the sets of steps that need to happen for that application to be created. You're just giving the instruction on it. Maybe you get a bit more specific. Companies like Cognition with Devon are trying to do that. I think Magic Dev, they're trying to build it into the model and things like that. I don't know, they're quite secretive, so maybe I'm misrepresenting them. They're trying to build this whole process. I think that's different than most sort of AI Dev tools today that try to give you a bit more control. So they're still assuming that there is a software developer involved and they give you kind of more tools to be able to create with them.

15:23There is a bit of a chasm in between. You can go to Anthropics, Artifacts, or to the new OpenAI interfaces, and just say, hey, create an app. And for small apps, it's kind of like agent behavior, but it's very tiny. Then the question is, how do you break through to something bigger? Do you build a new software development methodology to be able to say, okay, let's work together with the machine to define what is being created? That's kind of more the workflow approach, frankly, more the Tesla path. And then there's the outsourced lens, which I think a gentick tends to be embrace the chaos a bit or embrace the customer perspective.

15:55I've seen that be quite a lot of skepticism. I think it was towards cognition and their abilities. Is that fair? Well, first of all, they slightly overplay their hand with the initial video of it. So it's very easy to create amazing videos in the eye, amazing demos. There's a big, big gap between that and creating a reliably working product, hence the jagged edge of the eye. I think most of the flack that they got was that they just overrepresented what they're doing from everybody that I Talked to them has tried to them and they all think it's really cool and that it doesn't really work When I look at magic I look at cognition honestly even a cuttassel You guys are so much money so early Do you need to or is this not just peaking the easy asm from a venture crowd that has too much money?

16:39Too fold you know one part of it and I think that's true for magic for sure and I think that's true for a cognition is that they use a big chunk of that cash for GPUs and sort of, you know, training models on it. I can tell you that it's a, that's not our path. We talked about not wanting to train our foundation model. Yeah, I still costly, you know, before coming here, the team has run a quick evaluation cost them a thousand bucks. You know, they, because you need to run statistical evaluations and it does take more money. That said, you know, I think a lot of these are go big or go home type propositions.

17:10So when you're trying to say, hey, this will either flop or it would change the world, then you want to have good reserves so you'd be able to run the distance. But also, you're trying to be ahead and right now these systems require a lot of iterations to get it right. It definitely reduces optionality when you raise a lot for me that wasn't an issue and I am very much not looking to make some quick $1 ,2 billion exit. Now we spoke about some of these products from Magic to Cognition to many of the others before. A lot of these platforms are pretty closed off gardens, magic boxes, whatever you want to call it.

17:45How do you think about open versus closed in the future of software development? Yes, I'm actually quite worried about it. So, closed environments, closed development platforms, closed ecosystems like in other big platforms. When they add AI, it's a little bit easier to think about how to make it useful, how to live within that. And eventually they create these magic boxes, and you have no ability to interact with it. That's a concern, right? If the web becomes two, two, three, four companies in the world that have these very, very powerful large compute models to which you can give an instruction and get a result.

18:17You really have no ability to build tools that would plug into them or modify some things out. So why would that ever be the case? Like if you look at like an open AI, for example, they are clearly doing a platform play where they are not expected to build all the different tools around it. They are fully trying to develop an ecosystem and it will be multipot participant who service very different fashion as a customer base. I think the platform themselves actually want to be the LLM foundation models actually want to be platforms because they want everybody to build on top of them. So I don't know how deep they will go into the application layer.

18:51I think that's a debate for them. You definitely see OpenEI getting into the application layer. The question is how far do they go? I think the next layer of the big companies are the ones that are more concerned about. Whether you think about the cognition, they're not doing that yet, right? Like there's a lot of the beginning of it, right? But if that model works and you just give it the instruction and that succeeds and again, they're building great stuff What happens next you're building these platforms and you're just giving them the instruction and they go off and they they build it out How does the rest of the tooling ecosystem?

19:20Plug into that if the core of software creation becomes dependent on this one magical understanding of your code of the applications Of all the domains and you just become a customer to that system I worry that we go into a place in which there are a small number of players that have that broad capacity and that the rest of the dev tooling ecosystem becomes minor and delegated versus today's world where we have many thriving developer tooling companies. How likely is that the concentration of these platforms being with the sole providers or kind of few providers to really dominate the space? Is that like a 10 % like a 50 %?

19:57There is a pretty high chance that we're going towards this path. Ease of use of software development is already amazing and you're seeing platforms like a Versel, for instance, make it very easy to generate applications and go end -to -end with them. That's great because again, they represent a subset of the ecosystem and specific type of application. When you think about GitHub now, for instance, going to actually deploy applications, they're going from the ADE to the deployment and also the engine that writes much of that code, I think that's a pretty high probability. And the more you lean into the agentic magical creation of it, the less control that the developer has.

20:36How long do you think it will be before we move out of the experimental budget phase for large enterprises? I think it's incremental. So I think assistance are already providing value today and some of them are moving out of the experimental budget into real budgets. the resolution ones, the ones that are like true outcome oriented autonomous activities. I think they're in very specific fields. Support that we talked about right now I think is already real. I think to an extent the SDR ones, although there's like a lot of blast radius over there, if you mess that up, you could really hurt your band.

21:06Some of those sales tools are getting out. I'd say they're probably still somewhat experimental. Mostly it's the assistance that are getting the dollars. I think the rest are probably a couple of years away. And as I said, I don't think the LLMs are the limiting factor. I think it's more the processes and the systems around them to make the LLMs more reliable. You said the assistance to one's getting the dollars. I recently interviewed Sam what went on an open -airized Dev day. How far did the models go in terms of the application layer? It's pure speculation, it's unfair. But how far do you think they go in terms of the entrance in the application layer?

21:39I think as long as we are dealing with the same data and just the different way to interpret it, I think they will go all the way. And so search is a good example of that. There is no reason why they wouldn't go all the way to actually replacing web search because it's the same type of data that they need. They don't need a different type of expertise. To an extent, user experience of code generation or image generation might fall under that bucket as well, because it's the same data. It's the same ingested data that they would go into. I think those domains are there. I think when it becomes different data or very dedicated, elaborate workflows, like it's not about the magical maker call get back a response, but a whole system around how do you process those.

22:19I think that's outside of their domain and wouldn't make sense for them because it doesn't really threaten their core business and they actually would be more better off if many, many different applications were built on top of them. Okay, you have three options, right? You got chat to open AI at 150 or 160. You have anthropic at 40 where you have X or AI at, I mean, the latest pricing says it's 50. You can only buy one, which one do you buy? I think I go on Throbic. I feel that it still holds to the quality of the team, or maybe the stability of the team and their opportunity to grow. And yeah, the value arbitrage is a - Do you know what I think it's not?

22:58It's a safety and alignment team moving so much out of open air. I don't know what's happening inside of there, but it's hard for me to imagine that that the organization is not suffering substantially from the churn in leadership, even regardless of whether the reasons are legitimate or not, having such a big turnover must really rock the boat. Final one on this, and I, you said about search kind of being the obvious one for them to do, and obviously they've got the LGBT search, I think it is. Would you rather invest in public state nine? I still think perplexity is not going to go as far as people think.

23:33mostly not because of OpenAI but because of Google. I think people, again, we, I think we underestimate. Because of Google? I think we underestimate how sharp it is. It's, we underestimate how hard it is to change people's habit. And I think Google is so deeply ingrained and have the distribution around search. Google is generally pretty terrible at product on many fronts, but the one product in which they actually do a pretty good job is the search product and they have all the traffic. I don't understand what you mean. I think the world of you, you're smarter than me, you're an engineer, I'm not.