20Product: How Scale AI and Harvey Build Product | Why PMs Are Wrong: They are not the CEOs of the Product | How to do Pre and Post Mortems Effectively and How to Nail PRDs | The Future of Product Management in a World of AI with Aatish Nayak

25:40And then how do you go about deciding?

25:46Okay, And how do you go about deciding what to build, and then what are the operating cadence should be? Is something that I write, or engineering leaders write, on, OK, how should you think about making decisions? Why are we making decisions? There's both the meta aspect of process, like move really fast, or iterate quickly, or prototype quickly, and philosophies attached to that. And then there's things like tangibly, what should the UX be? And how much should be invested in technical that clean up versus not? Those are two different types of memos and thinking and reasoning. But I think there's like multiple layers and kind of like a fractal nature to this stuff.

26:20When you think about writing as a skill for product people, is it the most important skill for a product person to be a good writer? I think the most important skill is communication of your ideas. Whether that happens in a written way, whether it happens, you know, probably people love slides, whether it happens in slides, whether it happens if, you know, nowadays you can build prototypes very quickly and communicate your ideas like, you know, PMs on my team create prototypes all the time. using cursor or using the cloud to show their ideas. And so I think communicating ideas being very crisp about why you want to do certain things and communicating that is super important.

26:57Do we lose the design stage in a world of seamless prototyping where anyone really with some competence can spin up a prototype pretty quickly and we don't have to spend a long time in the design pre -prototype phase. Sure answer is no, I don't think you lose the design function or you lose kind of designers. I think you actually get better designs because you can prototype much faster, especially with AI. I actually always say prototype before the PRD, whenever you have an idea, whether it's engineers, designers, PMs, whatever, you should do the prototype before because you're not going to really know how to build the product or what the exact you actually should be, particularly like again with AI when these new patterns that you can establish, without without actually having, you know, you play with it, some internal customers play with it, you know, whatever it is.

27:46And then when you do the whole formal PRD kind of design process, whatever you end up writing PRDs and whatnot and just doing proper designs, but because you've done all this pre -work and prototyping, you can actually save a lot of cycles and just move faster at the end of the day and then ultimately get a better outcome. Well, it makes a fucking great PLD. Yeah, good question. I view PRDs as alignment docs. It is not a Bible for how something should work or something should happen. It is a alignment or kickoff doc. It explains, I think, a really good PRD explains why you're doing certain things, motivation with ideally data or customer quotes or some evidence that why we're doing what we're doing.

28:29And then what it does also include is kind of like a straw man user journey of like, here's what the user should do, step by step, here's what, you know, how it'll work, and ideally including different personas. And then the other big thing that is really important is what are all the hot questions and hot debated questions. Like I think a good PM should have an instinct for looking around the corner. What is engineering going to say? Why is this complicated? Or what are customers going to say about this little feature? You want to make sure you call out all the elephants in the room and all of the debates at the bottom of the PRD.

29:02So that when you do the kickoff, you can start to get alignment very early on. and looking ahead, and again, this comes with practice in your organization, looking ahead when you actually start building, when you start rolling out, what are you gonna be the gotchas, what are you gonna be the pitfalls, and really calling those out at the end of the PRD is super, super important. I just wanna dive as granular as possible in the process that you build with today. So when you think about that, and I have written a PRD now, what do I do? I don't know off, I'm a venture investor for a living, I said, do I like submit it to you, and then we discuss it in a meeting, who comes, how does that work?

29:39Yeah, so one general thing that, again, this is actually coming from a lot of learnings that's killed. One thing you want to avoid is this idea of feature factories. And what that means is like, imagine a factory, like a conveyor belt system, you have, okay, the customer says something, and then PM writes to PRD, and then designer does the mock, and then the mock is handed off to the engineer, then the engineer hands it off to QA, then the QA hands it back to PM, and then PM and goes to enablement and then goes to sales. You have this very linear process, and that is how you make really bad products that I say how you demotivate your whole team because no one wants to be just stuck in what they're the specific section.

30:15And so the biggest thing when you've written a PRD is get everyone in a room who's going to get involved in it, who's ever even going to touch it, debate it, discuss it. That's why the questions are so important because you want to say, where am I wrong? What are you worried about? What is going to cause problems? or what is actually going to be delightful, like how can we make this delightful? And really having that first, we do like PRD reviews, ERD reviews in like an hour setting every week where anyone can bring any dock and we debate it, we discuss it with everyone in the room who is even going to touch the product or kind of be consulted on the product.

30:48And so it is not something that it should just be the PMs doing this in a hole in the back room, being like we're going to do this, this, this, and then head off to engineering. It is super important to compress everyone together. What is your prioritization framework between Net New Features versus TechnicalDat? I interviewed the CTO Microsoft and he said that basically his most exciting application for AI today and moving forward is how AI will be used to basically demolish TechnicalDat. To what extent, how do you think about that balance between Net New Feature and Ravignu Upside or Product Upside or Usage Upside and then TechnicalDat Reduction as the alternative?

31:25Yeah, as you can imagine, engineering candidates ask me this all the time because they want to know what the header probably thinks about cleaning up code. So the nice thing I'll say is how does he all codepaces? How much debt is there really? If you're growing really fast, things will break and engineers have to accept that. The important thing is if something breaks or something goes wrong or something goes slower than necessary because you have to like re -factor something and it didn't work in a certain way, It's really important to do a postmortem on that and say, what could we have done to avoid that?

31:57If the answer is like, we could have spent one week in two sprints ago and said, let's clean this up before we do this thing. That's great because then you actually want evidence of technical debt affecting revenue affecting customer outcomes. You want to compile the evidence because otherwise, if you don't have evidence or you just refer to as technical debt overall, It is just like you can go forever cleaning up tech and we're going to be quite honest and so I think it's super important to identify Examples of where cleaning something up would have avoided problems later when you say okay We're gonna work on technical debt because of x1z you have to be very very crisp about what does the outcome look like is it a bunch of tests are written is it you're gonna spend two weeks in the refactor something you want to time bound it like with any other feature because otherwise like engineers will love to just forever, you know, clean up code.

32:47And there's obviously like more way more you can do. The engineers rather clean up code than right. And you could. My take is I think engineers would like to clean up code in startups more so than then write net new things. Wow. In terms of post mortems there, can I just ask you, is there a structured time every week for post mortems where you're like Monday, 5 p .m. we do post mortems? Is it on ad hoc basis? What does that look like? So there's there's retros and post mortems post what's a retro? What's a post? Yeah, so so retros Retros are you want to look back at some period of time and say here's what we did well Here's what we didn't do well and retros are should be you know generally more regular like every month for example at the end of the month You say what did we accomplish this month?

33:32What could have gone better? Here's what we should do for the next month post mortems are when there's an incident or issue or we made the wrong decision you want to evaluate that very specific decision or event that happened. Like, if there is an incident and our app goes down, you know, for X amount of minutes, you want to make sure you do a post -mortem to see like why that happened, how do we resolve it, how could we have prevented it. And so, post -mortems are more ad hoc. One thing we're starting to do more of, and this is again, me learning as a product leader at Harvey, is pre -mortems.

34:03Pre -mortems are, before you start something like a big project or big initiative, you want to sit down with your team and say, what does success look like? And what will prevent us from achieving that and what will go wrong? Give me all your worries. I'm generally a very anxious, paranoid person when I'm operating. I assume everything's gonna go wrong, and so maybe it's a self therapeutic for me to be like, guys, I think this is gonna happen, this is gonna happen, this is gonna go wrong. But I think, you know, we've done a few of these, and I think it does help with anxiety, it does help with like alignment, so that people aren't confused on what success looks like, or how often is the things identified as causing the problems, the things that cause the problems, you know, some might Tyson, you know, kind of getting punched in the face or whatever, you never expect it, whatever that quote is that I'm going to get put for now.

34:50But how often can you predict what it actually is versus something that you never expected? Yeah, oftentimes it is the things that are called out are the things that go wrong. If you say that we're not going to do testing fast enough and then you don't put an owner on this, like an owner to say, make sure testing goes faster. And it's gonna fail because no one's thinking about that. I don't think there's generally a lot of things that can go wrong that are out of the question. Of course, there's like unknown unknowns, like customer, yeah. Or the customer's not gonna like the feature or a new model's gonna come out and it's gonna wreck all our plans.

35:26That happens sometimes. On the post -mortem side, how quickly do you know when a new product or feature is not working, versus it just needs time to sink in to customer behavior as an adoption. This is something I think a lot about particularly for our user base because law firms don't like to move fast or are a versatile change, at least the like admin IT teams are a versatile change. And so we can have a product launch and it's not G8 and roll out to 100 % of users for a few months because people are just slowly adopting it. So I think you need to let it bake depending on the customer base. The ideal thing, what we do is you branch out user testing to concentric circles and that your consecutive circles get bigger and bigger.

36:12The first thing we do when we build something is we have an extensive set of lawyers internally doing various roles. We give it to those lawyers and they test it and they give feedback and we iterate. And then we have selected design partners that are consistent for most features. Externally, we give it to them. They feel part of the process. It's really good. They give feedback. And then there's a broader list of beta testers, like beta customers. We give it to them, iterate. And then there's the whole general public. And so, you know, this is probably true for most products and most enterprise teams is you want to make sure you're testing this way like much more often with kind of expanding circles.

36:51What have been the biggest mistakes you made in user testing or biggest mistakes you see other people making user testing? I think one of them is biasing the users for what you want them to think. This is most useful maybe with an example. So we built this product called Vault last Juneish and the essential thing is it allows you to do large -scale extraction of documents like if you have a bunch of lease agreements, credit agreements and you want 10, 20, 50 deal points out of it. out of each one, it creates a whole grid for you. And the way we, when we're building that, is when you upload a bunch of files, you say, hey, I want these 10 things, and Harvey will decompose your initial prompt, and suggest like, okay, here's the 10 things, am I on the right track and you know, check, check, check, and then you launch it, it demos really well, because it's like takes your prompt and then converts it and shows the little thing.

37:43And when we showed it to people, they were like, oh my god, wow, that's amazing. And then we didn't really put it into the hands of people for live matters, live use cases. We just showed it and they were like, oh my god, cool. It's a demo and it's like, you know, interpreting my intent. The thing that we did wrong there was actually people want to just make those terms in the grid itself. People don't want the AI to convert it. They just want to say, for at least 10 ,000 agreements, I just want this one thing creating in the grid itself. They want to get really tailored with what each of those terms means and not have the I guess all 10 terms.

38:18You get in your head sometimes and you buy a suit, users, you give it to them for a real use case and then you can end up in some of these traps. What's the biggest product mistake that you've made? And how did you learn from it? Yeah, so there's a few things. I'll pick one of them was the one I kind of just set right now around, vault and doing attraction. Another one that we were doing was, again, at Harvey Exit's recent, last year in Q4 we were revamping our core, or assistant product, we introduce basically two modes in assistant. One is kind of assistant mode, which is doing, you know, Q &A and an analysis, and then another one is a draft mode, which helps you draft contracts, draft clauses, emails, you know, whatever lawyers do.

39:01And the thinking was you want to give lawyers some choice of like, okay, what am I doing and pick the right, you use our interface and the model for the job because there are two different models and use it in interfaces. Okay, and this is the joys of user testing. We didn't use our test list that much. We should have done it a little more. We were just trying to move fast and get something out. As we built it, we launched it, rolled it out, and almost everyone was super confused and went to use what. They didn't know when to use this as a draft. I think it's maybe lawyers, maybe something else.

39:34They just didn't know, hey, when I am creating a response to a complaint, which should I use? draft or assist. And this is like everyone had this complaint. And then, you know, we started solving this by enabling, saying the model, this model was good for this reason, and the UX is good for this reasons. And so here's what you should do here. And some of those complaints died down, but it was mostly because people just got tired of complaining in general. And then obviously they started using the product and they said, oh, the draft mode behaves this way and is more legal ease than the assist mode.

40:07But that is an example where the right user experience is the AI should just pick for you. The AI should just know what user query and intent is or it should ask you questions to understand it better and then should just route you to the best system for the job and not have the user think about what that what they should do. I think it's preposterous that we're choosing all models. Whenever I'm on Thrope, I'm like, what the fuck are they? What these ridiculous names? Yeah. And I've no idea what any of them are and Grock have them too. And I'm just like, this is, I think it's ridiculous. You will never have the n5 years time.

40:41You will not have a choice of which model, correct? Yeah, exactly. And is user choice good? If you think that the paradox of choice and users get confused when they have too much choice, if you're just told, I think humans are lazy. Just tell them which one. On average, humans are lazy and just need to be told what's best for them. The whole model choice, and again, we found this trap. I think that whole paradigm was created for the silicon value or like tech audience because everyone loved playing around with like oh one versus you before Oh versus you know these models we hadn't I mean we still haven't quite figured out what the models are good at such that you can route Appropriately the biggest problem with these models and we have this problem is just like the capabilities are very evil constrained You don't know exactly which model is better for the job.

41:29Sometimes it's user preference We do a lot of side by side testing. Sometimes, still not super clear. And so I think the whole industry as a whole kind of fell into this trap, put everything in a drop down and give users a bunch of choice. To what extent are your eVals the same as the public eVals that are done on benchmark? I had Tom Ola CPU and president of clean on the show recently. And she said actually that their eVals were very, very different to the benchmark public displays of eVals. Deals correlated or yours were the different. Yeah, so our eVals are very different a lot of public legal benchmarks and then even things like you know scales Humanities last exam they're all multiple choice.

42:08I would love if legal work was multiple choice But every any lawyer will tell you there like a million options of what you could do and so one It's like multiple choices is is not the right thing and so we created this benchmark called big law bench consists of tasks that are real billable work tasks that lawyers do on a daily basis at our biggest customers and big law firms. The nature of these tasks are, they're very much open -ended. The problem with open -ended tasks is how do you evaluate them consistently across tasks? If you're saying Harvey Generator Chronology, that is very different than Draft Media Motion for Summer Rejudgment, which is another litigation task.

42:47Then what we had to do is create rubrics for each of those tasks as well. Those got very specific and basically we have like hundreds of different tasks in each task as a rubric now. And the benefit that we have and we kind of architect some of the getting is we have a lot of lawyers that we have brought from big law to work with our AI team with our engineering product team who sit side by side with them and say here's how legal work actually happens because I'm not a lawyer, I come from AI, I don't know what good often looks like. It's really hard to do product management for sometimes for something you don't know what good looks like.

43:22So it's really important to rely on that domain expertise. And then another thing is our models of choice are open AI right now. Has that changed? So they've been investors in us from the beginning. We've gone our access to open AI models for a long time. We build some custom solutions with them. For the most part, that's still the case. We are seeing some capabilities now with Cloud, with other models that are better for some tasks. And so we're now exploring. Can I ask why it's a little better than I can add? Right. Claude, so we did these evals recently. Claude 37 in particular for legal reasoning.

43:58It's better at long form legal reasoning and drafting long form outputs. So a whole section of a merger agreement, for example, and making it super consistent. It's really good for. And then things like extraction where extracting some terms for them, SPI, like a share purchase agreement is very nuanced because the answer isn't a verbatim text in the agreement. It's like this is getting a little in the weeds on legal, but if you're getting the indemnification cap on a share purchase agreement, you have to reason over four or five different clauses because that cap is not a name thing in the deal.

44:35You have to reason over the dependencies of four different clauses in that agreement and then extract that out. For some of those types of extractions, cloud is 3 .7 in particular starting to get better. Every single model leads that's come out from OpenAI, from other competitors. We've benchmarked for since beginning of time at Harvey and 3 .7 is really where we're starting to see some of some of the performance better than OpenAI. When you think about the distribution of value and usage in the model landscape, what do you think that looks like in three to five years time? Yeah, I think the model companies are going to have to start to become product companies way more.

45:11Do you not think they already, if you look at like, you know, Bunny, what OpenAI, which is deliberately chosen, it is now going to be a consumer product company, 12 .9 billion and revenue consumer product company, I think very clearly. I think anthropic not as much as the fucking should be, but it's choosing very much to be a developer first and API company. Yeah, but I think more of these labs should start to think about what design product you want to deliver and the cloud companies, they're going to run the best software, they're going to drive margins to the ground. And so competing on inference with cloud companies is really hard over time.

45:46And so if all your revenue comes from inference and developers, it's not an ideal place to be. And instead, you want to build products, maybe work with some particular application layer companies like Harvey to deliver your products intent. You're building one of the hottest AI products in Silicon Valley today. I'm fascinated to your thoughts on this. Cursor is the golden child. there's also code in. To what extent is there lock -in for cursor? And how did the two compare in your mind? I think it's like UX is super important. You know, I've heard some engineers say they like cursor is agent mode much better than when surf.

46:22Some of people have said the opposite. So it's like what is the most delightful UX? And sure, maybe you can copy UX, but I think it's a little more nuanced. And then it's like data and governance, particularly in the enterprise. Like the models are only as good as the context you give it. And so any of these products can you actually tap into knowledge that the enterprise has about its code, its products, its architecture and use that in more helpful ways to tailor the outputs that do your engines, right? And I know Codium has focused a lot on getting a lot of the enterprise data, the enterprise understanding of existing code bases and making sure those, that knowledge architecture is really good.

47:08As a user, which do you prefer? I currently use actually Cloud for prototyping because I actually don't want to deal too much with the very new on stuff that WinServer cursor has. And so I end up using Cloud and then I've actually been using also Replit agent mode. The reason being is because Replit takes care of deployment for you. Like, I don't want to mess with deployment front end, back end servers, all that. And it really has a nice interface because you can give it some task of Give me a legal chatbot app or something and it'll say here's my plan. Here's the additional features I think that would be useful and you can check check check and it starts building it and then you can you can see it as it's Building it and then you can see the code of course and modify it and so things like for again for my target audience things like agent Replay agent mode, lovable, bolt are a little more attractive than some of the IDs Do your team prefer code more cursor?

48:01They prefer cursor, I believe, is the latest. I think it's mostly because we haven't procured Windsor if you're out. Curseers have a better developer brand. They do, they do. And you know, Silicon Valley everyone talks and so a lot of the cursor folks are good friends and networks of a lot of our team and so they're done to prefer cursor but again, I think time will tell. Final one for you to do a quick fire. When we look at AI at changes so much of the product leadership and product management role, what do we not know about the future of product that you think a lot about in your leadership role today?

48:34In a world where prototyping and the actual execution of ideas is much cheaper, the ideas will matter a lot more and how you incorporate unique knowledge will matter more, like domain expertise. Maybe the future is lawyers, doctors, domain experts are driving more of the product decisions because you need to bridge the gap between what the model and the UX is to how you actually apply it to a certain profession. Those ideas domain experts will matter a lot more than they then even do now. Like I think we've seen the benefit of having lawyers in house like literally sitting next to engineers. final one and it is on the future of AGI, but you said some really interesting comments that you have to touch on for a quick five.

49:22You said humans are a bottleneck to AGI. Why? Yeah, everything in Silicon Valley thinks that AGI will just happen. You'll see a lot of GDP growth and everyone will live happily ever after, UBI, whatever. I think realistically, you will run into cultural, legal, and like regulatory barriers to actually wide -scale adoption of AI, like kind of breaking each one down. If you end up, again, AGI, you assume it can do really high level tasks, like advising on a merger, advising as a board member, if you can see you. What are the governance frameworks for regulating, you know, an AGI that is running a company, like no one knows yet, and that's gonna take time to happen.

50:06I'm thinking legal, just because I actually have asked this question a lot to our partners and our customers on like what happens, and everyone's like, like we're gonna need some indemnification and liability of like all these different things that if AI autonomy starts to take actions in the world, what happens and who is responsible. I think on the culture side, it's like what are the last domains where humans actually want AI involved? So as an example, in Silicon Valley, I and many of my friends actually talk to like cloud or or a chat chat with you for kind of therapy or like emotional advice of like, I'm thinking through this problem.

50:43It's really hard. You do? Yeah. Exactly. That's the reaction I get from my hometown friends. Well, it's because I'm European and so we actually get it talking to other humans. Exactly. Rather than transactional nature, if you say the convali abuse. So you talk to them about like emotional stuff. Yeah. Do they give good output? Cloud does. Catch me too, doesn't. Wow. Not as empathetic. That's really interesting. Grock does too, actually. Grock does really good advice. I might try it. Yeah, you should. So that's another one where it's like, I think everyone should have AI therapists. That should probably happen at some point.

51:15And there's actually a study done recently where someone did a study where they had a bunch of participants talk to a robot that was based off of opening I or something for advice. And I think like 60 to 70 % of them reported higher sentiment and more happiness after doing that for like four or five months compared to the control group who mostly talk to humans. And it's because people don't want to be judged by other humans, even if you do have a therapist that is in a safe space or something. Yeah, I think your willingness to open up is probably so much greater when you're speaking to it. And Al, I'll, um, uh, final one, why do agents need humans more than humans need agents?

51:52There's just one interesting about Genai where, so the, who is the consumer of the product or service or the work that's been created and who is the producer of it? You know, examples here are in architecture, it's the client versus the, the architect creating the design, or there's the marketer versus the brand person for content, or it's like the client versus the lawyer. And do you end up like what model do you end up taking? Do you help the producer 10X there output and make them more productive or go direct to the consumer? And a lot of people have thought they can go directly to the consumer and say, here's a bunch of leads or here, but here's a bunch of legal work, whatever.

52:30But ultimately, I think the humans don't just always trust AI. They trust other humans using AI. I think again, depending on the domain, the right thing is more likely that the agent partners with the producer of the work to deliver for the consumer. I don't think anyone suspects that we're going to get rid of law firms and have direct consumer law for consumers, do they? I mean, that's not a thought. It's like saying we're going to get rid of accountants because we're going to have the AI do accounting for you. No. You'd be surprised how much some people have that opinion. Wow. People who'd earn not familiar with technology.

53:03And same thing with marketing. Maybe that is super, super low level. I need a rental agreement for a one -bed flat. This is the address, one page, please. Done. Do you know what I mean? Yeah, yeah. And again, the other thing is, if you're booking a flight, sure, let the agent do it. It's easy or you're ordering dinner or whatever. If you're doing something much more complicated, like an architecture drawing, like legal work, tax work, the agent is going to need a lot of human context to actually make that work, whether that's from the consumer, whether that's from the producer. And so it's going to have to ask you a bunch of questions.

53:34and even more complex knowledge work that enterprises do. I think people just have this assumption that agents will just do everything automated and black boxing will be fine. But people don't trust black boxes. There are unique human elements, like context, like emotions that you need to factor in. And so I'm thinking beyond just like the booking flight agents, something way more like doing very complex work agents. Dude, I want to do a quick fire with you because I could talk to you all day. Yeah. What's the most controversial opinion you hold that many disagree with. Yeah, so mine is like, I think we should scrap the Department of Education.

54:08That's happening. And, well, I mean, it's not in the UK. In the UK, yeah, yeah. It's a department of culture and department of sport. I mean, what the fuck? Yeah, yeah. I mean, ridiculous. Yeah, I actually have a list of these that, some of them are not a fish. Say, well, I would, Well, you have a list of your controversial opinions. I do, yeah. Well, on notes or like? Yeah, yeah, on notes. I can pull it up if you have said. Yeah, I'm fascinated.

54:35That's so cool. I've never heard that before. It's because I, whenever I have an opinion, I write it down and then my recall I'm reading my brain is not great, so I just knew I write everything down. No, I don't do it. I mean, my thing is I have a pen and paper for the side of my bed. Because I often get my ideas when I'm lying in bed and I don't write a scribble down. Yeah, yeah.

54:59Or I send it in the subject line to my EA. 2AM is like new show. Like. Um.

55:12Okay. Yeah. I got one that I can say a lot. I'll see the other one. Not on recording. Yeah. So it was nice controversy. The opinion. Yeah. I think. People generally welcome. I'm stability, but I think the to truly be happy, to truly be fulfilled, I think you need to embrace chaos, you need to embrace instability because that makes life much more interesting. I think there's always the, I'm going to settle down and be fine and retire and be chill. You could do that, but I think the road to getting there, you should actively embrace conflict you should act to be embraced uncomfortability because you'll be much more resilient to whatever life throws at you over time.

56:01What was the most uncomfortable thing you embraced? So there's a few things in different parts of my life. Maybe in college, I willingly took probably like one of the hardest like courses at Corny Bell and like operating systems, you need to build operating systems from scratch and I didn't have to take that. I could got in my degree without it. I just heard a lot of crazy things of people staying up the two nights in a row doing it. I was just like, let's do it. Why not? Another thing is, you know, out of college, like a lot of CMU graduates, we were offered jobs at the big companies, Microsoft, Palantira, Stride, whatever.

56:40I decided to not do that, move to LA, join my friend's company, which ended up failing in a year. But just something off the beaten path, and when I live in LA, and so I ended up doing that, and then me and DERD my way through startups, inside of kind of going for the big companies. Biggest advice to graduates today, entering a new AI workforce world. Yeah. So I think it's going back to what I said in the beginning. Focus on the skill sets that you want to learn. I think I still get this where a lot of engineers like I want to be a product manager. It's like one of the most valuable things you have is your ability to code if you're an engineer, let's say.

57:26And I wish I I stuck to it more, but I think it's super important to stick to that at least because it's much better with vibe coding whatever now. And then don't be afraid to try a lot of things. I think people expect that they have to figure everything out in their early 20s. That's just like not the case, especially now with AI. Like, you don't know what's going to be valuable when you're 26 when you're 22. So I think try a bunch of things, experiment, see what you like, see what you don't like. How much of Harvey's code is AI written? There's a good question. I mean, I think it's probably, it's not that much.

58:01It's probably like 20%. 20 % it's that normal today. I would guess so I think we can probably use AI way more than we are How can you use AI in a way that you're not already? Unit testing is I think can be way more automated with AI I think I mean in Vagranger still use AI somewhat for writing their own unit tests But that whole layer of unit testing should just be all AI. What would you most like to do in your role? But because of decision making or resources you're not able to do Go on podcast. No, Martin. I think I would like to work on. So there's Harvey, like the productivity suite that we have today, which is the core product.

58:49And then there's Harvey that does a lot of the end -to -end work in collaboration with law firms and enterprises. And that is stuff like you know that enterprises pay millions of dollars to do like I want to just tackle that that work That is way you know way out you need to do research you need UX Experimentation so we'll get to that but right now we have to focus on Harvey the product to sweet the software that lawyers use and not the the Harvey that that does the work and dad I say that it's easier to build an AI company in London and the San Francisco And it's like, we have the supply of AI talent from some of the best institutions and universities, but crucially, we don't have the churn.

59:36Is the churn in San Francisco AI is brutal as it looks from the outside? What do you mean by churn? Like in PlayChurn? People going to other hot company because opening AI put 2 million on the table for your best engineer or podium or wind cursor or whoever. Yeah, yes, I think I think churn is very real. I think the talent Go and go war is is I think very real? I mean this is not just in AI like Some of the best executives in the world live in this like tiny slim peninsula in the Bay Area and especially if you are Growth stage companies where you need to bring in experience people experience executives There are all construction there, have families, they have all these people who just want to stay there.

1:00:26And so, sure, you can have a lot of their early stage talent in other places. But I think the mind melting you get, the experience you get there, I think is second to none. Which company is the hottest company right now for the best talent you think? I mean, opening up an Anthropic. I think I would have said opening eye maybe six months ago, but I think in Theropic is really starting to appeal to a lot more people. Yeah. By no one, what recent company product strategy in the last 12 months? Have you been most impressed by other than Harvey? Yeah, of course. Yeah, like I had this below, but I'd like...

1:01:09So for me it's actually like Canva. I think Canva has been super smart in how they've done 10 plates and how they've open up a marketplace to the dynamic where you can kind of bluntly borrow add on, lend to anyone's creative whims in a really cool way. Yeah. It really solves the cold stop problem. Yeah. So, okay. I will say, I do think perplexity is honestly just killing it. Like I have a lot of respect for that team and the speed that they're moving. Like, sure, you can say, oh, they're doing a lot of things now. But I think the focus on the core experience of making an answer really, really, really fast.

1:01:45And everything else out of the way, it is, I think, you know, second to none. And this is actually one where product is the king. Like, I mean, short, they have distribution and stuff, but people adopted it. Again, it's consumer, but people adopted it because the product is so good. And I think they're, they're starting to now focus on certain verticals like shopping, like travel, like finance, where you need a lot of internet that knowledge and I think that that's right. It's really smart. As an investor in complexity, I'm thrilled to hear that. I think Aramund is amazing. Listen, I've so enjoyed this.

1:02:19Thank you so much for putting up with the slightly wayward approach to the schedule, but you've been fantastic. Yeah, thanks so much and glad it was useful. God, I so enjoyed that episode. And if you enjoyed it too, then I would love it if you left a review all -lighted, it makes such a difference for discovery and really do so appreciate that. But before we leave you today, are you struggling to beat model benchmarks or implement Gen AI in your product? If so, you need churring. Churring is an AGI infrastructure company, backed by incredible investors like Foundation Capital and Westbridge Capital.

1:02:54And they do two things. Number one, they help leading companies in AI labs. Like Salesforce, Anthropic and Meta, enhance their LLMs with advanced reasoning, coding, multi -linguality, multi -modality and more. Number two, they combine human and artificial intelligence expertise to deploy cutting edge AI systems for awesome companies like Rivian and Reddit. Right now, Turing offers a free five minute self -assessment to help you pinpoint your place in the Gen AI journey, get tailored next steps to optimize your model strategy, and then finally learn how Turing can refine and implement your models for better performance.

1:03:32Take the guesswork out of Gen AI, visit churring .com -4 -2 -0 -VC to start your free assessment today. And once your Gen AI strategy is on point with churring, make sure your team stays just as sharp in the day to day. Hey, managers and leaders, want to make your meetings way more efficient and save time with over a billion meetings processed, or to AI is the ultimate AI meeting productivity tool. it gives you real -time transcripts, quick summaries, action items and insights, plus a voice -activated agent that keeps you focused and productive. It's unbelievable. Imagine cutting down your meeting prep and follow -ups, and freeing up more time for what matters.

1:04:12Orders trusted by over 25 million users, including Fortune 500 companies, to boost productivity and collaboration. So are you ready to take your meetings to the next level? will head over to get .otter .ai forward slash 20vc and grab a 30 % discount. My word, that's so generous. When you subscribe to a business plan for your team, that's getotter .ai forward slash 20vc. Let's make your meetings work for you. Alright, meeting sorted. Now let's fix your software. Whether it's the software you build or the software you buy, your text stat should be creating results, not creating roadblocks. Will Pendo's No Code Software Experience Management Platform makes your software better with tools to see where users get stuck, guide them within our messaging and constantly improving your UI.

1:05:00It's so easy that over 14 ,000 businesses use Pendo to increase revenue, lower costs, and reduce risks. Businesses love the control, engineers love the freedom. Everyone wins! Star for free today at pendo .io -4 -20 -product. As always, we so appreciate all your support and Stay tuned for an incredible episode coming on Monday with Victor Lazarte at Benchmark.