Scaling Legal AI and Building Next-Generation Law Firms with Harvey Co-Founder and President Gabe Pereyra

0:05Gabe, thanks for doing this. Of course. Yeah, thanks for coming. Maybe we can just start with, like, for anyone who hasn't heard of Harvey, what is the company? Can you talk about the scale and who you serve today? At Harvey, we're building AI for law firms and large in-house teams. We're almost at 1 ,000 customers, 500 employees. Started about just over three and a half years ago. And so been kind of scaling quickly since then. And kind of you guys were some of our OG seed investors. So, yeah, good to be here. Maybe from a most basic perspective on the product, why is it not just, you know, CoPilot or ChatGPT or Claude?

0:43Yeah, I think that's how the product started. So when we first raised from OpenAI, we got access to GPT-4. And I think GPT-3 to GPT-4 was such a big model jump that the intuition at the time was just give the model to lawyers and have them play with it. And I think that industry was so text heavy that you got so much value from just interacting with the models. And then I think as soon as you gave it to lawyers, you also ran into all of the sharp edges of the models of they hallucinate, they're not connected to a bunch of our context. And so I would say the past, the kind of first two years of the company were, how do we build essentially the IDE for lawyers around these models that connect it to all of the contexts you need to be productive as an individual lawyer?

1:33But I would say in the past year and going forward, the big problem we're solving is not how do you make individual lawyers more productive? It's how do you make a team of lawyers working on a client matter more productive? And more importantly, how do you make an entire law firm working on thousands of these client matters more productive and more profitable? And so I think when you get to that scale, a lot of the problems you're solving are not just model intelligence problems. They are these orchestration, governance, and kind of all of the enterprise product problems that you run into at scale.

2:03You've also been broadening from just law firms into enterprises, into big companies using you in concert with both their in-house legal teams and external counsel. So can you talk more about that and how that's been evolving as well? Yeah. So we started selling to the largest law firms. And something that started happening about a year and a half ago was these law firms started showing Harvey to their clients. And their clients both wanted to collaborate more effectively with their law firms. And they also wanted to use this directly in their in-house department. So we recently announced we signed Walmart.

2:37We're working with AT &T, a bunch of these Fortune 500 large private equity firms, Global 2000, kind of the largest consumers of legal services. And what we're starting to build is a platform for the in-house teams to do the work that they do internally. So things like contracting and this long tail of all of the legal operations you need to do that you typically don't send out to law firms, but also the collaborative tissue of I'm working on a large transaction, a litigation. I need outside expertise. I want to securely share this data with my law firm. And there's a lot of technical problems there around security, data privacy that we want to solve so these law firms and their clients can collaborate effectively.

3:21I think we have a largely technical audience, but also most people don't know exactly what legal workflow looks like. I think before we really started working together, I imagined it is just like, well, what do you mean? I like email my lawyer and he thinks about it and reads a document and then send something back. And there's like redlining involved somewhere. Maybe there's negotiation. Can you paint a picture of just what workflow means to you guys? Yeah. Yeah. And I think a lot of people, when they think about legal, they think of consumer legal. And so I have a lease and I need to get input on that.

3:57That's completely different than what these massive law firms are doing. And so I think a really good example of what these firms are doing that, I mean, you guys will be familiar with. And I think a lot of people in the startup space will be familiar with is what law firms do for venture capital firms or private equity firms. And so VCs, PE firms, they do two main things. You raise money and you invest it. And so the main legal - And we do podcasts. And podcasts, which is actually important, but there's less legal work there. And the important things you need to do there are fund formation. So how do I structure the entity that is going to hold all that money?

4:34And it sounds easy, but if you're a large private equity firm, you have a sovereign wealth fund that comes in and they say, we need to structure it in this way because of tax implications. Then you have a pension fund that has these other requirements. And so it ends up being this incredibly complex process of how do you draft the limited partnership agreement, which can be 100 pages. every investor that's investing, you can have a hundred and they all have side letters that modify that. And you need to understand if I modify it this way, it's going to have these implications. And a lot of it is the project management that also goes around coordinating all of these products.

5:07If you're raising a billion dollar fund. And then once you've created that fund, there's all the investments you do out of that fund. And so, for example, when we did any of our series, you need to get a data room. We share a bunch of data. You look at that, You need to understand the contracts we have to make sure that the revenue we say we have is actually structured in the way we've claimed. And, you know, are there litigation? All these things. And so it's this massively complex process of understanding. And I think one analogy you can think of like understanding a code base, but the code base is all of these contracts and all this legal work.

5:43And I think the reason legal is so difficult is the workflows aren't structured. So the same way with programming, it's really hard until these models to build tools for programmers. You basically just had an IDE and then programmers did stuff in all the different languages, but you didn't have like, oh, here's a tool for Python. Here's a tool for C++. Legal is kind of the same way. And so I think a lot of why you're seeing traction in programming and legal is I think there's a lot of analogies of these workflows, of they're so text heavy. and the workflows until you had these models, you couldn't structure them in the way I think you can now.

6:20So one of the directions that people are going on the coding side is to build things that are being called like agentic. And it's very early on in terms of what agentic means, but basically being able to deconstruct a logic tree in terms of what are the set of actions that you need to take in a certain situation. And then having the AI agent go back and sort of check each one of those items, do it, go on to the next item, double check it against the prior one. Do you do that from a legal perspective over? Is that something that's a little bit more in the future relative to where code is today?

6:46Yeah, we're starting to do this now. And actually, when I was at DeepMind, a lot of the RL research I did was that. And so when we first got access to GPT-4, we had the very strong intuition of, okay, you're going to be able to string a bunch of these model calls or eventually do things like reasoning models where the full agent is differentiable. And even the first day we got access to GPT-4, Winston went in his room for 14 hours and just redid a bunch of his associate tasks. And when I looked at the work he was doing, it was essentially like this hacky agentic where he said, okay, I would need to go look up this case law, summarize it, take that summary, use it to draft.

7:27And so seeing him do that gave us the intuition very early on of that's the direction this is going. And you can kind of think of associates as agents. They get this task from a partner that's, hey, I have this high-level case strategy. I want to see if I can find a bunch of case law that supports it. Can you go research that, look it up, cite it, write me a memo? And so a lot of the systems we're starting to build look a lot like that. And I think one interesting direction that the coding labs, the research labs are going is building these RL environments where you deploy these agents and they can interact with a code base and see if they can pass unit tests.

8:04And in legal, that RL environment is a client matter. So you have all of the context of a fund formation, an acquisition, a litigation, and the models are starting to learn, let me go in the document management system and see if I can find this, go in the data room or do case law research, get feedback from your partner. And so I think that research direction is super, super interesting. It's really interesting to make the associate analogy because I remember when I led your guys' series B, which I think was maybe two years ago now, it was a while ago, I called a lot of your big customers and talked to the head of the law firm or talked to the head of some of these institutions.

8:43And one of the things that I thought was really striking is number one, they were adopting legal software, which before was really hard to sell into them. And because of what you were doing being so striking and important, they were adopting you really fast. The second is they weren't threatened by it. And I thought, oh, they'd be threatened because it may augment or eventually replace certain aspects of law, et cetera, or help sort of change that dramatically. And one of the insights they kept bringing up that I thought was really interesting is they said, as we think ahead, as this sort of AI tooling and agentic workflow spread through Harvey and companies like you, how do you think about the future of a law firm?

9:17Because instead of hiring 100 associates of which you assume 10 will be partners eventually, maybe you only need 50, maybe you only need 20. And so are you even hiring enough people to know who'd be a great partner? Yeah. Because you're going to shrink the set of people that are needed to do certain tasks over time, right? Right now, that isn't true. It's augmentation. It's expanding business. But that could happen in the long run. How do you think about the future of law or what law firms will look like or the evolution of all that? Yeah, this is a great question. I think it's changed a lot in the past couple years.

9:46I think something we are starting to talk with law firms a lot is how do we think of training the future generation of partners where, to your point, these law firms have these leverage ratios where you have a lot of associates, but much less partners. And there is value to that because not everyone is going to become a partner. And part of going through that process is how you find, oh, this is the person that I would trust to do this very complex acquisition because they've gone through that experience. And so I think the part I'm optimistic about is if I think about over 10 years ago when I learned to program, it was super painful, right?

10:22Like you had to go on Stack Overflow. It was hard to learn multiple languages because you're like, okay, I'm just going to like learn Python. I'm going to learn TensorFlow or something. It was just like hard to even learn that. It was hard to ask questions. When I was at Google, you don't want to ask a bunch of questions because people would be like, oh, you don't know that. And I think - It gets stuck all the time. Yeah, exactly. And now with the models, it's like programming is so fun to learn because you can just be like, here's how to write this in Python, translate it. Why is it written this way?

10:48And you can learn this so much more quickly. And we see lawyers doing that with Harvey, where they'll say, generate this merger agreement. Why did we structure it that way? And so we're already starting to see some of that. But I think the really big opportunity for law firms is how do they take all of the internal partner feedback and data that they've created and use that to start training. I think that's one big piece. I think another conversation we're having is, to your point, how do you just generally start restructuring firms? I think this is one where we have some intuitions, but a lot of it is going to depend on the firm, the region, the size, their specialty, the types of clients they serve.

11:28I think one of the things that's very challenging with law firms is they are really a collection of all these practice areas. And so the firms that specialize in litigation look different than the firms that specialize in large transactions versus like midsize transactions. And usually the big firms do a collection of these. And so a lot of what we're spending time is practice area by practice area. Can we go and sit with, here's the fund formation group and their private equity clients and start thinking about what that would look like in terms of the workflows, the staffing, the pricing. And I think it is a really interesting problem where a lot of the value in the product and the platform is not just the product itself, but how do we help enable these firms to transform?

12:13And so when you think about it from that perspective, our goal is how do we make these law firms more profitable? And it's not just a product problem, it's thinking about their holistic business and where do we fit in in that bigger picture. Yeah, it's really interesting because when you look at the set of functions that partner fills. And I'm thinking in particular of consulting firms and less about law firms, simply because I'm a little bit more familiar with consultancies. Some of it's the pattern recognition, the high level thinking, the strategy, and then part of it is like the sales and really being able to make that client connection.

12:44And so it's interesting to your point, think about more broadly, how can AI augment all parts of their business versus just the legal workflows? Yeah. And to your point, I don't think that part changes where it's like, Like when we think of the, like we're now larger consumers of legal services. And when we think of the best partners we've worked with, I don't think the models are doing what they do anytime soon. And I think what's interesting is I think the role of law firm partners actually doesn't change that much in the same way. I don't think the role of very senior engineers changes with this because you're largely delegating work and what you're getting paid to do is here's the high level strategy.

13:19Here's the right abstractions. go write the code or write, do the legal research to help me do it. And I will interface with the client. And so I think that my guess is that doesn't change too much, but some of the like lower level functions do change because of this technology. One of the things that you said that I thought was interesting, like in another conversation that we were having was that there's an analogy that you could make between like a great senior partner, like a Gordon Moody type. That's like a distinguished engineer working on systems at Google. Right. I think for a more technical audience or just general business audience that doesn't really know what Gordon Moody does.

13:55They might assume what Alad said, which is like, isn't like 50 % of that like his network or his reputation? But what you were pointing out is like, there's expertise in the ability to predict like a sequence of arguments that is going to like get you to the answer you want or manage risk. Exactly. How does that translate to an RL environment or a task for you? Yeah, that's a good question. So for maybe for background context for the audience, Gordon Moody was a partner at Wachtell, which is one of the top transactional firms in the world that joined us early on and is now an advisor. And kind of the analogy I was giving is why is a senior distinguished distributed systems engineer at Google so valuable?

14:39And a lot of it is the experience they have architecting these systems that none of this is public. This won't go into of the models for a long time. And so if you're building search at Google, these people can just point out, hey, if you build a system this way at this scale, it's going to collapse for some reason that is super not intuitive. And one of the examples that Gordon talked about early on was he was a part of when Michael Dell took Dell private and then restructured it and took it public again. And this was like a multi-year, super complex financial and legal restructuring of an incredibly large business.

15:19And what he, when you talk with him is incredibly good at, it's the same feeling as when you talk with a very senior engineer where he can just, he has the whole picture of this legal entity in his head. At the time they had to do the largest debt offering of all time. They had to create, they had to invent a new financial instrument. And so it's just understanding if I need to raise this much money to do this part of the transaction, this is how I would structure it. And so a lot of the value he brings is not just the relationship, but it's just that technical understanding of how you architect these things the same way that how you architect like very large software projects.

15:55And I think when that translates to an RL environment, part of what is missing from the public models is the process of looking at one of these entities and figuring out, given all of the context of, I want to do this merger, this is the right way to structure it. Just that process. That's a reasoning trace, right? Exactly. For an expert, just like it would be in code. Yeah. And if you looked at that data set for one of those transactions, it would be client comes to Gordon, says, I want to do this large merger acquisition. And then there would be meetings and emails talking about, OK, this is the background of the two companies.

16:35This is roughly how we would structure them. These are all the things we need to look into. And a lot of the data would be Gordon giving these tasks to associates to say, okay, look into these risk factors of similar transactions we've done. They would do research and say, okay, maybe we could structure it this way. And then he would point out this really subtle thing that, hey, actually, in this case, if you structured it this way, this thing's going to happen. But none of that shows up. All you get from these public mergers is an SEC filing. And so you do see the final result, but most of the value or what you need, I think, to eventually improve these models is the decision-making process, the same way you need these reasoning traces to train these models to do any of these reasoning tasks.

17:18One of the, as you mentioned, like the labs are all very focused on RL scaling in like coding and math domains. I think those is like highly verifiable, right? Not perfectly so, but like how do you think about the appropriateness of like law for RL given it's not as easily verifiable? Yeah, this is one of the biggest problems. And I remember we had conversations early on when we were trying to figure out what is the right evaluation structure. So I think the hardest thing about legal is most of these tasks are very long form tax generation. And so there are definitely subsets of legal work that are super verifiable of go in this data room and just find all the change of control provisions that you can kind of build these traditional data sets.

18:07But for something like generate this merger agreement, it's really hard to just give some binary like this is good or this is bad. And I think this has been like a big research problem, like with all the labs we work with. And also internally, there is just this open question of how do you build that reward function? And if you think of what that reward function is at the law firms, it's the partners, right? Like at the end of the day, there is no way to verify this. Besides the senior partner who's done a bunch of these said, yeah, this looks pretty good. And so internally, these law firms have a bunch of data of here's all the edits that went into this and the feedback.

18:43And so we are starting to think about how do you use that to train these reward functions. But I would say that is one of the really big problems. But I think one of the interesting things is, I think you actually have the same problem in programming, where I think in the short term programming is verifiable, where you can look at unit tests. But once you get into real software engineering, like the unit, there is no unit test. It's like, I deployed this and a million users used it for six months and it didn't crash. And so you get, and it's like mergers are the same where it's like, you can make sure the filing is correct.

19:18But at the end of the day, it's like three years later, the companies are still merged and they didn't take on litigation they didn't expect or something like that. And it's like, that is eventually the really valuable human experience, right? That's what you pay really good software engineers or really good lawyers where they have that decade long track record of they can build these systems and they haven't fallen apart. And a lot of these stuff, the same way you can't unit test, it's like hard to verify. So it is, I think, this really interesting open research problem. One thing that you guys are doing on sort of the other end from pushing the bounds of what Harvey products and the models can do is like just get them deployed.

19:58And you recently started this Ford deployed engineering force. Yeah. This is confusing to me because I'm like, well, you're not necessarily like an application building company, which is how people have traditionally thought of FTE. Like, why are you doing this? I would say the model that we're operating under is not a full Palantir, let's go into the code base and kind of build custom software. I would say this is closer to like Sierra's agent engineering program. But what we're starting to run into a lot is, I think early on, we did a really good job of building a horizontal platform. We did not that much customization for customers in the sense of building specific things for specific customers.

20:44I think the thing that was nice about legal was we could build things like Workflow Builder and things into the product that would let customers customize the product. And then for very large customers like PwC, we did some customization. But now we're getting to the point where when we're starting to talk with law firms about, hey, we want to take a bunch of this data and help you build a model or build agents. there is some amount of, we need to go into your environment and figure out how to connect all the data. We're starting to connect to a lot of their business systems. So their billing systems, governance systems.

21:21And then especially when we start working with the Walmarts, the very large banks, the Fortune 500, they're much less standardized than these law firms. And so there is just this massive amount of work where we go to a large bank and they say, we don't have any document management system for our legal department. Can you just build us one? And there is a massive amount of demand of we just want smart technical people to sit here and help us think about our business and our operations and how we should start mapping that into Gen .AI systems. And for us, it's a really good way to figure out the roadmap where, for example, Glual is like one of the fastest growing private equity firms that we recently started working with.

22:07and we meet with them all the time and they're just like, there's all these things that we feel like we could map into Gen.AI. We don't quite know what it's going to look like, but let's just sit together and figure it out. And so I would say that's a lot of the genesis of the program of how do we just get more people that can work with all these customers and start kind of paving the way of some of these new roadmaps in different verticals. Yeah, I think what you're describing too is a very standard enterprise playbook. And I think in Silicon Valley, people almost forgot because of the SaaS era that if you're Oracle, if you're Dell, if you're IBM, if you're any of these larger organizations, this is how you sell software.

22:44Exactly. Right. You have a platform, you have a bunch of customization around it. People have bespoke data sets. They may not always have the ability internally or enough people to implement certain connectors or systems. Right. And this is like the standard way to do it. And then as you do it over and over again, you start repeatedly turning that into part of the platform. And so. And I think a lot of these started with doing something that resembled FD and then you get big enough that you get this like implementation ecosystem. There's all these third parties that will come in and implement.

23:10They'll be like a certified like vendor. Exactly. And I think the interesting thing we're actually starting to see is law firms are starting to do this for their in-house clients. So they're starting to go and take Harvey and go to their clients and say, hey, buy Harvey and we'll help you build all the workflows and implement it because we have the scale and the expertise to build this where typically these in-house teams, the smaller ones don't have like the budget or the in-house to build this. And so, yeah, I think there is kind of a lot of this. That could be a good revenue driver for the law firms that you work with in terms of a new line of business that they can offer.

23:43Yeah. Some of them are starting to think about it that way. I was really struck by, it wasn't, I don't know if it was day zero, you can correct me, But it was within the first year where the very first version of Harvey was really an individual lawyer productivity tool. Right. I'm an associate or a more senior personal law firm. I want to get a piece of work done. Can you just make it less painful? But the transition quickly to like, how do we transform the business, make the business more profitable, like organized teams in the ecosystem, I think happened, happened pretty quickly. And like anything that is a business transformation just requires like, you know, a lot of engagement.

24:20Yeah. And so given how much you guys have invested in customer success and how that's like driven adoption, I feel like a big piece of it is just how quickly AI has happened. Yeah. Right. I would not necessarily have predicted that all the customers you're working with would be like, yes, in year one and two of this company selling, we're adopting. Yeah. But, you know, part of it is you guys are helping them. Right. Yeah. I think this was like, when I look back, it was still surprising how quickly some of these law firms adopted this. Like our first customer, we actually met through, you introred us to an ex-partner who's doing business school here.