Deal Velocity, Not Billable Hours: How Crosby Uses AI to Redefine Legal Contracting

0:00I think lawyers are quite good at learning, but in a law firm's structure, as much time goes into apprenticeship, it's a teaching hospital. You don't actually spend that much time getting really good at teaching, because you just do it through reps and reps and reps and reps and reps. And so actually explaining things, is something that I think is going to be a very prized skill for not just lawyers, but for any domain experts, but in particular lawyers. And we're seeing it, like, when you can make an AI, do this thing that you've been doing and not enjoying doing, it is like the most magical experience ever.

0:26And we're watching that like on a weekly basis.

0:46In this episode, we explore a fascinating approach to building services as software, with the founders of Crosby, an AI first law firm focused on legal contract automation. Ryan and John share why they chose to build a law firm rather than legal software. And how doing the work themselves creates unique telemetry and feedback loops that traditional e -vals don't capture. Ryan and John also discuss their approach to agent orchestration from paralegal level routing to senior associate level contractor view and share their vision for a Robinhood like future where anybody has access to high quality legal services thanks to AI.

1:20Enjoy the show. John and Ryan, welcome to the show. Thank you so much. Thanks for having us. All right, first question, maybe kick us off. What is Crossbe? Tell us a little bit about the company you've built. Great. So Crossbees and AI First Law firm, we focus entirely on contracts. So the theory is that we can automate human negotiations, so getting people to agree on terms. And the best way to do that is with a contract. Contracts turns out are everywhere. They're your lease, their offer letter and their under most business transactions. And it turns out the best way to do this and to automate it is by building a house to law firm, understanding how lawyers work and then trying to replace as much of that agentically to get a contrast close faster.

2:04And can you talk about decisions to build a law firm versus to build legal software? Because many entrepreneurs in your path, you know, not in the legal space necessarily, but have chosen the path to build software companies, why build a services company? So maybe I'll talk about structurally why we sort of chose this and then John you can talk about on a more applied level How we've got the team and how that works? So structurally, assuming that a little bit, you know the market for legal services has always put a huge premium on human capital And it's in some ways like a VC firm I think like you're really investing in the people who work there and getting them better and better and better And trying to keep those experts or your partners around for as long as possible And that made a lot of sense where a lot of that work was very specialized and couldn't really be offloaded to technology.

2:50And there's a growing sense in the last 30 years, but in particular, the last three that's changed. And you're looking now for a new model that allows you to actually innovate in the kinds of tasks you can offload from humans to technology. And it's just like has never happened in a law firm before. But all the raw materials are there. The experts in know how to build things are there, but a law firm partnership for example doesn't allow you to invest in speculative things like building technology because you can't sell equity and only law firm partners can take loans that are recourse so it just doesn't work.

3:22But we have this theory from a structural standpoint that if we get the structure correct and atrium proved this five years ago, you would start to see really quick iteration cycles in the way that you can actually innovate. So that's a theory at least. Yeah, and then putting that into practice, I think having your domain experts sit side by side with your engineers Create such a unique feedback loop where it's more than just evels It's actually using the product experiencing it doing user research and understanding what are the critical workflows? What are the biggest gaps and actually being able to you know recognize and say from start to finish like you know It's more than just a benchmark It's actually like an environment to create that opportunity to whether it's innovate on like these specific Like how do you review a contract faster?

4:06But also the individual bits and pieces of like hey, is this red line exactly correct or is that the right word choice for what we need here? I think you know historically people have attempted this they've combined some software as well as the services side together Tell us a little bit more about why some of these previous attempts might not have worked but also today, operationally, you have AI engineers sitting next to lawyers. How actually does that work? Like, how do you make that successful? So maybe on that first part of the question, why hasn't this exactly worked before? And what might be different now?

4:39Other than the obvious, which is the state of the art for AI, I think it actually hasn't been tried that much. And it's an extremely daring thing to do. And we've seen great innovation in accounting, actually, and management consulting. And when you even look at the corporate entities, fewer and fewer of those entities are partnership structures and they move to C -Corp. So that's a really good proxy for how much they invest in long -term technology. And technology we've had two or in law, we've had two or three companies, atrium and before that clear -spire actually try this in a meaningful way.

5:13And a bunch of things didn't quite pan out. One was that tight coordination, which I think John, you should speak more to, between lawyers and engineers, and we literally in our office have desks staggered, lawyer engineer, lawyer engineer, in order to get those feedback cycles and to instant them to really collaborate. And the other piece I think is, and this is what is truly unique, is that huge chunks of the qualitative, thoughtful work that lawyers are doing each day can actually now be delegated to machines. And this is like, it's hard to overstate how complex that is and how new it is. Yeah, and then putting that into practice, you know, I think that means two things.

5:51It means product velocity and it means actually instrumentation. How do you think about metrics? What we're doing is an operationally heavy business, right? You can actually instrument and think about we get a contract. It comes in. We send it out really quickly. We need to make sure we're meeting things like our SLA, which is measured by our turnaround time. How many human touch points were involved in actually every contract from start to finish? And if you can actually understand and say, hey, here are all the individual touch points. Here's how long each touch point took. And here are the highest value touch points we can automate today and just start hammering away at those.

6:27I think the other thing when you think about product velocity is it's more than just shipping product for the sake of shipping it. It's do these leading inputs actually lead to customer metrics because even though this is an operational, heavy business, you need to be thinking about what is the end customer goal that we're trying to solve for here. Ultimately our customers come to us because they trust us and we do high quality contract reviews for them. And you know, it's companies like cursor because they're growing so quickly. Like these companies are growing faster than ever before and we really need to just like be able to meet their standards and their bar of like what excellence and what speed looks like.

7:02And actually taking you know all this product velocity these input metrics and actually converting those into these lagging indicators of like are we meeting their SLAs? Are we delivering on time, are we reducing the number of human touch points? And then that's actually how you can drive the impact. Maybe I'll just add something that we've only started to like really get our hands around now. I think a year ago when John and I started working in this idea, we had a strong intuition that structuring this like a law firm was just accessible for companies who wanted to buy some sort of AI enabled service because there's so much excitement and people want to try it.

7:36But there's something subtle which is legal services, I think economists call it a credence good, which means that you only know how good it is after you've experienced it and consumed it. And you need an expert to actually tell you the quality, a layperson or even not a layperson, a sophisticated CEO often doesn't know by definition how good their legal work is. And so you want to know that a lawyer is looking at it. And so by structuring as this law firm, I think we're taking like the most promising aspects of what AI can do, but still putting all of the sort of right safety parameters around it and structures of having an expert tell you this is good quality, and this is safe.

8:16How do you think about innovating on the pricing and packaging layer? Like I think of law as one of the most interesting for the AI wave because it's one of the most technology disruptive, but also in terms of pricing model, if you're building by the hour, making it more efficient doesn't really help anything. And so say you're worried on how much pricing innovation you're doing. Yes, so this was an early decision that we almost made without thinking too hard about it. And it's just, and then it's just been a constraint, which is no billable hour. And that's like for us very dramatic, but for us it was so obvious that it would be an interesting wedge to get people's attention and it would align incentives within the company that we didn't think twice about it And so that looks like today is building by the document and I Think there's a few things to say about that the first is the billable hour people have been predicting the death the billble hour For like 70 years.

9:10It only became popular in the 50s. It's kind of novel But it's just really durable and it makes a lot of sense for really sophisticated work where it's hard to X and D predict how much work it will be. So I think part of what we're innovating on, which we didn't, we really could not have guessed a year ago, is being able to predict from time zero how long this piece of work is going to take, so that we can price it and still, you know, make sure that we're doing the work properly, that we align with the value that we're giving the clients. And you know, that's like where things get really interesting, because you have to predict, you know, how many times those contract go back and forth, we have to look at it five times, three times, two times, in order to price the amount of, you know, in order to price document properly.

9:51Yeah, so interesting. Yeah. And you've been a lawyer before when you say you guys are automating legal contract work. Maybe just take us through what a human lawyer does today when they go shooting contract. And what types of contracts are we talking? Are we talking NDAs? Are we talking merger agreements? Employment agreements? Everything in between. Yeah, so so many points I could I could go on. So they'll cut me off. So right now we focus on NDAs, MSAs, DPAs. I think every tech person will be very familiar with this because you're selling a B2B product. This is what you're dealing with. And I'd say NDAs are sort of from a complexity standpoint, you know, a good deal less complex than MSAs and DPAs.

10:35It's actually quite a step to get to the MSA DPAs. You know, NDAs, two pages MSA is 15, but they're probably what would you say, like, it to be 80 times more complex. There's just so many more terms to negotiate. And then you can go all the way up to the merge agreements, which is probably 1 ,000 X more complex. Lawyers today really try to predict in looking at all the terms in a contract, the right things you shouldn't agree to, based on mental models of what's safe and unsafe and kind of having to calibrate for your business. We want to get you to sign this contract. And so some of the clients we're dealing with are signing dozens of contracts a day, right?

11:10These are cursor and clay and unified that are growing so quickly, but you want to insulate the business from too much risk. And I think, so that's what lawyers do. And there's a lot of guidelines and general benchmarks consensus on like what is sort of market here. But it's really like a theory that exists in the minds of lawyers and two reasonable lawyers will disagree on what those things are. And this is a subtle last point that I'll just touch on, which is, you know, lawyers are essentially these private actors that are building a public good, which is the legal infrastructure, which is all the contracts that ever get negotiated, go into this public domain sort of of like what is a reasonable allocation of risk.

11:46And so by automating this, and this is where I think John gets really interested in the generalizing benchmarks, we can start to actually say quantitatively, here's what that infrastructure should look like. Not like here's my best guess of what's a reasonable contract, but like we can actually give you a metric or a statistical guess on like, Here's a risky or safe bet to take. Yeah, I think concretely taking the vibes and heuristics inside of a lawyer's head and converting that into an actual probabilistic quantitative number and saying like, hey, how likely should you be to accept Delaware or California or New York governing law when you're in the specific negotiation?

12:24I think there's something innately human about that in terms of that agreement. But at the same time, there's a lot of market data and you can actually start dissecting and understanding what are the levers you have to allocate risk in your business. And circling back to what founder should be thinking about as or thing about these businesses, which state you go to court in is there's only 50 states, thankfully, for now. Extending that into the long tail of what work you could do, I think it's really easy to confound PMF of your customers asking you to do this thing. and we'll pay you to do this thing versus like being an actual services business.

13:05Like you need to actually understand what is automatable, what is repeatable. What are the specific signals and levers you have to push that automation forward? Maybe let's dig into that. Actually, what parts of even the product today, what parts of it are the LLM or AI model, what parts of it are the lawyer, what parts are very more workflow software, and how does that change over time too? Yeah, I think it's been so interesting in the past, you know, five, 10 years, the amount of investment. I'm sure you guys have seen this firsthand. So I love your take on this too. Like the amount of investment in NLP research.

13:40It's just like, you know, problems that were state of the art five years ago are now like warm up problems in a Stanford freshman year class. Yeah, and I think it's just been so interesting to say like, okay, this is what tools we have available to us now. in terms of like zooming back out into like the specific models and what we can automate today, you know, it comes down to context. When Alex Net and all the first models were starting to do image classification, right, you had a picture of something and you would try to classify it, all you needed was the picture. You know, now you start thinking about like you're looking at an MRI and you're trying to classify it.

14:18Well, you're looking at the MRI, but you also might get some additional context from maybe the patient's history and some other metadata about the patient. I think language models are now taking that to the extreme. The language models say, basically, it gives me as much context as it's necessary to understand this problem and make some type of decision on it. And to answer your question around the workflows versus the language models, today it starts with, how can you context engineer for a human lawyer? How do you give all the right tools and these building blocks to expand what's available to the lawyer and actually automate their manual workflows that they were doing?

14:52Once you have those building blocks, then it becomes a question, okay, what are the parts that a lawyer is doing that you can now align the language model with and say, hey, how accurately can you replicate what behavior this human is doing? And I think my hot take here is I think everyone focuses on these general, like, you know, reinforcement, like RLHF and all these general purpose, like aligning it to the maximum amount of data available. Really, what you want to do in some of these specific scenarios is a line to a specific person, right? Because like Ryan said, like even getting two lawyers to agree on one specific contract may require a lot of back and forth intention, even if they work for the same firm.

15:33And if you can actually align it more accurately with one individual, you can start to say, like, this is correct, this is wrong, because that person is internally consistent, even though two people might disagree on that. So you would have almost individualized models for each of the different lawyers. Is that exactly. So I think the two the two PCs here are I think also fundamentally people are not thinking about per customer e -vows or per customer fine tuning enough I think per customer fine tuning and e -vows are a really high ROI way Especially if you have a large ACV to make your product reach multiple nines of accuracy One of the dangerous traps of language models today is they get to 90 % for basically free the foundation models have done an incredible job and we love our partners at OpenAI and Therapeutic and Google for giving us such wonderful tools.

16:22But one of the dangers is they get to 90 % and getting them to 99 or 99 .99 is actually extremely difficult. And part of the what all the levers you have are thinking about like what are the ways I can adjust these prompts per customer? What are the ways I can fine tune this model per customer? And then you can actually deliver you know, not just a four star experience, but a five star product experience because really that's what you're trying to achieve at the end of the day. But I'd be curious for you guys to read in terms of the amount of the investment for the language models and how you've seen it shape in terms of these per customer, these more enterprise -focused verticals.

16:56I think it partly depends on what data is actually in distribution for them, because I think whatever data is in distribution, they will, I think they are continuing to improve at a rate where they will probably get there very, very far. Versus, I think if your point is for a certain enterprise, most of their data, the really interesting data for the contracts, you will never have access to as a foundation model company and we could get access to individual specifically and I can create, you know, more like RLD fine tune model for you that understands your context. Like I think that makes sense because it is data that is not in distribution for the large model companies.

17:34I agree with that. Good take. You guys have some very discerning customers. I think I heard Clay and Kurser mentions. Tell us about what they like most about you. Is it that your smart AI people? Is it that you can turn around the contract extremely quickly? Is it that you can turn around the contract cheaper than somebody who's not using AI? Like, what is the customer facing value for? For these companies, we've really crystallized those over several months. It's the old velocity. And what we think a lot about is just the acceleration of startups today and everything is going faster, right? The sales motions are going faster.

18:18I mean, you look at Clay, right? Like they're enabling sales teams to move that much faster. The way that you hire everything and contract negotiations, which is like the sort of critical piece. It literally is the sort of way you plug in with your customers. The API for business is what we call it. is kind of unchanged for 40 years, basically since the word processor came out. And so this idea that we can unlock speed in two ways, from the time that they send us a contract to the time that they get it back, this is typically the AE or the salesperson, we're just unlocking their speed that they get back to their client, or also, doing reviews in a more thoughtful way, so that rather than doing five or six back and forths, we can sort of predict, okay, if we just agree to this term, and only push back on these three, we're going to save one turn.

19:02This will be a whole week faster. These are the main things. And what's kind of interesting is, as John was saying, sort of quality and taste are these somewhat intangible things that are threshold questions for these kinds of agreements. You have to hit some reasonable amount of quality. But I think most great lawyers, the GCs that I really admire and spend a lot of time with, understand their job as business drivers. And they are unlocking the growth of their companies in the position themselves that way. And so that's the key. And structuring ourselves with lawyers in the loop in a really clever way allows us to have the oversight of lawyers kind of all the safety and the quality, as I mentioned before, but see how much we can really push the limits on, you know, right now we do a contravene, median times are a little under an hour.

19:51Can we get that to minutes? Like we're still having, making sure the right terms get in front of the right lawyers at the right times. And these are like the fascinating questions. And what does it take to get there, you think? Like what needs to happen for that to be possible? So I think what's really nice, and I think the reason why John and I were so excited about doing this interview together and why we typically are speaking together is we have this really unique marriage of, you know, technical expertise, really interesting in the cutting edge of AI, as much as I love chat to BT. And like what lawyers are really good at.

20:22And what's interesting is like when John started working together, I don't know if this is a fair call out, you know, John was constantly asked for e -vails. And I was like, it's fine, we'll work on this, but I can look at it in 10 seconds, tell you, right? Like, just put it in my hands. And lawyers have a very visceral sense because it's taste -based based on their training based on their risk profiles, which is a little unique to each client, just immediately what looks good. And so the more we've been able to get, you know, prompting tools and actual sort of input and outputs in the hands of the lawyers that we have at Crosby, it's amazing just like, it's just velocity.

20:52like we had, we actually hired more engineers than lawyers at the beginning. And we were sort of, I felt like a little bit, like we were sort of revving the engine a little bit without clicking the gear and then lawyers came and it was like we stepped into gear. They just had people to give them all the inputs of, no, no, that's like right, that's sort of we should go that direction. So that's like a high sort of structural thing in terms of what it takes to get there. I do think that because lawyers provide almost an insurance to make sure that an expert said something looks good, they're always being a loop for a lot of physical work.

21:24There are like, and you know, I can get into hot tags on what kinds of legal work is going to be fully automated. But I think the key here is knowing exactly when to escalate something up to a lawyer and to get their thumbs up and to know that liability is covered. And you know, right now we have malpractice insurance. We take liability for all the work we do. I don't think we exist as a business if we don't do that. That's just that's the key here. Is we have to be so certain about the quality that we're able to stand behind it. So cool. Can you give us a peek at what's happening under the hood on the technology side.

21:52You mentioned, you know, you have three beloved foundation model partners. Like, what are you using each of the different models for and what scaffolding have you built on top? Yeah. I think what we found is the state of the art is not specifically tuned for contracts because there's not enough data in the corpus out there. Yeah, diversity of data is like better lesson, even if it's just for the legal domain, you want models that are trained generally. And these contracts are so hidden away. The best data set is Edgar. It's when you attach SEC filings and that data's just been so overutilized and doesn't apply to these small.

22:29So yeah, it's been an issue. I think the other part is in terms of infrastructure. Really constantly, like Ryan said, benchmarking this with all the environments we set up for these agents to understand what they're good at, what they're bad at and bringing that back to that quality score that I mentioned earlier. This is why I think every team needs someone, both the domain and the engineer. And I think this is why we'll see more of these vertical AI startups. It's because having that person in -house gives you such a competitive edge rather than maybe buying a data set from a large company that outsources a lot of like labeled data to contract lawyers or things like that.

23:07And then in terms of like, so, you know, the way it's set up today, right? We have lawyers. most of them have big log experience and are pretty proficient in these kinds of agreements who are driving, right? And so they're seeing, and this is what's quite interesting and to John's point, like the subtleties of the ways you change language within a contract are something that AI seems to struggle with a bit more than we'd imagined. So John and I like to joke, the difference between the term commercially reasonable and reasonable are actually subsidimally different things to a lawyer. They look very similar in embedding space.

23:39So those are like, and these are the new ones you really have to pick through. But what AI is amazing, so you have lawyers for driving there really measuring the kind of intervening for the quality on actual edits to a contract. But AI is great at summarizing, right? So being able to predict the right comment to make to explain a change. Interesting thing we learned about May, if you can give a really thoughtful explanation as to why we can't accept this language or we really are pushing for that language, the counterparty will get, will sort of understand what you're getting at and accept it.

24:09It reduces turns, right? So it's worth investing in AI to explain what you're doing. Another thing we realize is this is actually very specific to a company. These aren't legal questions. This is like, you know, some of our clients are not super sophisticated, but you have to understand the fundamentals of the cursor IDE or the fundamentals of how clay, you know, uses sales data. It's just like to, if you're a procurement person to know what you're buying. So this all, and this happens in the contract, right? And so you can unlock a lot of speed for lawyers to feed them all this data in the right place at the right time in the right parts of the contract.