The surprising case for AI judges

20:38Nilay Patel:Posting a job isn't always the hard part. It's finding, connecting with, and screening the right candidates. Hiring Pro streamlines the entire process from drafting your job to shortlisting candidates and conducting AI-powered interviews for initial screenings. Conversational Interface lets you describe what you need in plain language. No recruiter jargon needed. Nearly 60 % of hirers find a candidate to interview within a week. With Hiring Pro, you spend less time searching and more time connecting with the right talent. Hire right the first time. Post your first job and get$100 off towards your job post at linkedin.com slash partner.

21:19Nilay Patel:That's linkedin.com slash partner. Terms and conditions apply.

21:27Hi, everyone.

21:27Nilay Patel:This segment of Decoder Sessions features my boss, Helen Havlock, the Veritas publisher, and L 'Oreal Group's global vice president of tech and open innovation. I think you're going to enjoy this conversation.

21:37Bridget McCormack:We're going to start with a Decoder classic question, Giv. What does tech and open innovation mean at L 'Oreal? Who is on your team? What kind of projects do you work on? Open innovation is all the partnerships that we have in L 'Oreal working with startups outside. And it's really a great time right now to be doing open innovation because we're doing things in vertical farming and sustainable cultivation and biotech. So we do all those partnerships and our team is responsible for them. And the augmented beauty team is all the tech that started 15 years ago when we kind of had a blank page. And now how can we bring beauty and tech together?

22:11Bridget McCormack:How do you decide which projects to invest in? At the beginning, I was trying to push as much as I could to get people to think that beauty was relevant for tech. So we're really tech centric. And then over time, we started thinking about how to look more at beauty products that we can upgrade thanks to tech. And so we have a little bit more kind of process behind how we choose projects now. We try and kind of do things like upgrading the hairdryer to be able to do three out of four people have a hairdryer at home. And so how can we make it better? Or this year, like the flat irons that we're using and LED masks and stuff like that.

22:45Nilay Patel:So we do have a little bit of that kind of process, but we leave some space for serendipity and some creativity.

22:52Bridget McCormack:So we have scientists all the way to engineers and we let the scientists kind of think of some new clever ideas too.

23:03Nilay Patel:Support for this show comes from Doppel. Maybe that ping you just got is an urgent message from your CEO. Or maybe it's a deep fake trying to target your business. Doppel is the AI-native social engineering defense platform fighting back against impersonation and manipulation. As attackers turn to AI to power increasingly sophisticated strikes, Doppel uses it to fight back. Their digital risk management dismantles attacker infrastructure, while human risk management builds team resilience through simulation and training. With automated takedowns, multi-channel coverage, and AI defenses that build intelligence with every fight, Doppel works relentlessly to protect people, brands, and trust.

23:44Nilay Patel:Doppel, outpacing what's next in social engineering. Learn more at doppel.com. That's D-O-P-P-E-L dot com.

24:00Nilay Patel:We're back with Bridget McCormick, the head of the American Arbitration Association. Before the break, Bridget was explaining the nature of arbitration, how it differs from traditional court here in the United States, and how her experience as the former chief justice of the Michigan Supreme Court shaped her philosophy on judicial outcomes. Now I want to get into the heart of the matter, which is the ambitious project to bring large language models and agents to dispute resolution, something called the AI arbitrator.

24:29Nilay Patel:I'm starting with this issue of fairness because I feel like when you automate the systems, all of the things that make things feel fair or unfair get heightened, get magnified in very specific ways. And I just want to ask one more question here and then I do want to go into why should we automate some of these systems. The idea that just getting a hearing and some outcome is substantially more fair, I feel like we could unpack that for another week. There's a reason these companies want to not be in the formal justice system, right? They don't want precedent for the awards that they're made to give to the parties that come and sue them.

25:05Nilay Patel:They certainly don't want it in the public record that any of these things ever happen. They don't want discovery. All that stuff you don't have to do in arbitration. So maybe it's easy to participate in, but there's still a benefit to them that accrues over the long run. How do you balance that out?

25:19Bridget McCormack:Yeah, that's not true. of employee arbitration. Those cases are reported. So they actually do have to live with those results. And it may well be that for some businesses, they're choosing arbitration because it's a more efficient way for them to resolve disputes. I don't know. You see them going back and forth between arbitration and litigation. I think Amazon recently wrote litigation back into all of their consumer, maybe not all, but a lot of their contracts. Because I think, you know, and I'm not going to speculate about why, but I think parties are always thinking about what the best forum is.

26:02Bridget McCormack:You know, there's like a robust literature on procedural fairness, procedural justice that goes something like this. I won't waste time on the details of it, but it's if If parties feel like they were heard and that they understand the process, they understand what happened and why it happened. In other words, if the neutral deciding the dispute can explain it to them, they're far more likely to grow trust in institutions. It used to be a sort of a big deal in training judges. We would remind judges how important it was that people feel heard. And even if you're going to rule against them, they will take bad news for them and still grow trust in the institution if they feel heard and understood and they understand what happened.

26:42Bridget McCormack:And that matters because today's parties are tomorrow's witnesses and tomorrow's jurors, right? Like you want to grow trust. And, you know, in every case, usually one person is disappointed. That's just how disputes go. I'm not sure I agree with you that getting some award isn't important. There's data a bunch of arbitration scholars or really dispute resolution scholars did some recent work on employment arbitration. And the number of cases that just get summaried out in court where an employee has a claim or believes she has a claim against her employer is quite high, maybe not surprising to you.

27:22Bridget McCormack:And that just simply doesn't happen in employment arbitration. In arbitration, you're far more likely to actually get to go tell your story. I'm not sure I agree with you that there's like a clear fairness narrative based on the facts.

27:35Nilay Patel:The one thing I will definitely say here is in my mind, we cover big tech companies. I'm just thinking about the clauses that everybody has to sign in their terms of service agreements. And you're obviously thinking about employment arbitration. And there's like wildly different universes. Another week we could spend on is I think terms of service agreements should be illegal. But that's a different podcast for a different time, maybe with more booze. But you're mentioning the fact that people just feeling heard leads to trust, I think, is the transition I want to make to AI. You mentioned that idea to me the first time we spoke about this.

28:07Nilay Patel:I've been thinking about it ever since. If there's one thing an AI system can do, it is just make you feel heard. Like in every positive way that that can happen, in a startling number of new and quite frankly shocking negative ways, the AI systems will just listen to you over and over and over again. Is that something as you have developed the AI arbitration system, is that something that you've leaned into? Is that the heart of it? Because that feels like, well, if the technology can do one thing, it is just listen to these parties until they're done.

28:42Bridget McCormack:I do view it as a significant advantage of an AI dispute resolution system. And we can talk more about when that's appropriate and when it's not. I don't think every dispute should go to an AI dispute resolution system. But when parties prefer it, it was for me at the front end of what we built, which is a very narrow product right now. But I undervalued it and underappreciated it. So at the front end of our AI arbitration process, which is really a series of agents that operate across the process on the back end, even though the parties are interacting with one, the agents take in the party's complaints and their, you know, whatever pleadings they're filing and whatever evidence they think supports their claims.

29:30Bridget McCormack:and then a series of agents like parses the claims, the elements of each claim, the evidence that may or may not support each claim, but the parties believe supports each claim and the legal framework that surrounds it. And then it goes back to the parties and says, here's my understanding of what the claims are, you know, what your claims are party A and yours party B and what the elements are and what the evidence is and what the legal framework is. And did I get that right. And the parties get to say, you did, or no, you didn't. You missed this one element or this one claim or the fact that this evidence supports both of these claims, not just one.

30:07Bridget McCormack:And then it goes to work again. The agents go to work again until the parties are satisfied that they have been heard and understood. I mean, maybe we could do that in courts, but we would have to spend a whole lot more money. Like imagine if judges in trial courts or even in like appellate courts. In appellate courts, you file briefs and then you wait by your computer for months and months and months to see like when you when some white smoke emerges from the, you know, state Supreme Court building and then you get a decision and the decision may or may not have even addressed all of the issues that you raised.

30:42Bridget McCormack:You know, that happens all the time. You know, I reviewed 2000 cases a year, 2000 applications a year when I sat on the Michigan Supreme Court, and I can't tell you how many of them were, the intermediate appellate court didn't even rule on issue number three. They just ignored it, and we just want an answer. So imagine if courts could do that. Imagine if trial judges or appellate judges could pull the parties in and say, here's my understanding. Did I get it right? And then we could actually be satisfied that they did. That's probably not going to work in our, you know, bespoke, built-for 18th century norm system.

31:22Bridget McCormack:But it can work for disputes in an AI dispute resolution system.

31:28Nilay Patel:So let's talk about your actual product. Right now, I think it's just documents only in construction disputes, right? That's where the product is. Describe why you picked that, and then just walk me through step-by-step. How do you use this thing? Is it an app? Is it a website?

31:44Bridget McCormack:It's a website. Although I assume we will have an app version of it, and we have an app version of all of our services. You can log in on your phone, but you're logging into the case management system. So really it's two things. We built an AI native case management system for the AI arbitrator to operate on. That effort is time well spent because it will replace our legacy case management system for all disputes within two years. Like that's just going to be something everybody has an opportunity to benefit from, which saves time and money. But the AI arbitrator is, like I said, a bunch of different agents, probably 20, sometimes more, depending on the complexity of the dispute, that operate across the arbitration process.

32:26Bridget McCormack:A bunch operate at the front end. We talked about those that are parsing claims and organizing arguments and organizing claims, making sure the party is satisfied. And then there's a bunch of reasoning agents. And those reasoning agents take the summary of the dispute that the parties are satisfied with and start reasoning across it. And then there are agents that do a draft award. There's a human in the loop throughout. In the beginning, it's the parties. They are the human that are making sure that they're heard and understood. Then we have a cohort of construction arbitrators who serve as the human in the loop arbitrator for the reasoning and award drafting.

33:04Bridget McCormack:And it's the human arbitrator who ultimately issues the award and she makes any changes that need to be made so that she's comfortable with the award. We started in construction. We have a long history with the construction industry and a very good relationship with the lawyers and parties and arbitrators in that industry. It's an industry where arbitration is important because, as you know, in any big project, there are always going to be disputes. And if you have disputes along the way, you don't want your project ground to a halt. You want to be able to keep going. So they need speed and efficiency.

33:37Bridget McCormack:They're not in it for confidentiality. They just need to be able to continue to move forward. It's also an area where AI is already impacting the underlying business, right? The construction sites are being infiltrated with AI that's making what they do significantly faster and better. So it's an industry that we knew would be open to it. And we do so many of those cases. We had a good library of documents-only construction cases that we could ground our agents in. You know, our agents are ultimately operating off a handbook, but that handbook, we were able to build that and train our agents on those historical cases and with the cooperation and collaboration with a bunch of our construction arbitrators and lawyers.

34:24Bridget McCormack:So it was the right place to start because we knew they'd want it and we could work together with them.

34:31Nilay Patel:Let me ask you a really dumb question. What is a documents only construction dispute look like?

34:37Bridget McCormack:I mean, it could be a lot of things. There are a million different ways it shows up, but something was supposed to be completed on a certain timeline and wasn't. and who's responsible for that delay. There might be supplier disputes. There's all kinds of disputes that happen like in the course of a construction project that don't need testimony, right? You can decide them completely on the paperwork. And that was also important for us in starting this in a first offering of an AI arbitrator. We were not prepared to have witness testimony evaluated by agents. That might come one day, but we're not there today.

35:14Nilay Patel:Yeah. So you ordered the pallet of steel beams. It had a delivery date on it that showed up. You can mark the delivery date and you can say, OK, that's obviously later than what we said. You owe me some money and then you can.

35:27Bridget McCormack:And here's what the contract says about why that should have happened. And here's, you know, but here's my response. And and then it's pretty straightforward, like many disputes are, honestly.

35:36Nilay Patel:I 100 % feel like I'm a 1L back at the University of Wisconsin right now talking about construction litigation and contract disputes in this way. So you had the library of previous arbitration in cases that looked very similar to this. You obviously have the experience and the history with the industry. Yep. Did you have the software engineers you needed to build this? How did that work? Did you go hire out to do that? Did you hire in? Where did this come from?

36:05Bridget McCormack:No, it's a great question. So we started transforming our operation in early 2023. I took this job in September of 2022, but I didn't start until February 2023. And everybody knows that, you know, in November of 2022, we all thought like, oh, wow, what's happening? So I spent most of my six weeks off learning everything I could about large language models and trying to figure out what it was going to mean for the legal profession and therefore, you know, our little subset of the legal profession. And I was convinced it was going to be extremely impactful. You know, like you said, it can make people feel heard.

36:41Bridget McCormack:And that's a wake up call for any dispute resolution provider, right? Like that's obviously going to have an impact. So when I got to the AAA, we gave everybody enterprise chat GPT licenses. And I mean, everybody, like not just our engineers, but our caseworkers and our legal team and our marketers and our because I because I think with any general purpose technology, you need the domain expert domain experts to figure out where it's going to impact them. And we started building point solutions. And so our AI engineers were not historic AI engineers, but like everybody else, they learned pretty quickly.

37:12Bridget McCormack:And so we do have a very talented set of AI engineers with this particular product we built with a partner. We sort of, we did, you know, co-pilots. We built it with Quantum Black. Quantum Black is McKinsey's AI team. And we did, for the first four or five months, the QB engineers sat in seat one and ours sat in seat two. And then they swapped for the last five months of it so that our team then was driving. Because they left after we delivered the MVP. And now we're already building out the next products that are built on the same architecture.

37:47Nilay Patel:I feel like it's another full hour decoder on McKinsey giving its AI team a cool name.

37:52Bridget McCormack:I'm super interested in that one. Have me back for that one. Really interested in it.

37:59Nilay Patel:It's got to be really cool, you guys. So you have the system now. You have the platform and the frameworks. As you know better than anybody, how the platforms and frameworks are built in the beginning have a pretty big effect on where they end up. Yeah. And even just talking about witness testimony, right? Okay. Now there's a dispute on like, did it happen before midnight or after midnight? And you need the loading dock operator to say, actually, it was the next day. Like this document is wrong. It's the next day. Did you build a system that can take witness testimony?

38:28Bridget McCormack:So it can take witness testimony as long as it's in written form, which you may or may not know this. I didn't until I started in this job. A lot of arbitration disputes have written witness testimony. In fact, most cross-border disputes, that's how they take testimony. It's literally affidavits or just written expert reports. More and more in American arbitration cases, it's depositions. So it can do that, but it's not going to take—it's not going to say, have a witness show up on Zoom and we're going to listen to them and see if we think they're telling the truth or they're twitching a little in their eye.

39:04Bridget McCormack:And so, therefore, we know they're lying. It's not doing that.

39:07Nilay Patel:Does it kick that back to a human arbitrator anywhere in this system right now? Because, I mean, that's what you would have a human arbitrator do, right?

39:14Bridget McCormack:We do. There's a human arbitrator assigned to the case from the beginning. And anytime the parties want the human arbitrator to come in, the human arbitrator is ready and willing to come in. And you're probably right that there'll be some case where along the way the parties will decide that they need testimony that they didn't think they needed at the front end of the case. And that's probably going to be a case where the human arbitrator takes over because we're not having our agents do that for now. It's not not not in our roadmap today.

39:40Nilay Patel:How many cases has the system resolved so far?

39:44Bridget McCormack:There's one case in the system. We stood it up in November. The first case came in, I don't know, a couple of weeks ago. Everybody was so excited. So, as you know, in arbitration, both parties, they have to have an arbitration clause in the contract that says we're going to arbitration. So for now, we will not take any case unless both parties agree. And obviously, there's no businesses that have put it in their contract. Well, I hope now there are because we've been talking to people now about it for two months. But we'll start seeing cases once contracts start showing up that have it in their contract.

40:18Bridget McCormack:One of the things we've heard from a lot of parties is they're eager to use it as an early case evaluation tool. They want to be able to run it just on their own. with their, you know, they want to put all of their evidence up into it and get an early case assessment about where their case is likely to go so they can figure out whether they even want to spend any time in arbitration. So that's actually one of the next use cases we're building out, which isn't very hard based on what we've already built.

40:44Nilay Patel:There's a gap here that it just seems very striking to me. A party that understands it needs early case evaluation because it is a repeat player in very lucrative construction disputes and they just need to keep moving and sort of the cost of the settlement is enough and they can just keep moving. That's a pretty sophisticated actor. How do you bring all of this down to, well, most people can't even access the justice system? Is there a path? Yeah. Yeah.

41:10Bridget McCormack:Actually, in fact, you're right that many larger construction projects with extremely sophisticated parties on both sides have these smaller disputes along the way for which this tool will probably be perfect. Many of those disputes at this point, they just think are not even worth, you know, even the large parties who know how to access every dispute regulation system don't bring them because it's just not worth their time. And that'll change. This will give everybody an option to resolve every dispute. We're seeing, not yet again, because nobody has it in their contract yet, but our self-represented parties in commercial and construction cases are just on the rise and have been for a couple of years.

41:53Bridget McCormack:I mean, I follow our sort of case filings day by day. And the self-represented parties in commercial and construction disputes are right now, I mean, it's one month, so who knows, maybe it was a weird month. But January of 2026 was double what it was in January of 2025. So in fact, I think this system is kind of perfect for the smaller party construction disputes. And as you know, As you know, there are also lots of those smaller construction projects where people really don't want to have to hire a lawyer to go figure out how to sort it out. So I think it's actually already perfect for those disputes.

42:29Nilay Patel:The documents-only construction case, pretty constrained, right? And like things happen and there's a lot of documentation. What's the next one you think that is similarly constrained that you could bring the system to?

42:40Bridget McCormack:Yeah, I mean, it turns out in lots of industries there are documents-only disputes. I was talking to an in-house litigation leader at an energy company, and they have supplier disputes that are all documents-only disputes, and it's probably perfect for those. Payer-provider disputes, disputes between hospitals and insurers, which are, I'm sure you know, like an enormous docket. It's probably perfect for those if you can get them both to agree. You know, time value of money means some people like to hold on to their money and they're willing to have a dispute resolution process take a very long time.

43:17Bridget McCormack:But those should be perfect for an AI dispute resolution process, right? We should be resolving those quickly and getting people their coverage. So I think there's actually an unlimited, you know, number of disputes for which this is appropriate. There are also some for which it will never be appropriate.

43:35Nilay Patel:Where will it never be appropriate?

43:36Bridget McCormack:I mean, this is just my view, but I believe that criminal cases, cases where the government is accusing you of something and, you know, wants to take your liberty as a result of that accusation, or cases that individuals or organizations or even businesses bring against the government should happen in public courtrooms and with publicly appointed or elected if you're in state court judges. Those have to happen in courtrooms.

44:09Nilay Patel:One of the things you and I briefly talked about the first time we met and talked about all this was the idea that there's transparency in the state courts. There's transparency when the government sues you, and you need that transparency to build trust. Like you brought up the healthcare industry. There's zero trust in the healthcare industry, and there's a lot you could say about the consequences of the absolute lack of trust in the healthcare industry. If my medical billing goes to an AI agent, do you think that's going to result in more or less trust?

44:34Bridget McCormack:Yeah, I don't know. It depends on the AI agent who built it, what their audit trail looks like and how transparent they are about showing you what that audit trail looks like. Let me go back for a minute. I said payer-provider disputes. I'm talking about disputes between healthcare, hospitals, and insurance companies. Do you think there's a lot of confidence or trust in insurance companies? Because I don't think so. No, not at all.

44:58Nilay Patel:That's kind of what I'm saying.

45:00Bridget McCormack:Yeah. Yeah. But I mean, so imagine, you know, disputes between those two that could be resolved instead of taking two years. So somebody is waiting for their insurance company to tell the hospital that they're going to cover their treatment and they're dead before they get an answer. Imagine if we could just resolve those right away. But in a system that transparently shows its work and shows its audits. Right. You tell me. My guess is some people would say, like, I'll take that tomorrow. Right. I'm waiting for my treatment. Yes. Please send my dispute to an arbitration process that is transparent and shows its work.

45:36Bridget McCormack:Great.

45:36Nilay Patel:I think some people would. Yeah.

45:38Bridget McCormack:Yeah. Some people would. And as I'm sure you know, there are lots of things that happen in public dispute resolution systems or in arbitration processes where the results are reported, where there's not a lot of transparency. Like, do you understand how judges come to the conclusions they come to? Like, we don't get to see the reasoning that goes on in their brain. There's no requirement that the judge shows her work and how she got from step A to step D. You know, what was her thought process? I mean, there is a hopefully, you know, most judges are supposed to write opinions that should show some of that.

46:10Bridget McCormack:But as you know, in most intermediate appellate courts, there's a lot of opinions that are not written. And in trial courts, very often there's no written opinion. So I always want to ask, as opposed to what? Like, if you believe that the current human-led overburdened justice system is one where people have lots of confidence, I want to introduce you to some folks who might disagree.

46:35Nilay Patel:Look, I was told that Samuel Alito can look into George Washington's heart and soul and determine exactly what he meant. And that's just, I've been operating under that assumption for some time.

46:47Bridget McCormack:Yeah, I mean, there is that. So if you think that's a system that instills confidence, like maybe, but I'm not sure everybody would agree with that. Yeah.

46:59Nilay Patel:We need to stop here for another quick break. We'll be right back.

47:10Nilay Patel:After decapitation strikes against Iran's leadership, what can we expect next in the escalating war? The big question is, if there is going to be a next strongman in Iran, what kind of strongman will that person likely be? I don't think that there's going to be another powerful cleric, supreme leader. I'm John Feiner. And I'm Jake Sullivan. And we're the hosts of The Long Game, a weekly national security podcast. This week, we sit down with Kareem Sajipour to discuss what to expect in this next phase of the war against Iran. The episode's out now. Search for and follow The Long Game wherever you get your podcasts.

47:49Nilay Patel:Are Democrats their own biggest problem? You know, a party becomes defined by who their central figure, who their quarterback becomes. Democrats haven't really anointed a effective quarterback since Barack Obama, pretty much. And this week, the Atlantic staff writer Mark Leibovich joins me to discuss the state of the Democratic Party and which races to keep an eye out for this midterm election. The episode is out now. Search and follow Stay Tuned with Preet wherever you get your podcasts.

48:24Nilay Patel:We're back with Bridget McCormick, the head of the American Arbitration Association. We were just discussing the AI arbitrator tool and the way she believes tools like it can actually increase trust in the judicial system at a time when trust in institutions continues to drop. Now, I want to focus on the ways in which the system could go wrong, through hallucinations, biased outcomes, and the many ways you could obviously see a tool like this having adverse effects. And I wanted to ask Bridget how she and her team are trying to protect against all this, especially now as the AI arbitrator is starting to hear real cases.

48:57Nilay Patel:So the flip side of this, and this is us covering AI at The Verge for years now, it'll just talk to you. And some people are very happy with that, and that is far more trust than they have in even the other people who live in their houses, right? Like we see that play out all the time. The downside of that is that these systems hallucinate at high rates, that they are tuned to please you. And we can see that all over the place, that they work differently, they layer differently. In fact, the chances of them getting something wrong kind of exponentially increase as you stack them up in these ways.

49:30Nilay Patel:How have you protected against that here? Because it feels like, yes, you can increase this sense of agency and trust. But the downside is this thing might just be making it up as it goes along.

49:41Bridget McCormack:Yeah, huge issue, obviously. If you were to just take your dispute, all the documents in some dispute you're having, and throw it into ChatGPT or into Claude, you could get a result right now, right? Anybody could do that. It might be okay. It might, in some cases, not make a mistake. As you know, hallucinations are, you don't know when exactly or why the frontier models are making mistakes. But that's not what we built, and that's why we're moving so narrowly and so slowly. Your system has to be governed, trained in and grounded in the kind of reasoning that you're asking it to do. So it sounds, you know, extremely narrow that we're doing only documents, only construction cases.

50:34Bridget McCormack:But there's a reason for that. That's where we could build a governed and harnessed, you know, agentic system. We keep a human in the loop to make sure that before an award issues, there were no hallucinations, even with our governance. And we're going to be very transparent about all of our audits. And that's, I think, really critical to growing trust. You know, I think a lot about whether the frontier models get so good eventually that they can just do this, that you don't need, you know, an institution training and governing an agentic system in a specific kind of dispute because, you know, we move past the age of hallucinations.

51:22Bridget McCormack:It doesn't sound like that's imminent today, but, you know, I don't know if it's imminent six months from now. I mean, I didn't know, you know, that I didn't know last weekend, last week that agents were going to be on a subreddit talking about us. You know, I wasn't, And I didn't know Maltbook was coming, right? So you can surprise me. For now, I think you have to have governed, transparent, audited systems so people can grow that trust. We have a white paper. We led an academic under the tent, John Choi, who's a law professor, but he's a technologist as well. And we just led him under the tent to kick the tires of what we were building to be able to test how it performs.

52:06Bridget McCormack:against the human baseline, and the results are excellent. So I'm excited for folks to see that when it's all ready to go. But it's an important question. Again, as opposed to what? You've met humans, right? And you've met humans who are judges. If you think they're getting it right every single time they make a decision, I want to introduce you to some folks. Let me just, this is such a, anecdotal stories are kind of useless when we're trying to talk about something important like this, but I was the chief justice of, you know, the Michigan Supreme Court. I've been in that role like three years.

52:41Bridget McCormack:And we were working on some reforms in the probate court system. And some of the probate court judges, I guess, didn't like them. And one of those probate court judges from a county not that far from where I live got on his listserv and was like just talking trash about me. He said that I was at a forum and I had said a bunch of stuff and like how outrageous it was. Fine. I want people should criticize me whenever they want. The thing was, I wasn't at that forum. My colleague, Megan Kavanaugh, also a white woman with an Irish name, was at that forum. And I know Megan, she probably did say those things.

53:16Bridget McCormack:So I called the guy. I said, Judge, you know, you made this claim about me on the listserv to every probate court in the state of Michigan, but I wasn't there. Like, I wasn't at that forum. I have an alibi. And you know what his answer was? His answer was, yes, you were. what do you do with that? Like, I mean, what do you do with that? He won. I was like, well, I don't know. Okay. I mean, you know, what do you even do with that? So if you think that the human beings who get tired, who get hungry, who like all of us come to the table with cognitive biases are getting every single thing right, then you have a lot more trust in the public justice system than I think most people do.

54:03Nilay Patel:I don't know that I have a lot of trust in public justice system. What I know about those guys is that they get old and sometimes they go away. I wish more of them would go away at higher rates, but sometimes they go away. Sometimes they get replaced by newer, different people with different biases and the system replenishes at some rate that feels like accountability, right? You can ruin that guy's reputation if you would just say his name. Bridget, I invite you to say his name out loud right here if you would.

54:28Bridget McCormack:His name was

54:29Nilay Patel:It was worth a shot. Go ahead.

54:33Bridget McCormack:Nah, not important.

54:34Nilay Patel:It was important enough for me to ask. Anyway, that's a system that people understand about humans, right? They have reputations. They have histories. They have experiences. You can Google them. And then sometimes they die, right? And they just go away. You get some new ones. And at least the system replenishes and evolves. An AI system running on a cloud service that you can't see on a data center that you might have hated being put in your community does not feel accountable to you in that way. And maybe it's getting it right more often on whatever metric that someone who is not you has decided is important, but you can't actually hold it accountable.

55:11Nilay Patel:And that to me feels like the biggest gap in all of these automated decision systems that no one wants to account for because the efficiency gain is so high.

55:20Bridget McCormack:Yeah. I mean, I think that's a completely fair point. And again, I go back to, you know, where I believe disputes should be decided publicly. And I think that that should be true no matter how excellent. I mean, eventually, probably the judges should be using some tech to help them make sure they don't make mistakes that will undermine accountability and undermine trust. But not all disputes are created equal, Neelai. I mean, we all have lots of disputes that some of which we just think like, well, I don't even have the time or the energy or the resources to get that one resolved. But if we did, if you could really resolve every dispute, if every human being could have a will or a plan for what happens if they become incapacitated or die, if every small business could plan for the disruptions that befall every small business, we'd have a better world.

56:21Bridget McCormack:I mean, you know, disputes are not great, right? Like, they make a mess of economic relationships. They make a mess of personal relationships. And resolving them, in fact, does lead to, you know, more peace, more stability, more economic growth, if that's your thing. Like, resolving disputes is good. Right now, we do not have any way to resolve most disputes because our one-to-one bespoke built-in 1776 system is no match for, you know, the kind of disputes we have today and the volume of disputes we have today. and it hasn't been for four decades. You know, it's been a very long time since public dispute resolution systems have been able to address the problems in their local communities.

57:10Bridget McCormack:So why wouldn't we have more options so people could choose the right option for them in the right moment? There would be some family disputes where parents prefer not to go to court. They want to work things out privately. They can do that now. They can go to mediation and just file a resolution, that to me seems like a good idea to be able to offer people a way to move through a dispute so that they come out on the other side both better with respect to each other, better with respect to the other people who are impacted by the dispute, and even feel like they were able to manage that with some agency.

57:51Bridget McCormack:I don't know why we wouldn't want to have more options for more people to give them agency.

57:55Nilay Patel:My wife, as it happens, is a divorce lawyer, And I do think an AI agent of her just talking her clients through the decisions that were made a long time ago over and over again would actually be very helpful for her in many ways. Because that seems like a lot of her job. I understand what you're saying there. You said this thing to me the first time we met that, again, I've just been thinking about ever since. You said that several years from now, 10, 20, 30 years from now, we would think it was crazy that we ever had human judges making as many decisions as we do today.

58:24Bridget McCormack:Yep.

58:25Nilay Patel:You know, you've got one case in the system. Is that borne out? Do you feel that as strongly as you did when we first talked a few months ago?

58:32Bridget McCormack:Yeah, I mean, I don't know exactly when the world turns and we're finally able to manage lots of disputes in different ways, including AI dispute resolution systems. But I feel pretty confident we will get there. I think I said to you at that time, and I still agree, I mean, for sure in some number of years, we will think it's amazing that we let humans drive cars, right? When my kids started driving, it was the most terrifying time of my life. Not really for them, but I couldn't believe we were letting them out onto the public roads with all these innocent people out there driving cars next to them.

59:07Bridget McCormack:I think at some point we'll think that was insane that we let people drive cars. And I think we'll also think it was probably crazy that we thought a human being had to oversee the disputes between private parties who want to be able to move through that dispute and be at a better place. That inserting a human with her flaws and her biases and her limitations was no way to manage this railroad.

59:34Nilay Patel:One thing I've been thinking about throughout this conversation is who gets access to these systems? Who gets to make it feel fair? Where does the trust in the agency come from? A pervasive criticism of arbitration, broadly, is, well, it's a service, right? It's fee-for-service even though the AAA is a nonprofit. There are other, I would say, more rapacious providers of arbitration services that do run them as for-profit businesses. You have clients. The clients have to be happy with the outcomes. And that does feel like it changes how people perceive the whole process. As you automate that, right, and you have big clients who are paying for lots and lots of arbitration, and they can see that the automated system is either helping them or hurting them, should that affect how people think about the fairness of the system overall?

1:00:19Nilay Patel:These are fundamentally your clients and you're building a tool for them. Yeah.

1:00:23Bridget McCormack:I mean, this is where I feel very, very lucky that we have a nonprofit mission. And our mission is to expand access to alternative ways of resolving disputes to as many people as possible. But we're like a court, right? We do serve parties, but there's two parties in every case. There's parties on both sides of every dispute. And we need to make sure both sides, both parties feel satisfied with the dispute resolution process they got. I thought you were going to say as the cost comes down as a result of even sort of AI automation for parts of disputes or full disputes, depending on the dispute, it's therefore harder for dispute resolution providers who need to make money to make money on that process that that would be a problem.

1:01:14Bridget McCormack:And I can see how it might be. But again, I'm really happy that at the end of the year, I don't have a bunch of owners that are looking for their profits. We have a mission. If we can bring the friction and the cost of dispute resolution services down significantly and therefore offer it to a whole lot more people, we're serving our mission.

1:01:41Nilay Patel:Yeah, but let me just contrast that to your previous role in the state courts as the chief justice in Michigan. The Michigan Supreme Court's belong to everybody in Michigan, right? No matter who the parties are, they're yours. You're paying taxes. You walk in and maybe they work and maybe they don't and maybe no one knows who was at the function saying what. But like it belongs to you in some way equally. In arbitration, there was recently a case that like Disney said to somebody to try to sue them in a theme park, you sign the arbitration agreement for Disney Plus. Like we're going to arbitration.

1:02:11Nilay Patel:That's pretty one-sided. That forum does not belong to both parties equally, right? And that imbalance seems like where the loss of trust, at least in the consumer side that I cover so much, comes from. How do you make sure, even as a nonprofit, that the party that is paying for and driving the system doesn't make sure the automated system, in particular the automated system, doesn't favor them over time? Because that's a pretty easy outcome to start programming in.

1:02:37Bridget McCormack:Super good question. As you know, easier to train a data set than a human. So frankly, and let me step back for a minute. In our consumer cases again, we do not accept cases, consumer B2C cases, unless a business has cleared their clause with our due process protocols. So we're, again, in a lucky position there, I guess. But back to the, you know, who's in charge. You know, for us, if the consumer feels like the process wasn't fair, that's not going to work. You know, we're just, you know, we're in the business of actually, you know, giving more options to more parties. And if half of the parties in one kind of dispute type, I should say, we mostly do B2B cases.

1:03:30But when we do B2C and they've satisfied our due process protocols, it's critically important to us that both sides feel in the individual case that they were treated fairly.

1:03:42Bridget McCormack:If it's algorithmic, I mean, if it's, you know, automated to some extent, even in part, that's just as true, but easier to show our work. You know, we did a lot of training, judicial training, and we do a lot of arbitrator training when human arbitrators decide cases to make sure that parties get, you know, equal treatment, fair treatment. But you can de-bias a data set a lot easier than you can de-bias a human. And so when you benchmark and, you know, do your audits of your AI arbitration system and you show your work, you can either convince the public that it's treating both people fairly or you won't.

1:04:18That's for us to show.

1:04:20Nilay Patel:So as you can tell, I'm fascinated. The reason I keep poking at this, our readers at The Verge, like every other consumer in modernity, you just live a life of signing contracts. And you don't get to negotiate those contracts, but you literally cannot participate in society without having signed dozens upon dozens of arbitration agreements. And I hear you say we have to get it right. We can't get it wrong. What is the mechanism of getting it wrong and then being forced to change when no one can negotiate the contract that landed them in arbitration in the first place? Do you know what I mean? Like if there's some public accountability that's like the AAA got everything wrong for a year with the AI system, then what?

1:05:01Nilay Patel:Because I can't go to all of my service providers and say I don't want to use the AAA anymore. I saw this news report.

1:05:07Bridget McCormack:Yeah. Yeah. I think you're asking, like, when you enter into any kind of B2C contract, you're buying a new refrigerator, you don't really get to scrutinize it and say, I want to, you know, change the last clause. You know, people aren't really entering into arbitration agreements. Arbitration clauses are just part of contracts they're entering into.

1:05:28Nilay Patel:In the construction case, right, you've got two parties, and presumably they've contracted for the sale of lumber. and they've come to, there's actually an actual like meeting of the minds and they might've negotiated in, we need arbitration because we've got to move fast. And it's fine. Automate that away. And if you get it wrong, maybe the next time you try something else. Right. And there's, there's some mechanism of change built into that. I signed my, the terms of service, LG refrigerator, which does have an arbitration agreement. And I'm like, that sucked. My refrigerator exploded and the arbitrator ruled against me.

1:06:00Nilay Patel:The next time I buy a refrigerator, I can't go to LG and say, I want a different arbitrator. And so I don't see where that mechanism comes in for most people because most people are just signing tons and tons of arbitration agreements all the time.

1:06:12Bridget McCormack:Yeah. You don't mean I want a different arbitrator. You mean, because you can't go to, I mean, if you're buying a refrigerator from Joe Refrigerator Store on your corner that doesn't have a contract with an arbitration clause and you end up, you know, suing in court, you're going to go probably to small claims court. You're going to have not been able to figure out how to file properly and your case will have been dismissed. I'm telling you that's what happens in most cases where people try and file cases in court. Are you going to be able to go back to Joe? I would have gotten it right. Well, you would have.

1:06:48Bridget McCormack:That's fair. But, you know, most people didn't go off the wall. And they aren't going to get it right. And they are most often never going to make it past go. I mean, look at the high-volume dockets in state courts. Just take a look at how many cases on the consumer debt docket, which is, by the way, the modal case in state court right now. Literally, the modal case in state court is a consumer debt case. and go take a look at how many of those are just dismissed because the consumer, not the business, didn't meet some, you know, I don't know, court rule 26.4a that said you had to have a triplicate when you, you know, responded to the thing.

1:07:29Bridget McCormack:You're not going to be able to go back to Joe's refrigerator service and say like, hey, that court process wasn't fair because there's no way I'm going to be able to read the Latin and figure out that I have to file and triplicate my thing. So I want to go to arbitration where at least I can just show up and in plain language tell a person what happened to me and a person will listen to it and answer it. I mean, I understand that there has been like a pretty successful narrative that this process doesn't work for individuals. And I am sure there are places where it doesn't. But not every arbitration provider and dispute resolution provider is created equal.

1:08:04Bridget McCormack:And some have different missions than others. And that matters, right? So I think, you know, there could be a campaign to say to, I have no idea who, you know, I don't know if LG uses us or some other provider, to say to LG, like, we want you to switch to a provider that actually has due process protocols. Like, that's what we want you to do. Or you could go to Congress and, you know, that you could tell the tell Congress to amend the FAA to say no B2C cases, B2C cases should ever go to arbitration. I I understand that, you know, you can apparently that's how legislation works, that you can talk to Congress and they'll fix things.

1:08:49But I don't know.

1:08:50Nilay Patel:There's a lot of hopes and dreams in this episode.

1:08:52Bridget McCormack:Yeah, yeah, yeah. And I have no horse in that race. If Congress were to say tomorrow, like, we don't want B2C cases in arbitration, we want them all in courts, I think it would probably be a mistake because I've seen how courts operate and the resources they don't have. Believe me, there are lots of judges who are trying to do really well by all those folks on their dockets. They just don't have the resources or the time. And they have these archaic rules. So I think it probably wouldn't be good for everybody, but I certainly have no horse in that race. Like my view is I can create dispute resolution services and processes just to give everybody more options.

1:09:26Bridget McCormack:And I want people to want those options. I don't want people to like be stuck with those options. I want to create something that's so good that you want it. You want to go to LG next time and say, can you please put an American Arbitration Association clause in my contract? Because that's where I want to resolve my dispute.

1:09:43Nilay Patel:I appreciate the ambition. It's a lot to think. The idea that I'm able to negotiate the terms of service from my refrigerator, maybe that is what I think Congress should allow us to do. But that's a different – yet again, a different podcast. When you think about the scope of this, the timeline of the investment that you're making, you've got one case in the system now. We're going to see how it goes. You've obviously run some simulations on previous cases and you think that they're going to – you've got some studies that say it's going well. How fast does it go from here? What's next? Is it 10 cases?

1:10:15Nilay Patel:Is it you're on a sales trip? Is it what's the speed?

1:10:19Bridget McCormack:Such a great question. I mean, so there's at least two variables that that that we're following. One is just how quickly our team, our engineers and and and the related folks that work on that scrum team can build it out for different case types. Right. Different documents, only dispute types. And then whether there are some institutions that want to use it for internal dispute resolution. We've had some some interest in that. That's going to take a while. You know, that's going to take us a couple of years to build out each, you know, dispute type by dispute type. So that's slow. And then there's the when do people, you know, when does the worm turn?

1:10:57Bridget McCormack:And, you know, your guess on that is as good as mine. I don't know if it's two years, five years or 10 years. I'd be very surprised if in 15 years people are still opting for a slow, human-driven, and people, businesses in particular, B2B, you know, documents-only disputes, are opting for a slow, expensive, human-led process. I'd be very surprised. But that's a really conservative answer to your question. Do you have a better answer? When do you think we're, how quickly do you think, I mean, it's going to go with a lot of the rest of the way the technology is going to disrupt our lives, right?

1:11:40Bridget McCormack:It's all kind of connected. Yeah.

1:11:42Nilay Patel:I mean, if you look at our stack of guests over the past few months, we just had DocuSign on the show. They're like, AI is going to write the documents for you because we have access to your business intelligence. We have LexisNexis on the show. He's like, the lawyers will just start doing the research here and drafting out of these claims. And I don't – that seems very dangerous to me. But you see this universe of legal work is being automated very quickly because the AI systems are good at words. Like you can either hire a 26-year-old who's been drunk for several years. That was me. And they can be whoever they are as a first-year associate.

1:12:18Nilay Patel:Or you can have a robot do it. And like maybe that's the same. I think the question that I have in sort of a response is I think it will happen very fast and then it will slow down because everyone will realize there's not a next generation of people who understand how to control the systems. And that investment still needs to be made. and the outcomes of those systems are not as good as we wanted them to be.

1:12:39Bridget McCormack:It's a completely different training. You know, my podcast is on AI and the future of law, and I'm just talking about, and you might know, two careers ago I spent 15 years on a law faculty. So the training model is kind of broken for what lawyers are going to do in, I don't know, again, if it's two, five, or ten years. And I don't see anybody moving extremely quickly to figure out what the 2.0 training model is. Now, I want to say, I don't think the 1.0 training model was that great. I'm not sure that sitting in the basement looking through a bunch of boxes of documents really made you an excellent lawyer or strategist or advisor.

1:13:13Bridget McCormack:I'm not convinced that that was perfect. So I do think there's opportunity. I'm an optimist. I think there's opportunity to build a better system. But you're right. I mean, there are going to be ways in which some of this moves faster than the rest of us and we're not ready for it. I mean, one other thing that I spend a lot of time thinking about is how many B2B contracts are going to be negotiated and executed by agents. Like Walmart is doing some significant number, has agents negotiating and executing a significant number of their contracts. In a B2B agentic commerce world, and so you probably read these estimates as much as I do, and maybe some people say that by the end of 2027, as many as 40 % of contracts might be negotiated and executed agent to agent.

1:13:56Bridget McCormack:Sometimes agents will make mistakes, right? Like, I mean, as good as they are, they will make mistakes. What's the dispute resolution process there? And is it just, is it a, is it a on-chain process that, you know, maybe it's better? Like it's an upstream automated dispute resolution process. I'd like to be in the conversations to figure that out because you need that somewhere, right? Your agentic commerce is only as good as your process for fixing it when it breaks. But nobody's talking to me about that. I want them to. So So, yeah, a lot to do yet.

1:14:31Nilay Patel:Literally, just before we sat down, Goldman Sachs announced that it was putting AI into more of its accounting functions and other functions at the company. And you can see the acceleration. I think my prediction is we're going to accelerate into it and then we're going to pump the brakes really hard when we realize these systems are not as predictable as we want them to be. But I will have you back faster than 15 years to see how it's going. I've loved this conversation. Thank you so much for being on Decoder, Bridget.

1:14:57Bridget McCormack:Yeah, so much fun. Great to see you. Thanks.

1:15:27Nilay Patel:Kate Cox-Nikstat, it's edited by Ursa Wright. Our editorial director is Kevin McShane. The Decoder of Music is by Breakmaster Cylinder. We'll see you next time.