Can we ever trust an AI lawyer?

0:28Support for this show comes from The Home Depot. Hello, and welcome to Decoder. I'm John Fort, CNBC journalist, co-host of Closing Bell Overtime, and creator of the Fort Knox streaming series on LinkedIn. This is the last episode I'll be guest hosting for Neelai while he's out on parental leave. We have an exciting crew who will take over for me after that, so stay tuned. Today, I'm talking with Richard Robinson, who's the co-founder and CEO of Robin AI. Richard has a fascinating resume. He was a corporate lawyer for high-profile firms in London before founding Robin in 2019 to bring artificial intelligence tools to the legal profession, using a mix of human lawyers and automated software expertise.

1:10That means Robin predates the big generative AI boom that kicked off when ChatGPT launched in 2022. too. As you'll hear Richard say, the tools his company was building early on were based on fairly traditional AI technology, what we would have just called machine learning a few years ago. But as more powerful models and the chatbot explosion have transformed industries of all types, Robin AI is expanding its ambitions. It's moving beyond just using AI to parse legal contracts into what Richard is envisioning as an entire AI-powered legal services business. AI can be unreliable, though, and when you're working in law, unreliable just really doesn't cut it.

1:51It's impossible to keep count of how many headlines we've already seen about lawyers using chat GPT when they shouldn't, citing non-existent cases and law in their filings. Those attorneys have faced not only scathing rebukes from judges, but also in some cases, even fines and sanctions. So I wanted to ask Richard about hallucinations, how he thinks the industry could move forward here, and how he's working to make sure Robin AI's products don't land any law firms in hot water. But Richard's background includes not just law, but also professional debate. Richard was the head debate coach at Eaton College, and so much of his expertise here, right down to how he actually structures his answers to some of my questions, can be traced back to just how experienced he is at the art of argumentation.

2:37So I really wanted to spend time with Richard talking through his history with debate, how it ties into both the AI and legal industries, and how these new technologies are making us re-evaluate the difference between facts and truth in unprecedented ways. Okay, Robin AI CEO Richard Robinson, here we go.

3:11Richard Robinson, founder and CEO of Robin AI. Great to have you here on Decoder. Thanks for having me. I really appreciate it. It's great to be here. I'm a big listener of the show. I'm going to be all over the place here, but I want to start off on Robin AI. We're talking a lot about AI in a lot of different ways nowadays. I started off my Decoder run with Cassie Kozarkov talking to her about decision science. But this is a specific application of artificial intelligence in an industry where there's a lot of thinking going on and there ought to be. So legal industry, tell me, what is Robin AI?

3:50What's the latest? We're building an AI lawyer and we're starting by helping solve problems for businesses. Our goal is to essentially help businesses grow. It's one of the biggest impediments to business growth is it's not just about revenue. It's not just about managing your costs. It's legal complexity. Your business is dealing with legal problems, actually slows them down. So we exist to solve that problem. We've built a system that essentially helps a business understand all of the laws, all of the regulations that apply to them, and also all the commitments that they've made, their rights, their obligations, their policies.

4:32We use AI to make it easy to understand that information, easy to use that information, to ask questions about that information, to solve legal problems. We call it legal intelligence. So that's how we're taking the latest AI technologies. We're taking them to law school and we're giving them to the world's biggest businesses to help them grow. A year and a half ago, I talked to you and your description was a lot heavier on contracts, but you said, yeah, we're heading in a direction where we're going to be handling more than that. It sounds like maybe you're more firmly in that direction now. Yeah, that's correct.

5:08I mean, we've always been limited by the technology that's available. Before ChatGPT, we had very traditional AI models. Today, we have, as you know, much more performant models. And so that's just allowed us to expand our ambition. And you're completely right. It's not just about contracts anymore. It's about policies. It's about regulations. It's about the different laws that apply to a business. We want to help them understand their entire legal landscape. Give me a scenario here, case study-wise, on the sorts of things that using your technology, your customers are able to sort through. And then recently, you guys have amped up your presence on AWS Marketplace.

5:52So there are a lot more different types of companies are going to be able to plug in Robin AI's technology to all kinds of different software and data that they have available. So case study, what's the technology doing now? How is that kind of hyperscaler cloud platform potentially going to open up the possibilities for you? We help solve concrete legal problems. So a good example is every day our customers, organizations, they want to know whether or not they're doing something that's compliant with their company policies. And those policies are uploaded to our platform. And anybody can just ask a question that historically would have gone to the legal team or the compliance team.

6:37They can say, I've been offered tickets to the Rangers game. Am I allowed to go under the company policy? And we can use AI to intelligently answer that question. Every day, businesses are signing contracts. That's how they record pretty much all of their commercial transactions. And now they can use AI to look back at their previous contracts and then use that to help them answer questions about the new contract they're being asked to sign. So if you're doing a deal with the Rangers and you've worked with the Mets in the past, You might want to know, what did we negotiate that time? How did we get through this impasse last time?

7:15You can use the Robin platform to answer those questions. So I got to go back to that Rangers game situation. Please tell me you're going to be able to do away with that annoying corporate training about, you know, can I take these tickets or can I not? If that could be just a conversation with an AI instead of having to watch those videos, oh my goodness, all the money. I'm trying. I'm trying my best. This is exact, you're hitting the nail on the head, though. A lot of this stuff caused a lot of pain for a lot of businesses, either through compliance and ethics training, through long, sometimes dull courses.

7:53We can make that so much more interesting, so much more interactive, so much more real time now that you can use AI technologies like Robin. So we're really working on it and we're helping solve a whole vast range of legal use cases that in the past you needed people to do. Are you kind of taking away the work of the junior lawyers? I mean, is there any, how is it, it's a little bit of a straw man I'm throwing up there, but how is it changing the work of the entry level sort of law student, intern, whatever, who would have been doing that tedious stuff that AI can perhaps now do? Is there higher level work or are they just getting used less?

8:37What are you seeing your customers do? I think what we've observed is that if you think about the past, if a business had legal problems, they would either send them to a law firm or they would try and handle them internally with their own legal team. With AI, they can handle more work internally. So they don't have to send as much to their law firms as they used to, because they essentially have this leverage to tackle what used to be quite difficult pieces of work. And so there's actually more work that they can do themselves now instead of having to send it outside of the house. And then there's some buckets of work where you don't need people at all.

9:19You can just rely on systems like Robin to answer those compliance questions. So you're right. The work is shifting, no doubt about it. For the most part, AI can't replicate. It's not a whole job yet. It's parts of a job, if that makes sense. So we're not seeing anybody cut headcount as a result of using our technologies. But we do think they have a much more efficient way to scale. And they're reducing their dependence on their law firms over time because they can do more in-house. But how is it changing the work of the people who are still thinking? AI goes first, basically. And that's a big transformation.

9:57But you see this in the coding space, which I think they got ahead of adoption in the legal space. But we're fast catching up. But a lot of engineers who are using these coding platforms, if you talk to them, they'll tell you, for the most part, they want the AI to write all of the code first. But they're not necessarily going to just hit enter and use that code. In production, they're going to check. They're going to review. They're going to question and interrogate it and redirect the model where they want it. Because these models still make mistakes. And so their hands are still on the driving wheel.

10:34It's just that they're doing it slightly differently. They're using AI to go first, and then people are being used to check. And pretty much everything we do, we make it easy for people to check our work. We include pinpoint citations, references. We explain where we got our answers from. And so the role of the junior lawyer or the senior lawyer is now, hey, use Robin first, and then your job is to make sure that it went correctly. It's been used in the right way. How are you avoiding the issue of hallucinations? We've seen these mentions in the news of lawyers submitting briefs, arguments to a judge that include stuff that is just completely made up.

11:19We hear about the ones that get caught. I imagine we don't hear about the ones that don't get caught. I know those are different kinds of AI uses than what you're doing with Robin AI, but there's still got to be this concern in a fact-based, argument-based industry about hallucination. There is. It's the number one question our customers ask. And I do think it's a big part of why you need specialist models for the legal domain, because it's a specialist subject area. It's a specialist domain. You need to have applications and people like Robin, who are not just taking ChatGPT or Anthropic and doing nothing with it, you need to really optimize its capabilities for the domain.

12:06So in terms of what we're doing, to answer your question directly, I mean, we include citations with very clear links to everything the model does. So every time we give an answer, you can quickly validate the underlying source material. That's the first thing. The second thing is we are working very hard to only rely on external, valid, authoritative data sources. So we connect the model to specific sources of information that are legally verified so that we know that we're referencing things you can rely on. And I'd say the third is that we're educating our customers and reminding them that they're still lawyers.

12:50You know, I used to write cases for courts all the time. That was my job before I started Robin. And I knew back then it's my responsibility to make sure every source I reference is 100 % correct. It doesn't matter which tool you use to get there. It's on you as a legal professional to validate your sources before you send them to a judge, before you even send them to your client. So some of this is about personal responsibility because AI is a tool. You can misuse it no matter what safeguards we put in place. So we have to teach people, don't rely exclusively on these things because they can lie confidently.

13:27You're going to want to check for yourself. Right now, all kinds of relationships, all kinds of arrangements are getting renegotiated globally, internationally. Deals that made sense a couple years ago perhaps don't because of tariffs or expected tariffs or frayed relationships. relationships and I imagine certain companies are having to look back at the fine print and say, what exactly are our rights here? What exactly, what's our wiggle room? What can we do? Is that a major AI use case? How are you seeing language getting combed through, getting compared from perhaps how it was phrased 20 years ago to how it needs to be phrased now?

14:11that's exactly right any type of change in the world triggers people to want to look back at what they've signed up for and you're right the most topical is is this the tariff reform which is affecting every global business for sure and people want to look back at their agreements they want to know can i get out of this deal okay is there a way i can i can exit this transaction because we entered into it with an assumption about what it was going to cost and those assumptions have changed. And that's very similar to what we saw during COVID when people wanted to know, hey, can I get out of these agreements given that there's an unexpected huge pandemic happening?

14:49We're seeing the same thing now, but this time we have AI to help us. And so, yeah, people are looking back at historic agreements. I think they're realizing that they don't always know where all their contracts even are. They don't always know what's inside them. They don't know who's responsible for them. And so there is work to do to enable AI to be effective. But we are absolutely seeing customers who are global businesses trying to understand what the regulatory landscape means for them. And that's going to keep happening every time there's regulatory change, every time there's new laws passed.

15:26It causes business and even government to look back and think about what they signed up for. I'll give you another quick example. When Trump introduced his executive order relating to DEI universities, a lot of universities in the United States needed to look back at, okay, what if we agreed? What's in some of our grant proposals? What's in some of our legal documents? What's in some of our employment contracts? Who are we engaging as consultants? And is that in danger given these executive orders? And we saw that as a big use case too. So I think it's permanent change is a reality for business.

16:01And AI is going to help us to navigate that. What does the AWS marketplace do for you? Well, I think it helps give customers confidence that they can trust us. I think when we had the cloud and businesses started to adopt, the biggest reason that it took time for that adoption, as well as change management, was concern about security. For business, keeping their data secure is probably their single most important thing. It's a never event. You can't ever let your data go insecure. But for business to really get the benefit of AI, they're not going to be able to build everything themselves. They are going to have to partner with experts and with startups like Robin AI.

16:42But they need confidence that when they do that, especially with their most sensitive documents, that they're going to be secure, they're going to be protected. So the AWS marketplace, first and foremost, it gives us a way of giving our customers confidence that what we've done is robust, that our application is secure. because AWS security vet all the applications that are hosted in the marketplace. So it gives customers trust. It's like Costco, it sounds like, right? I'm not a business like vendor or software company like you are, but this sounds to me like, I know if I go shopping at Costco, there are certain guarantees.

17:20I know their reputation because I'm a member, right? They curate what it is that they carry on their shelves. They sort of stand behind it. And so if I have a problem, I can just take my receipt and go to the front desk and say, hey, I bought this here and blah, blah, blah, blah, blah. And you're saying it's the same thing with these AI-driven capabilities in a cloud marketplace. Yeah, that's right. That's right. You get to leverage the brand and the reputation of AWS, who are the biggest cloud provider in the world. And that second thing you get, you mentioned it there too, is we get basically a seat at the table for the biggest grocery store in the world.

18:00They have lots of customers and a lot of businesses make commitments to spend with AWS. And they will first and foremost choose vendors who are hosted on the AWS marketplace. They'll go to them first. And so it gives us a position in the shop window to help us advertise to customers. That's really what the marketplace gives to Robin Ayo. We have to take a quick break. We'll be right back.

18:48How many secret spy missions would have ended in disaster? So why would you accept just anyone when hiring for your business? When you need the right person for the job, you can turn to LinkedIn Jobs. And now LinkedIn Jobs is stepping things up with their new AI assistant. So you can feel confident you're finding top talent that you can't find anywhere else. With LinkedIn Jobs AI assistant, you can skip the confusing steps and recruiting jargon. It filters through applicants based on criteria you've set for your role and surfaces only the best matches so you're not stuck sorting through a mountain of resumes.

19:23Hire right the first time. Post your job for free at linkedin.com slash partner. Then promote it to use LinkedIn Jobs' new AI assistant, making it easier and faster to find top candidates. That's linkedin.com slash partner to post your job for free. Terms and conditions apply.

19:50We're back with Robin AI CEO Richard Robinson discussing the landscape for legal AI tools. Now I want to take a step back, get a little philosophical about this. We got a little in the weeds with the enterprise stuff, but part of what's happening here with AI, and I think in a way with legal, is we're having to think differently about just how we navigate the world. And it seems to me that the two steps at the core of this are first, how do we figure out what's true? And then two, how do we figure out what's fair? And you are a student of a practitioner of debate. We'll get a little bit into that in a bit too.

20:32But figuring out what's true, I'm not a professional debater, though I have been known to play one on TV. Figuring out what's true is step one, right? Yeah, I think it is. It's increasingly difficult because there are so many competing facts and there are so many different communities where people will selectively choose their facts. But you're right. You need to establish the reality, the core facts before you can really start to make decisions and start to debate what you should be doing and what should happen next. And I do think AI helps with all of these things, but it can also make it more difficult.

21:13These technologies, they can be used for good and bad. And so it's not obvious to me that we're going to get closer to establishing truth now that we have AI. I think you're touching on something here that's interesting right off the bat, the difference between facts and truth. Yes, that's right. It's very difficult to really get to the truth. I mean, facts can be selectively chosen. I've seen spreadsheets and graphs that technically are facts, but they don't really tell the truth. So there's a big gap there. How does that play into the way we as a society should think about what AI does? AI systems are going out, training on data points that might be facts, but the way those facts or those details or those data points get arranged ends up determining whether they're telling us something true, right?

22:13I think that's right. I think that as a society, we need to use technology to enhance our collective goals. We shouldn't let the technology just run wild. Now, that's not to say that we should regulate these things, because I'm generally quite against that. I think we should let innovation happen to the greatest extent reasonably possible. But we should have a say, as consumers, we do have a say in how these systems work and how they're designed and how they're deployed. I think as it relates to the search for truth, the people who own these systems and who use them, they've grappled with these questions in the past.

22:57So if you want to Google search certain questions like the racial disparity in IQ in the United States, you're going to get a fairly curated answer from the Google search. And I think that that is in itself a very dangerous polarizing set of topics. And I think we need to certainly ask ourselves the same questions that we've asked with the last generation of technologies to this new technology, because that's what it is. AI is just a new way of delivering a lot of that information. In some ways, it's a more effective way. It's going to do it in a more convincing and powerful way. So it's even more important that we ask ourselves, how do we want information to be presented?

23:44How do we want to steer these systems so that they deliver truth, so that they avoid bias? It's a big reason why Elon with Grok has taken such a different approach than Google took with Gemini. If you remember, the Gemini model famously had black Nazis and it refused to answer certain questions. It was alleged to have some political bias. It was because I think Google was struggling to answer and resolve some of these difficult questions about how do you make the models deliver truth, not just facts. and they maybe hadn't spent enough time passing through how they want to do that. Well, I mean, yeah, Grok seems to be having its own issues.

24:26It is. Right? It's very much like people, right? Like somebody who swings in one way has trouble with certain things and somebody who swings in another way has trouble with some other things. There's the matter of facts and then there's what people are inclined to believe. And I'm getting closer to the debate issue here, but sometimes you have facts that you string together in a certain way, and it's not exactly true, but people do really want to believe it, right? They embrace it. And then sometimes you have truths and people just completely want to dismiss it. The quality of the information, the quality of the truth or confusion doesn't necessarily correlate with how likely your audience is to go, yeah, Richard's right.

25:16How do we deal with that at a time when these models are designed to be convincing, regardless of whether they're stringing together the facts to create truth or whether they're stringing together the facts to create something else? Well, I think that you observe confirmation bias with or without AI throughout society. People in search of facts that confirm their prior beliefs. There's something comforting to people about being told and validated that they were right. And whether you've got whatever technology you might use, that's just a baseline for All human beings, I think, is this desire to feel like they're correct.

26:07And so I think if you want to shape how people think, you want to convince people of something that you know to be true, you have to start from the position that they're not going to want to hear it if it's inconsistent with or incongruent with their prior beliefs. And I think what AI can do is it can make these things better and it can make these things worse, right? For people who are looking for facts that just back them up and validate what they already believe, it's going to make that much easier. It's going to give you the world's most efficient mechanism for delivering information of the type that you choose.

26:46But I don't think all is lost because I think as well, people who are trying to provide truth or help change people's perspective, show them a new way, we have a new tool in our armory to do that, right? We have this incredible research assistant called Deep Research that we never had before, which means we can start to deliver more compelling facts. We can get a better sense of what types of facts or examples are going to convince people using technology. We can build better ads. We can make more convincing statements. We can road test buzzwords. We can be more creative because we have AI. and fundamentally, we've got a sparring partner that helps us to craft our message.

27:32So I think AI is basically going to make these things better and worse all at the same time. My hope is that the right side wins. People in search of truth are able to be more compelling now that they've got a whole host of new tools available to them if they use them and they learn how to use them. And it's not guaranteed that people will learn these new systems unless people like me and others, people like you, are out there helping to proselytize on the benefits and capabilities these things bring. But it feels like we're at a magic show, right? Where the reason why so many illusions work is because the audience gets primed to think one thing, right?