On Campus Hiring Is Broken, And This Isn't Helping

15 Oct 2025 · 38 min · 12 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Big law hiring and AI reliability. The episode argues (1) Kirkland’s decision to not publicize its new partner class is petty and disrupts recruiting databases, and (2) NALP’s 2018 deregulation has broken on-campus recruiting, forcing 1Ls to accept “exploding offers” after minimal screening, increasing early lateral moves and worsening inequities. It also discusses (3) federal judges withdrawing AI-influenced opinions with fake/incorrect citations, and Senator Grassley demanding answers, warning against AI hallucinations and hype.

Guest backgrounds

No guests are interviewed; the hosts discuss among themselves. One host mentions being a parent with a toddler and cycling; others reference consulting/recruiting experience and law-school processes.

Key claims

Hidden partner promotions harm visibility for recruiters and clients; deputizing students to recruit others can deepen peer-group segregation; AI should be used for summarizing transcripts, not for generating citations/quotes.

Notable examples

Kirkland’s undisclosed 75-partner class; Sullivan & Cromwell and Paul Weiss offering “walking around money”; Deloitte’s fake-citation report; Georgia judge rubber-stamping hallucinated citations; federal judges’ pulled opinions; Senator Grassley’s letter.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

Travel Tales and Toddler Costumes

0:45 to 4:40

Discussion about recent travels and the challenges of selecting Halloween costumes for a toddler.

“So I have been on the road back and forth.”

Kirkland's Silent Partner Announcement

4:40 to 5:24

Exploration of Kirkland's unusual decision to withhold publicizing new partners and the implications of this action.

“I think that we've been on this all small talk for a bit.”

Speculations on Recruitment Practices

5:24 to 11:25

Deep dive into the motivations behind Kirkland's decision and the potential impact on hiring and diversity.

“Yeah, they've decided that they're not going to publicize who has made partner.”

New Practices in Law School Recruitment

13:33 to 14:00

Discussion on the emerging trend of law firms using students to aid in the recruitment process for new hires.

The Breakdown of On-Campus Recruitment

14:00 to 17:46

Explore the changes in the on-campus recruitment process for law students and its impact on their job prospects.

“bank, you know, kind of expense accounts.”

Challenges of Social Segregation in Law Schools

17:46 to 22:48

Discuss the social dynamics and segregation observed among law students and its implications.

“And that was disproportionately the people who made those sorts of deals.”

Impact of Housing on Law Student Networking

22:48 to 27:30

Understand how law student housing arrangements influence networking and job opportunities.

“And one of the things, and about it, because again, not an indict against Washuah, but I do think it's like a national thing.”

AI Issues in Legal Opinions

27:30 to 28:01

Examine recent issues with AI-generated content in legal opinions and the implications for the legal field.

“AI and hallucinations, we've talked about them.”

Issues with AI in Legal Writing

28:01 to 29:30

Discussion on the problems arising from AI-generated legal documents and the need for safeguards.

“that pretty quickly, it seemed like there were some issues.”

The Role of Lawyer Hubris and Technology

29:31 to 31:35

Exploration of how lawyers' ego and tech advancements contribute to reliance on AI.

“where people who are pretty smart think that they're too smart to fall for the hallucinations, right?”
Show all 12 chapters

Judicial Trust and AI Hallucinations

31:36 to 32:44

Analysis of judges' reliance on briefs and the implications of AI inaccuracies.

“That is not what you should be using AI for.”

Understanding AI's Role in Legal Practices

32:45 to 34:31

Advice on how to responsibly integrate AI into legal workflows while recognizing its limitations.

“And so back to the federal judge situation, though, like obviously judges are overworked and all it was interesting.”
Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:28Hello, everyone. Aha, you said duty. Sorry, I have a toddler. Don't put that on them. Don't put that on them. You would have said it either way. I mean, probably not, but you know, you never can tell, I suppose. I know where you are.

0:48Time for some small talk. So I have been on the road back and forth. I've not been home in a while because I was at Relativity Fest in Chicago. and then I went to an event in Houston and I went direct. So I haven't really been back. But I want to talk a little bit about my trip to Houston because the show came up while I was there. Oh, ran into a fan. Did you? It did. It actually did listen, ran into a regular listener. But that actually isn't what I was going to talk about. I also ran into somebody who is not a regular listener. And this person had a conversation. I was asked the question, what did I think about this one topic?

1:32And I said, you know, I actually have talked about this on the show before, which I was asked, you do a lot of things with AI. One of these students has a thought. Explain your thought. And the thought was, I kind of think, I think clanker is racist. And I was like, I have absolutely had this conversation. We dug up the episode to show her like, yeah, we had a whole listen to the first like small talk of this. We talked about it for five minutes. And who brought that up? You're damn right. I said Chris brought up this issue already. We've talked it through. And yeah. Amazing. And you know, and here's the thing.

2:10People are like, part of me was like, oh my God, is this a hot take? And I was like, no, it's been a week. And we went from Clanker to Rosa Sparks. Like, this is just black people. This is just black people, y 'all. No, but it was great because the question was asked, like, here's this off the wall idea. And I was like, not off the wall. We've, in fact, had a whole section on this. So small talk for the win. There you go. Yeah. Yeah. You know. Well, I will make my own contribution to small talk, which is it is nearly Halloween. And I do, in fact, have an aforementioned toddler. So this week I spent some time trying to figure out Halloween costumes.

2:50And I feel like it's undersold how difficult it is to figure out a costume for a kid that is unlikely to keep anything on their head. I do not have the kind of kid that will like joyfully wear a hat or a hood or anything, even a clip in her hair. She fights it every step of the way. And so many costumes, particularly, you know, I live in the Northeast. There's a high likelihood that the outdoor events will require some sort of a jacket or something that without a head, without headgear that still look like the thing that you are. Yeah. It's like a challenge. So Eleanor Roosevelt, what do we got?

3:29I think we're going to do Abba Kedabby. Ah, well, that would have been my second guess. I think we're doing abby-kidabby because I found these hair bows, which again, she's not great at hair bows, but she's better at a hair bow than she is at like a headband or even a mask that she would not keep on her head for even a second. And here's the smart momming moment that I had. So I got like the costume. It's a little dress with fairy wings because, you know, she's a Muppet, but also a fairy. But she's pink. And so I found a pair of like pink fleece pajamas that I'm going to put on her and then the dress over.

4:06But it doesn't look ridiculous because it's because Abby is pink. OK. So it's actually looks like correct. So I felt really proud of that. I thought it was really clever of me. Well, that's nice. OK. So recently, I've been back on my bike. I've been cycling. I'm getting my bike back. Not saying like in an aesthetic way, but like it hurts less to ride my bike. and that's been like one of the big breakthroughs. I will get my bike butt back though. So if anybody wants to hit the Patreon, give me like two or three months. But yeah, it's been really nice. That's all. Well, okay. I think that we've been on this all small talk for a bit.

4:46I think we can transition into the main topics of the week. What is our top story? Oh, there are some new partners at Kirkland, we think. Question mark. Yeah, this is a really interesting story that our colleague wrote up. So every year, all these big law firms make a bunch of partners. Kirkland is like the largest firm, certainly in terms of revenue, but have like thousands of lawyers. That's a big deal. They make a giant class of non-equity tiered partners every year. And, you know, there's a press release that goes out like, oh, congratulate our 75 new partners in the following offices. But this year, mum's the word.

5:27Yeah. Yeah, they've decided that they're not going to publicize who has made partner. Do we have speculation on why this is? Yeah, there's a bunch of speculation that apparently every year after they make the announcement of who's made partner, many of them, or significant enough percentage, find work elsewhere. and in an effort, I guess, to stem that lateral flow of newly minted partners, they're not going to tell anybody who they are. But I mean, it's weird, right? Because it's still going to be on the website, right? Like they still, because you want your clients to know that you're being serviced by a partner, not just an associate anymore and that kind of stuff.

6:08So like, it's not like it doesn't exist out there. If somebody wants to lateral, they're still going to lateral. and it just feels like really petty and small for such a big firm especially but like I don't know like those press announcements that's the kind of shit you sent to your mom and you're like look ma it was worth it I know you haven't seen me at Christmas for the last five years but like this is why and I don't know it just feels really really petty I just imagine some world where and I'm assuming it's not this one where this is actually a good move and recruiters are like, Oh my God, how will we know?

6:46Like there have to still be other ways that people discover, Hey, I'm a partner. I'd like to move elsewhere. Like maybe like, I don't know, they put out their resume or they update their LinkedIn. Like there will still be other ways to people to find out and lateral if they want to lateral like this isn't, this isn't, I don't, I don't get how this is anything but a screwed ass moment for people that wanted to share their successes. 100%. That's an excellent point. So it is going to be discoverable through LinkedIn, who these folks are. But I will say LinkedIn is a, you know, you have to go there and kind of do some searching and know what you're doing.

7:20The from recruiting perspective, since I've, you know, done some consulting and know how that world works a little bit. This is a little disruptive to the extent that if it's not visible on the website, which is what's get scraped to be put into a lot of these databases. So if you're targeting, you need to find for a new opening a partner with this level experience at these sorts of firms. If it is not on that website, it's probably not getting scraped into this database. So when you make your initial touch out to a bunch of folks and say like, hey, this opening has come up. You're one of the people who seems like you would be a fit.

7:57You might miss them. It is still findable, but it is something that you have to kind of go piecemeal. through, which is really unfortunate. Yeah. But isn't it also true that like, OK, there's not going to be a press release that gives you the clear list of who's the partner. But I have to imagine and I guess maybe this is something we'll discover with time. But Kirkland's website will still be updated. If you're saying like Joe Schmo used to be of counsel and now that website that, you know, that kind of his bio is still going to be updated to say partner. Right. Which then it will get in the day right and then it will get into the database and it has to it has to be updated i think because when the clients get their bills and are like why am i getting charged this for you know who is this why is it this rate i think that you go you know it's a partner and i think that you have i think that information has to be available i assume clients will just accept that if the bill says this is a partner it's a partner but but i hear you like you would think that the website would address that uh yeah and listen the website's just gonna say attorney.

9:00I think it's got to be updated because eventually you have to know that they're a partner and maybe it just kind of delays and they shouldn't that side of that kind of bum rush or sort of onslaught of like the initial. But like, I don't know. It just, again, super, super petty. Yeah. And their website does distinguish between associates and partners, which, you know, there's an argument to be had that like, I know that there's some websites that go the route of only putting up partners or putting everybody up. And, you know, because who's really hiring a lawyer off? If you're hiring Kirkland, you're probably not doing it by going to the website cold, right?

9:41A hundred percent, sure. There are ways that they could scrub that information and not share it, but they don't currently. I just went to check because I thought that was a great point. Yeah, it's a weird decision. And I don't think it will slow things down all that much for hiring. If anything, honestly, I think it would probably only increase it, right? If you're someone who recently got made partner and I'm sure you made plenty of sacrifices to get to that point. And then the firm seemingly snubs you in this very public way that a lot of people are talking about that can't engender like good feelings towards the firm.

10:16If anything, it just kind of exacerbates whatever resentment you have towards the firm. and you're like, hey, maybe another firm won't be this awful. So I was gone all week. So I really haven't had a lot of conversation. Like we all talk throughout the day normally during working, but I have been in conference mode. So I actually didn't chat with you all as this was going down. Was there any speculation about the motivation for this potentially being to avoid any imperial entanglement? Basically, they just – and Kirkland is one of those firms that, you know, is involved with this administration is part of this logic.

10:54We don't want to put out a press release that's going to allow somebody to look and say, why did you promote all these black lawyers and women, which this administration might do? Yeah, like it keeps it under wraps and you can like fix it on the back end. But there's no there's no press release that somebody could just go down really easily. And yeah, I mean, maybe again, I don't know what their promotion class looks like, if that was even part of the thing. But you have to think they did sign the deals with the administration. So assumedly they're on the good list. They're no longer on the naughty list, right?

11:28So I mean, maybe I'd be more like, maybe that's part of the rationale if it was a firm who felt like they narrowly escaped the attention of the administration. But like, they're already giving, you know, $100 million in free shit to the - I feel like the lesson of what happened with the universities is that the first rule of proto-fascism is if you make a deal with them, that actually makes you more on their radar. Oh, fair. They will. They now are going to more police to make sure that you stay in line. Yeah. But this kind of goes back to a drum we've been beating a million times about these deals, right?

12:06They're terrible deals. They are spineless deals. and again kind of goes to the, you're not willing to stand up for yourselves. You're not willing to stand up for your people. You'd rather their accomplishments go unacknowledged on the fear that you'll get. That doesn't make me feel great about that firm, no matter which of these explanations are at the root of it or which combination.

12:36Imagine getting a free marketing consultation or workshop for your law firm every month without ever booking a meeting or paying it done. Sounds too good to be true, right? They don't have to imagine it, Conrad, because they can actually experience it. Our podcast, Lunch Hour Legal Marketing, which you can watch right now on YouTube, has dozens of episodes explaining how to update your Google business profile, what kinds of campaigns are working for lawyers, and what SEO strategies you should implement. And that's just scratching the surface. Now, there are a lot of people out there who think they know the ins and outs of SEO, social media, and paid marketing.

13:12But in reality, they're just greedy or grifters or faking it most of the time. As agency owners, my co-host Guy and I know what we're talking about and would love for you to be part of our growing community. Check out our library of episodes and subscribe to the Lunch Hour Legal Marketing YouTube channel or wherever podcasts are found.

13:33So back to the subject of recruitment, but we're gonna talk about it from a slightly different lens, But the law school recruitment above the law, we got some tipsters who were able to confirm a new practice that is taking over some of the top tier law schools, which is that big law firms are deputizing, for lack of a better word, law students that have already accepted offers at the firm to be their sort of eyes and ears on the ground and participate in the recruitment process for those 1L students. bank, you know, kind of expense accounts. Some have dollar limits associated with them. Some feel, at least from people who have experienced it, like they're pretty unlimited.

14:14So it's giving law students the ability to experience that expense account life before the actual crush of the hours that are accompanied with them. But the other sort of interesting, and listen, I blame this entirely on NALP, right? The National Association of Legal Placement. They kind of did in 2018, I believe it was, they kind of went hands off. There had been a bunch of rules about when law firms can approach students, when they can have interviews, how long offers that they make to law students have to stay open. And, you know, all this kind of things which really created the on-campus recruitment process for rising 2Ls to get their jobs at the end of their 2L summer, which, you know, generally led to their post-graduation employment as well.

15:03But in 2018, they kind of took the guardrails off entirely. And now it's terrible out there. Like, it's really awful. Like, you cannot overstate what a disaster buckle it is. There are exploding offers. someone else are getting offers before they have grades at the end of their first semester. And the offers explode before they get their grades and have an opportunity to interview with anyone else. So they're taking these big law offers only having interviewed at that one law firm, having very little sense of what these firms are, what they do or anything like that, especially if you come from a background or family that doesn't have a ton of connections in the world of big law, which, you know, a ton of folks don't.

15:44So, you know, you just get you wind up with offers. You have to take them. Otherwise, you lose them. Or maybe you do kind of keep them open for longer, but you just don't have the same process, the same sort of knowledge base that kind of the organized OCI process really imparted on folks. I didn't know what all these firms were or what they stood for or what it meant to accept an offer from Skadden versus Baker McKenzie or something like that. You just don't know kind of what the nuances are between big law firms if you aren't really seeped in it until you're kind of forced to have a boot camp with your kind of law school's OCI people and be like, OK, this is the booklet.

16:23This is what you need to know. This is what you need to do. So and that really, I think, is usually important for students to make the right decisions for where they're going to wind up post law school. And I think that we're also going to see a ton of lateral moves earlier and earlier in careers. It used to be sort of, you know, gauche to lateral before your second or third year. But with folks forced to make these choices before their first semester or first year of law school is even completed, I think you're going to see a ton more people being like, yeah, I don't I didn't know what I was even I didn't even know that this firm existed, you know, let alone anything else.

17:01And one of the other kind of interesting spins is, and I'm sure other firms are doing this, but the two firms that we have confirmed are kind of giving this walking around money to upper class law students are Sullivan and Cromwell and Paul Weiss. Yeah. Where have I heard those two on a list recently? Yeah, they are both sort of MAGA adjacent. Paul Weiss obviously was the first law firm that signed one of those spineless deals that we were talking about earlier with the Trump administration. Sullivan and Cromwell is Trump's attorney in his criminal – in personal capacity in his criminal case and, according to reports, was involved in the negotiation of the deal between Paul Weiss and the administration.

17:45Now, I will say in a charitable move, I will say the kind of firms that have money to burn where they say, hey, here's some walking around money tend to be the firms at the very top. And that was disproportionately the people who made those sorts of deals. So there is kind of a conflation on that front. That said, the less charitable answer is who we just had the associate report that came, the law students. It was law students, right? Like summer associates. It was summer associates. And we got that report back. And apparently a very large majority of that cohort have very dim views of the firms that took those deals.

18:32So, you know, it would suggest that those are firms that probably need to invest in fixing that reputation and recruiting. I think the other part of that story is that there is this information, this disproportionate amount of information that big law firms have all the information and the students oftentimes don't. But these stories about the big law deals with the administration kind of broke the legal industry contain, right? They became mainstream media stories. it was more than just legal news that was talking about them. Mainstream news, ABC, like all the big folks were talking about it.

19:07So if you don't have a ton of information about the legal industry, but are in law school, you've probably heard about these deals, right? In a way that you might not have heard about some of the nuances of the new, you know, products liability practice at, you know, you might not have heard about that, but you have heard about these deals. And so you have, you don't know a lot, but you have a dim view of certain firms and nothing like getting rid of that dim view than throwing around a bunch of cash. Yeah. The New York Times even did like a whole piece on this blog that covers it. Yeah. It was really insightful and striking the imagery that they used from that story.

19:44We looked like, we really did look like we were releasing a like 90s indie band album or something. One quick point. Yeah. And I'm happy to get back to our album cover. But my immediate thoughts on like the deputization of other students to go get other students. And I'd say this, I went to WashU, Washington University in St. Louis. I don't mean this as a particular indict to WashU. I imagine it's like this at most law school campuses. but I think that there's a tendency towards de facto segregation like I've noticed that a lot of the study groups tended to be white or like not necessarily necessarily white but they tended to be of the same racial makeup like there tend to be like white students studying over here black students studying over here a lot of the international students that tended to be Asian just because of the makeup of the school over there and one of the thing about more standardized processes that involve like actual applications and things like that.

20:47It's a lot easier to have equitable inputs when it's rather just people in cliques who are likely going to tell the people in those cliques, hey, join me. Because the way that that plays out is like most of the people there, like for example, you're going to get a bigger shot of what the majority looks like without any intervening factors that could help people that would otherwise be looked over get a shot and it's big law as is so it's like it's not like it's the blackest area of law to start with or the most asian area of law to start with so that was that was my thoughts and it's and it's and we mentioned the reason that this happened is because what was it the nalp or something now yeah yeah they kind of laxed regulation and it's one of those things where it's like oh well god this is this is this is dumb or have you or can you believe how foolish this is but it's also like there will be questions about like every once in a while people ask Why is law so white?

21:42Or what are the things that go into play to normalizing this, like knowing that things could have been elsewise? This is one of those things where a choice was made. Yeah, I think you're 100 percent right. And I also think it's weirdly bold on the law firm's position to like let students represent them. Like that's a lot of faith in students that you don't have a ton of information about. They don't work for you full time. At best, they've had a summer with you, which is not the same. You don't have a ton. Like, why would you want you don't know what they're going to be like when they're taking other students out to a bar late night?

22:21Like, that just seems like I don't know. I would not be comfortable if it wasn't somebody that I had a ton of experience with. You know, to Chris's point about the study group stuff, from my perspective, that speaks to the breakdown of a lot of how a lot of steps in the process are getting broken down in really dangerous ways. uh now not being there for this purpose means we're doing this right out of the gate uh you've just shown up in law school which is the time where you are most likely to have just like gone to probably implicitly biased driven peer group because you haven't had time to meet other people be you know all based on anything more substantive on top of that we have these groups who have been fighting the law review processes and stuff like that where the groups you would join throughout your 2L year.

23:14So before you would have this sort of, or throughout the years where you would start building these sorts of relationships that used to have diversity initiatives to make sure that you would be working closely with other people who you might not have gravitated towards in the first place and build relationships with them. Those are being broken down too. Like at all levels of this process, it's being broken down more and more such that, yeah, Those original, you just walked in and started talking to the person that you implicit biasly started talking to because they were sitting next to you. I like to cut your jib.

23:49Are the people there. Yeah. Yeah. And one of the things, and about it, because again, not an indict against Washuah, but I do think it's like a national thing. That's probably how most people's social groups are, just how through historical processes of acculturation. But anyway, to Joe's point, one of the things that I've noticed, I was like, hey, we are self-segregating. I tried to break out of that by going to like events held by affinity groups outside of FedSoc, which is generally the white Republicans. Which is also an affinity group. Yeah, yeah, yeah. But no, white people don't have identity politics.

24:24So you can't say that. But no, but like I would try to go to like, say, just like a Black Law Student Association events. I was a member, like Asian Pacific Islander law school, student associations, what have you. And as you start to see those types of groups die out, it becomes harder to deliberately break out of the tendency towards the fact of those segregating. Because at least at those events, you have an excuse to be social. There's an expectation that you can talk to people that aren't in your clique and have it be like not a weird, why would this person want to talk to me thing, you know.

24:57So my law school, thankfully, the way that the way that student housing was structured, we we mostly hung out with the people on our floor, which was, you know, a random sampling and diverse, which helped rather than that. But if you have, for instance, I know other other schools, the way in which they do law student housing isn't like here's one big floor of a dorm. So, you know, for grad students, a lot of them have like, oh, yeah, we got you an apartment over here. And you're like, that was my, you know, I was in part of grad school housing and my roommate for the majority of my law school career got her Ph.D.

25:36in some science thing that I remember. There's no there's no bonding with your class and your class. Yeah. Yeah. So I was very lucky. NYU did it the way they did, which, you know, allowed you to meet people who you, you know, outside of just like, oh, the person sitting next to me, whatever. But yeah. That feels so foreign to me because in my experience, my grad school housing was I went on Craigslist and I found a spot that I could afford. And it just happened to be in the area of St. Louis where it was other broke black folks because St. Louis, the history of it, it is you can see the racial divide in the infrastructure.

26:17Like there's a area called like the Big Bend. I think they call it like the Del Mar Divide or Abbey. And it's like a it's basically baked into the city. itself there's like a white affluent area and then everybody else but yeah like i would occasionally go to like parties where my um like people i would meet in my cohort that happen to be different um different socioeconomic backgrounds and we'd have to travel like i'd have to ride out to see them i was like oh they're living different i noticed they tend to live near each other you know um so yeah so much of the like the just the finding a place to live during law school what like indirect can indirectly impact where you get a job, if you get a job, who you know enough to reach out to try to get you a job, you know?

Read the full transcript

27:00And it's the thing about it. I think these are like ground level common sense things, but they're harder to take account for. These like interpersonal aspects that set the groundwork for some people, literally the rest of their lives and their careers. It's these sort of like intangible things, like who are your friends with in the first week of school. And ideally, that wouldn't be such a big determinant. But with things like this, that's what it's going to, I'm assuming that's what it's going to skew toward. Yeah. All right, let's move ahead to, let's do this story. AI and hallucinations, we've talked about them.

27:37They are the periodical fun story that we get almost on a daily basis at this point, some lawyer puts in a brief with a bunch of fake cases or something like that. And we all point and laugh. Mostly they don't get sanctioned too hard. Sometimes they do. The wrinkle is that earlier this year, there were two federal judges who put out opinions that pretty quickly, it seemed like there were some issues. Parties were named in them that weren't parties in the case. Sometimes there were some fake citations. Sometimes the citations weren't necessarily wrong, but the quotes from the citations were wrong.

28:15That seems suspicious. Then those judges, those judges have pulled those opinions, replaced them with corrected opinions and said it was a clerical issue and not spoken about it. Other than that. Last week, Senator Grassley wrote a letter to both judges demanding answers. Why? How did this happen? And on the one hand. Great question. how did it happen yeah on the one hand look on the one hand yes somebody should ask the question how did this happen what safe guardrails should exist to prevent it in the future i was less despite the fact and those questions are in the in here i was a little more negative about this letter because i feel as though the letter also took kind of a grandstanding position there's a lot of like, how dare you be using AI, which I think is probably throwing a baby out with bathwater situation.

29:12I do think that there's probably good ways in which it responsibly could be used to accelerate the writing workflow. But obviously, the real question should be, how did this happen? How are you going to help your colleagues and the rest of the federal judiciary avoid having this happen? I think right now we're in a weird moment with this technology where people who are pretty smart think that they're too smart to fall for the hallucinations, right? There was another case, right, where Deloitte very famously put out a report for the Australian government with a bunch of fake citations. A law professor was like, hey, you cited my colleague.

29:50This is not what she writes about. And quickly figured out that it was AI and they had to refund the government a bunch of money. But like, it's because they think they're too good to get caught. Of course I can use it. maybe, you know, make my life better, not necessarily pass those benefits on to clients or to any of the other stakeholders, but I can do it and I won't get caught. Well, so there's obviously lawyer hubris. We're never going to discount lawyers ego. But I, I actually think a lot of it is the companies themselves. Like we, we have tech bros who are raising money hand over fist from venture capitalists throwing it at them.

30:31Part of that process is to constantly pretend that the world is is radically changing every time they do this. Like we just had GPT-5 came out and right before it did, Sam Altman was going out going around talking about how it scared him, how good it was. And then it came out and it is to say not to say a underwhelming to say a step improvement over the last iteration is probably too much uh so we have this hey man i i remember what was it three years ago forget ai uh ai generated videos just the ai pictures we used to get look like stroke symptoms so i'm i'm like just imagine the orders of magnitude of improvement from that to this so it's right now people are like oh look at the hands before then i was like a toddler could do better than it so i'm i'm of the mind of give this a year and this is going to be full-length movies coming up so it's it's video and image creation no doubt no doubt that the video and image creation definitely getting better but what i what i'm kind of going with is that we we have these folks saying all this and we have an uncritical media infrastructure repeating how revolutionary it all is and i think there's lawyers who are hearing that and saying yeah it might have It might have screwed up things in the past, but it won't do that now, which isn't true.

31:58Sure. It just is not true. That is not what you should be using AI for. And there are ways in which AI is super valuable, and you should be using it. That's not one of them. It is a very good summary tool for getting through a bunch of deposition transcripts. Now, is it going to give you direct quotes? It can, but it could also make up some of those quotes, so don't do that. But if you want the thrust of what happened and the top 10 bullet points and where to go within the transcript to find them, it's going to do that really well. You just have to kind of understand what its limits are. And right now it doesn't seem like anybody does.

32:36And I think you're right. You've got the lawyer ego. You've got a media diet of everything's been fixed and now it's magic. And those things combine to get people doing this. And so back to the federal judge situation, though, like obviously judges are overworked and all it was interesting. I thought that judges got hit with all this because the historical knock on judges is that they don't actually write any of this stuff. They like decide which of the two briefs they like and copy and paste all the work that those lawyers did, which, of course, is the reason why people need to be very worried about lawyers hallucinate having hallucinations.

33:16That's why we had that Georgia case where a judge just uncritically rubber-stamps hallucinations that a lawyer had. It's another reason, by the way, why a few weeks ago that we had some – was this a good idea? Where a lawyer got sanctioned for having hallucinations, and the other party who didn't have any hallucinations also got a slap on the wrist because they didn't catch it. Yeah. And folks are like, well, that doesn't seem fair. But I mean, the reason for that is that judges more or less are trusting the briefs and trusting that the other side is zealously making sure that they've answered it.

33:53And if they are falling down on that job, that is a signal. So here, obviously, inserting new stuff was the problem. And, you know, that's it's unfortunate. It's almost like judges tried to do something beyond just copy and pasting the briefs and then introduced more problems. Never try too hard. That's the lesson. The lesson is never try. Yeah. But it is a real problem. It really does come back to, folks, don't not use this stuff, but you have to understand exactly what it can do and very much what it cannot. Yes. and you're being sold a bill of goods about what it can do that is not what it can do.

34:37But what it can do, if you understand how to use it and run it through your process responsibly, is actually super valuable. And that's going to be the challenge, I think, of the next few years, is convincing people not to listen to the hype and understand what it really does. All right. Thanks, everybody, for listening. You should be subscribed to the show so you get new episodes when they come out. leave reviews, write things, stars, that all helps. You should be listening to the Jabot, Catherine's other podcast. I'm a guest on the Legal Tech Week Journalist Roundtable. You should be listening to other shows on the Legal Talk Network.

35:11You should read Above the Law so that you read these and other stories before we get to them. It's mostly over at Blue Sky and still at Twitter too at ATLblog, but also on Blue Sky, theabovelaw.com. I'm at Joe Patrice. Catherine's at Catherine1. Chris is at Rights4Rent. And with that, we will talk to you next week. Peace. Peace.

From the publisher

Just in time for the AI slop to take over.

-----

Some law firms are handing out recruiting entertainment budgets to law students. While we don't fault law students some sweet walking around money, placing that power in the hands of students highlights the breakdown in the law school recruiting process and a real risk of baking more bias into hiring. Why has Kirkland memory holes its incoming partner class? The decision to opt out of its traditional announcement message seems like a move to shield its high-achievers, but there are some other possibilities. And a Senator wants some answers after a pair of federal judges issue opinions with possible (read: likely) AI hallucinations.

More from Above the Law - Thinking Like a Lawyer

All 62 episodes
On Campus Hiring Is Broken, And This Isn't HelpingAbove the Law - Thinking Like a Lawyer · 38 min
Listen in VO