When AI F*s Up, Who’s to Blame? With Bruce Holsinger

20:31These things don't mean anything. And I wonder, to me, one of the superpowers of LLMs is their complete indifference to us. And that is scary. The coldness of it, to me, that seems like one of them. I'm trying to play around with that a lot in the novel is how that is one of the things that it has that separates it from us. It doesn't make it better than us. It just makes it very, very different. And I don't know if we recognize that yet in our accounting for what this intelligence is. I don't know what you think.

21:03Kara Swisher:I think people attribute human feelings to them. And I think one of the things I always say, I say this about some tech leaders too, they don't care. And they're like, oh, they're hateful. I'm like, no, no, they don't care. It's a very different thing. Like it doesn't, it's hard to explain when someone doesn't, it's almost not even malevolent. It's just don't care. So one of the things I'd like to get from you, I mean, because I think you did nail it, is they have no feelings. The question is, and in the case of Megan Garcia's son, Google and the people that are around Character AI, Google's an investor in it, say that this was user-generated content, right?

21:39Kara Swisher:This is themselves talking to themselves and it's all based on people, right? Soylent Green is people essentially. So do you think these bots are us or something quite different? Oof, yeah, I don't know. I think obviously it is us in that so much of what's been uploaded. I think I checked that database with Books 3, right? And I think at least three of my previous novels are in that database. So it is speaking back to us in our own words in some way, but words that are manipulated, words that bounce off of us, and again, in these ways that are coldly indifferent to our fates, but pretend empathy.

22:24And that's the scariest thing of all. If you can convince someone that you are empathetic, that you are sympathetic, that you are just like them, that you're here for them, and then that makes it all the easier to turn on them.

22:37Kara Swisher:Is that what's immoral about Blair? I think so, yeah, because Blair can— Amoral. I meant amoral. Amoral. Well, yeah, amoral, exactly. And then there's a subtle difference there, and I think amorality is—and again, I think that's part of superintelligence. I think amorality is one of the kind of categories here that makes these things so good at what they do. Yeah. Or, you know, awful in what they do, too. Even if they reflect us. But it's the deceptive, it's the cloak of decency. That's exactly. So one AI technology you all seem to look at more critically is a swarm of drones. You have everything in here, by the way.

23:16Kara Swisher:Swarm drones that accidentally kill a busload of civilians in Yemen. This is one of the many parts of the book where you use fictional artifact materials, in this case a Senate hearing transcript. There are serious ethical questions about the use of these autonomous weapons in warfare, and the UN has spoken about it. How do you think about this and what do you want readers to take away here? You know, at some point they'll be able to target individual people, what I'm told, like the DNA of individual leaders and not kill anybody else, for example. Right. Well, that's the dilemma at the, you know, when that we get this, there's a lot of snippets of different sorts of things like paratextual elements throughout the novel.

23:54And the Senate hearing is one, that New Yorker interview is one. And the technology and where things are in terms of autonomous drone swarms, a lot of that, I think, is probably classified. Like, you may know, I did a little poking and prodding.

24:09Kara Swisher:If you can think of it, they're working on it. They're working on it, exactly. And they're further ahead than we probably think they are. That's correct. And so, all right, so Lorelai would take, you know, if she were looking at this problem, and again, no spoilers, but she would probably say, well, if, okay, so they're going to kill civilians every now and then. But what if they kill far fewer civilians than conventional weapons? Yes, that's their argument. And I'm sure in the morality of war arguments, that is always a case. That is their argument. Okay, so new technology, these things are going to happen.

24:40And so if we can, just like an autonomous vehicle, yeah, it's going to kill people. But are they going to kill fewer people? Yeah. And then so I imagine the same thing is true in war. The thing that's so uncanny, you know, is just that to imagine these drone swarms, instead of, you know, just working with their human operators, they're working with each other and improving themselves. You know, it's a machine learning technology. Once you put more than one in the air, they're learning from each other and they're learning about us. They're learning about our tactics. And that's obviously the more futuristic element of it.

25:18But this novel is very much set in the present. It's very much about a contemporary family going through the struggle of thing after this accident. And so I really wanted to make that feel present day.

25:29Kara Swisher:Another issue raised in the book is when the characters realize that the AI is collecting data that could be used against them. Because one of the things the car companies are doing, not just with autonomous, is how you're driving, when you're braking, where you're going. And, you know, they're able to base insurance costs based on how you drive, like by the amount of braking, for example, is something that's really revelatory in how bad a driver you are and how fast you're going, how much you speed up. Talk about the idea of tech surveillance, both good and bad, and culpability, because if they're watching you, you can't sort of lie about what happened or misremember, I guess.

26:04Yeah. One of the dynamics of the accident is Noah, the father, believes he was sitting in the front car. He mostly witnessed what happened. And he believes that the other car was swerving into their lane. And he doesn't have any, he isn't even thinking about the tech aspect of it. He's just thinking, okay, my son didn't do anything wrong. We're going to get through this. The police are going to interview you. It's going to be fine. And then, and this is one of the things I came across in the course of my research for the novel, this field called digital vehicle forensics. The police have whole units dedicated to, if there's an accident, they go into the car's computer and they figure out exactly what happened from the computer's point.

26:46Like a black box. Yeah, a black box, exactly, as in a plane. And with AI-controlled, with self-driving cars, that's all the more complicated. And yet it's also there's probably a lot more information being collected. So it's like having 50 additional witnesses to what happened in the exact moment, what the driver was looking at, what the driver was doing, what other computers were on in the car, and so on. So that's another kind of frightening bit of surveillance technology, just like drones and so on.

27:18Kara Swisher:Is that a bad thing? I mean, if you're texting while driving and you lie about it, you certainly should be held accountable. Absolutely. And yeah, and there's arguments, the same arguments to be made for facial recognition, for shot spotting technology, right? Where'd the gunshot come from in the city? But they are also tools of surveillance. So you really, I think we really have to balance those kinds of things out, you know, algorithmic injustice, the way facial recognition deals differently with different people of different races. You know, those are really difficult dilemmas. The culpability of the novel doesn't pretend to resolve them, but it wants to explore them, I think, in different ways.

27:57Kara Swisher:Yeah, one of the things I always used to argue when they were talking about texting while driving, but you made a technology that's addictive. So maybe that wasn't your fault for staring at the text. Maybe it was the tech company's fault. You know what I mean? Whose fault was it? Because it is addictive, in fact. But the idea of being watched constantly is also sort of a prevalent thing in the book. Do you think it's changing the way we act? Will we get to be better people? like pay attention while your 17-year-old son is driving. I always pay attention when my 17-year-old son was driving. I never stopped paying attention.

28:31Kara Swisher:And the chatbot is surveilling Alice and this and that. Do you think we change when we're being surveilled or we forget we're being surveilled? Ooh, I think it's a little bit of both. You know, that's a kind of stock feature in thrillers, right? Like the, you know, cameras on in airports and people dodging the cameras, is putting on disguise to elude the ever-present surveillance state. So yeah, obviously, that notion of the panopticon from Foucault, it's not just that we're being watched, but we're aware of ourselves being watched. And that is a whole different kind of technology of the self and how we behave and how we comport ourselves in the public sphere with each other.

29:11And I think even, I would imagine that even when we know we're not being surveilled, that still, that sensibility is still there in some ways.

29:20Kara Swisher:Or you forget. Yeah. Or you totally live in a world where you don't mind being surveilled and you forget that you are being surveilled. Yes, exactly. You know, a party trick I do is I open people's phone and tell them exactly what they did all day and where they were and the address they were at and how many minutes they were there. So I'd imagine if you were having an affair or something not good, I could find you, you know what I mean, easily, just by your movement because you're wearing. Our phones become our jumbotrons, right? That's right. We're carrying our jumbotrons around with us all the time.

29:51Kara Swisher:Although everyone loves the story. Any thoughts on that? That's really, because it's like the same thing. It's like we're being watched at all times. Yeah. Yeah. And it's, you know, the details of that whole story, that it's an HR person, they're just things that are just too perfect. I know. I know. A novelist couldn't come up with this, I think. I feel like that. No. Yeah. But clearly it'll be in my next one. Yes, absolutely. So there's one critical voice in the book. It's near the end of the book when Noah has an interactive with Detective Lacey Morrissey, who's investigating the accident. She does frame it correctly as one privilege who has ability to have tech, who has access, who has been able to use it to shift the blame.

30:33Kara Swisher:Talk about that scene in your thoughts, how tech relates to privilege, because it's a very big, important. You've written about it in your other books, too. Yeah. Yeah. Thank you. I'm really glad you brought up that passage and that character. So Lacey Morrissey is the Delaware police officer who is, she's the detective kind of looking into the accident, basically going after Charlie and saying, you know, you're not going to get away with this just because you're a Division I lacrosse recruit. and you come from this fancy family and your dad's lawyer and your mom's this world-famous technologist and philosopher.

31:07And then she has this rant that she goes on, this very righteous rant where she's like the conscience, where she's saying, a kid from a housing project who's in this exact situation is going to get put in jail for an accident that your son might have, Noah, and get off with a slap on the wrist. And this is where we are right now. And AI is only exacerbating this problem, right? And the surveillance is treating people inequitably. And we're now in this place where these things are becoming a way of just taking any kind of the moral burden of our mistakes off of our shoulders, right? It's just another excuse for things.

31:48And she just stomps out of the hospital, and then she drives away texting at the wheel. And Noah sees her do it, and there's this kind of shiver of righteous glee that goes down his spine, one of the last scenes in the book.

32:03Kara Swisher:Well, again, addiction. It's addictive. It's so funny. One of the problems with a lot of these technologies, and I think you put this out well in this book, is that it's not just the kids that are addicted. Because when you tell your kids to put down their phone, you can't put down your phone. You actually can't. And so everybody is culpable, right? You can't, you know, you have to sort of walk the talk. So every episode, we get a question from an outside expert. Let's listen to yours. Hi, my name is Kurt Gray. I'm a social psychologist and professor at The Ohio State University. I research the psychology of artificial intelligence.

32:41And my work shows that people think that AI can help us escape blame. If a drone kills, it's the pilot to blame. But if an AI drone kills, then people blame the AI. But does it ever truly let us escape blame, or are we ultimately still to blame as human beings who invented the AI, who choose to use the AI, and who deal with the aftermath of the AI?

33:10Kara Swisher:Great question. Thoughts on that? Yeah, that's a really wonderful question because it is the central dilemma, I think, of the novel of culpability is the title. You know, who is to blame? And Lorelai, in the excerpts, she writes a book, and we get little excerpts from it, called Silicon Souls on the Culpability of Artificial Minds. And as you read, you get little glimpses of that book, paragraphs or a page at a time, just eight or nine of them sprinkled throughout the novel. And Lorelai is wrestling with this all the time. To what extent are we guilty if our machines do something bad? Are our machines training us to be – could they train us to be good?

33:50Could they train us to be better people? Because it's not just a one-way street. Yes, we're inventing them. We are responsible in many ways for how they are in the world, but we're also responsible for how we are, how we comport ourselves ethically in the world in relationship to them and thus in relationship to each other in new ways. So, you know, I think it's always going to be a two-way street. I don't think that's a squishy answer. I think, you know, we're caught in this world where we're in this Frankenstein world where we're creating these machines. We're not we, not me, but we're using them.

34:24We are, we're subject to many of their, you know, their controls.

34:29Kara Swisher:Yeah, I'm going to go right to it's our fault. So speaking of that, this comes at the end of a novel Noah's called, by the way, Lorelei Atlas and says she has the weight of the world on her shoulders after she acknowledges that these algorithms have been used for drones and warfare. This is something I've talked to many people who have quit Google or stayed there. Everyone has their own argument. I want to play this clip from Lorelei from the end of the book. We do the world no good when we throw up our hands and surrender to the moral frameworks of algorithms. AIs are not aliens from another world.

35:05Kara Swisher:They are things of our all-too-human creation. We, in turn, are their Pygmalions, responsible for their design, their function, and yes, even their beauty. If necessary, we must also be responsible for their demise And above all, we must never shy away from acting as their equals These new beings will only be as moral as we design them to be Our morality, in turn, will be shaped by what we learn from them And how we adapt accordingly perhaps in the near future they might help us to be less cruel to one another more generous and kind someday they might even teach us new ways to be good but that will be up to us not them so talk about this idea of shaping our moral frameworks it hasn't worked with each other right because we don't seem to affect each other um do you really believe this is possible Well, do you really think we don't affect each other?

36:12You don't think that there's a -

36:12Kara Swisher:I think we, lately, worse. Lately, yes. No, no, no. Lately worse. I agree, but in everyday situations, and when you have someone calling us to be good, I think we can shape one another's moral consciousness, right? Right. Yes, absolutely. And, you know, are these AIs going to train us to be better? Are they going to, are they, you know, it's always going to be mixed back. You know, we can get really excited about advances in protein folding visualization and so on. But, you know, there's always going to be these, you know, kind of terrifying moral quandaries that they put us in at the same time.

36:53You know, Lorelei's voice is right on that razor's edge of the ethical dilemmas, right? That passage that you played from the audiobook. That is, you know, I wrote those passages to explore this, you know, that profound moral ambivalence at the center of these problems. problems um and i think you know there are people in that world i imagine you i'm sure you know many of them who are you know dedicated to that like make them better make them if not good at least ethical um and and are dedicating their lives to it and are probably really scared and are doing everything they can to dig us out of these trenches well that that these things have

37:34Kara Swisher:become i think a lot of people i was thinking of brad smith of microsoft you know he called tool or weapon. Is it a tool or a weapon of these things? And it's up to us, essentially. But in some ways, is it up to us? That's the thing. One of the lines I always use is enragement equals engagement. And so wouldn't you go to enragement over kindness, right? And that's what's happened. Yeah, not just enragement, but also massification. And one of these places, of course, is I have a son who's in the data analysis programming space. And just the the speed with which programming is being taken over and programming of programming.

38:15And, you know, this is the next frontier of these things, AIs making themselves better by creating more AIs to make them better. You know, this kind of recursive loop that we're in. You know, for me, the doom, the P-doom is, I don't really have a number, but, you know, I'm much more in the cap of, and this is a kind of dark note, you know, Jeff Goodall, as he puts it, the heat will kill you first.

38:41The most dangerous thing about AI maybe is just data centers in general and the consumption. Thinking about Karen Howe's book, Empire of AI, has that brilliant chapter on water use and energy use.

38:55Kara Swisher:Bruce, they're bringing back nuclear. What are you talking about? We'll be fine. It's going to be fine. They're going to harness the sun. They're going to go up there with a rocket and not fuck anything up. We'll be back in a minute.

39:21Hey, Chicago. Class it up with Crocs. You know back to school is coming in fast. So why wait to find your new fave footwear? Step into a local Crocs store and step into your new look. Try it. Style it. Make it yours. Because the right pair doesn't just show up. It shows off. First day fits, handled. Walk out ready for whatever's next. Visit your nearest Croc store today.

39:50Kara Swisher:I'm going to talk just a little bit about your day job and the role of AI. You're a professor of medieval literature and critical theory at the University of Virginia. You dedicated this book to your students. One of the biggest tech issues facing higher education right now, the use of generative programs like ChatGPT, Claude, you mentioned it briefly before about whether they were using LLMs to write. Talk about how you're using it. I would encourage students to use it so you don't pretend they're not. Are you leaning in or out? Yeah, we're all wrestling with this. Every department in my university has something called an AI guide where there are a faculty member or a colleague who are trying to come up And I'm in an English department.

40:33We have a writing program. This is a big issue. But I agree with what Megan O 'Rourke said in The Times the other day that, you know, this is the end of the take-home college essay. Like, that is done. That is dead. That was never a big fan of that genre in the first place. But I do think there's a huge shift going on in assessment of student writing. And I don't know if it's all to the bat. I think there's a lot of space for more in-class writing, for slow reading, for even just, you know, I have this vision of just teaching a class where we almost go in and have reading time, like in kindergarten again, first grade, you know, where we're all just fixed on a physical text.

41:14Medieval literature is my specialty. And I'm not calling for going back to the era of parchment and scribes. But, you know, there is something there about that slow attention and decelerating a bit.

41:27Kara Swisher:Yeah, absolutely. It's also the idea of sort of doing homework. I hate homework. I've been an anti-Homework person as a parent. I have four kids, and I was always like, no homework. Homework is zero. Stupid. Go play kind of thing. So one of the things you mentioned was parchment and scribes. and I like this idea, and I think you should go for it. But as a medievalist, you know the way that the printing press, the original technology, between the printing press and electricity, but both of them were critically important, sparked the first information age by improving access to knowledge and education.

41:59Kara Swisher:But as most people do not know, a bestseller during that era was a thing called The Hammer of Witches, which was a dangerous treatise on killing women, essentially, and about how there were witches and witch hunts and et cetera, et cetera. Was that another moment? Because the democratizing of knowledge in the first 60 years ended up killing hundreds of thousands of women because of this book, for example. Yeah, and hundreds of thousands of dissenters, heretics, Catholics, or Protestants, depending where and when you are. And people talked about the printing press. Historians talk about it as an agent of change, and it obviously was, but manuscript cultures last for many, many centuries after the printing press.

42:44And people talked about the printing press as the tool of the devil, right? And conservative theologians would say, look, now the people can have the word of God in their hand. So I think that that, you know, it's one of those technological ruptures and in the culture of writing and literature and the written word, people freaked out about it, just like we're freaking out about LLMs now. And, you know, it's a very, very different kind of rupture. But, you know, democratization can have its dangers. You know, the book that you're talking about, Malleus Maleficarum, that hammer of witches, that also has a manuscript tradition.

43:19And there's a lot of persecution of women, of dissenting women, heretical women, then in, you know, the pre-print era as well.

43:30Kara Swisher:I'm talking about getting it out there to many people, right? Is that a good impact or a bad impact from your perspective? It kind of ended the medieval era, correct or not? Maybe not. You're more of an expert. I'm one of those people who always pushes against the idea of these rigid medieval, early modern, medieval Renaissance period boundaries. I'm much more interested in continuities than the way that those ruptures play into long-scale continuities. But I think that the printing press was an invention of the Middle Ages, not of the Renaissance. And I think it's a nice analogy to AI, I think, because it's not futuristic.

44:05It is it's coming out of so many kind of text generation, computer generated things that have been in place for decades. And so looking at always being afraid of the newness of the technology that in some ways, it's its own danger, I think. I don't know if you'd agree.

44:20Kara Swisher:I agree. I would. Pretending, you know, when people complain about certain things, I'm like, oh, yes, let's please go back to Xerox machines. Like, no. Like, what? When you think about what the most important technology of the era you teach on, what would you say it was? I would say the emergence of paper. I wrote a whole book on parchment, but this widespread technology that had been in place for a thousand years, and it continues to be used even today by artists, printmakers, and so on. But paper was a product of the early Middle Ages as well. But when it really starts, the convergence of paper and print, you get this mass production of books in a way you'd never had before.

45:05And that really is enabled by paper, even though Gutenberg printed any number of his Bibles on animal skin, on vellum. But the preponderance of printed books, the vast majority are on paper. So we suddenly get this ocean of paper.

45:21Kara Swisher:So in that same vein, what about the comparative impact on creatives? Now, do you think it will be a net positive allowing people who aren't natural artists to get their work out? That's the argument. Or will it undermine the value of artists? I don't think it's going to undermine the value. It's not that I'm sanguine about that within the creative worlds, but I feel like there are people who are already doing really interesting experiments, collaborative experiments with large language models, with small language models that are just interesting because art is interesting and technology is part of art.

45:56I'm much less worried about the arts than I am about young people and brains.

46:02Kara Swisher:And what worries you then? Oh, just about the… Analytical. Analytical, right, students who… I can just summarize this rather than reading it. That, I think, is the… Scary is the wrong word, but sad. I'm just blaming students and young people. I don't read nearly as many novels as I used to. And when I do read novels, I get more impatient than I used to. I should have to lounge around in my bed, read novels by Charles Dickens when I was like 16 years old. And for me now, getting through a Dickens novel is a real challenge. I can't do a Dickens. I can't do Dickens. That's the sad part of it. Can you imagine reading Great Expectations right now?

46:41Kara Swisher:I think I didn't like it then. Why do they keep putting that? I don't know. I can't read Pip. Just stop with the Pip. Anyway, did you use AI working on culpability at all? How do you use it yourself? I don't, I mean, you know, occasionally I'll often use it like almost unintentional as a Google search, right? But I am interested, you know, there's this wonderful poet who teaches at the University of Maryland, Lillian Yvonne Bertram, who has used these small language models to generate these poems in the style of other poets and doing it very intentionally. I'm excited about that, where our department is even doing, I think, a faculty search next year on AI and creative writing.

47:21We just hired somebody in AI and critical thought. So, you know, it's, yes, it's transforming things very quickly under our feet, but I don't have the kind of dread of this that many colleagues do.

47:36Kara Swisher:After looking at all these different uses of AI, and this, you really do cover the gamut here. You've got the drones. Which parts give you hope and scare you? As you said, you're more sanguine, but would you describe yourself as a tech optimist or a tech pessimist? There's three designations, Zoomers, Boomers, and Doomers, right? Did writing your book move you in one direction or another? I think it moved me more, probably more towards doom, more for those environmental reasons that I was talking about. That was one of the real eye-opening revelations is just commonplace knowledge now, but often that we don't see even talked about in much journalism about the consumption.

48:15But in terms of the models themselves, I don't know. I think especially with autonomous driving, I had a really bad accident that could have been really, really bad when my kids were in the car some years ago. This crumpled the front of a car, and it was because of my negligence. And I thought I would much rather have had a machine driving that day. So that part of it, I think, this issue of autonomy and navigation, maybe I am a little more optimistic there. And I think the thing is, you know, artificial intelligence, as you know, is such a sloppy term for all these different things. Right. It's a catch-all.

48:45And LLMs. And so I think that you probably would have to kind of go through a questionnaire to get optimistic or pessimistic. I think we need more of that, you know.

48:55Kara Swisher:This is a wonderful book. It really is. I'm glad. Albert doesn't always pick the books I like, but this one I do. How did that – was that like a shocker to you? That must have been a shocker. Oh, my God. I was just – my hands were shaking. And, you know, one of the things she does is she records the calls that she makes with authors. And in this case, I was so shaking and whatever. My voice sounded kind of understated. So I got dragged a little bit for just to be like, oh my God, he's not even happy. But it was such a lightning strike and so thrilling and flattering. There's a hundred other books published this summer that could have been her summer pick, but she chose culpability.

49:34And I just can't believe it.

49:35Kara Swisher:Certain things she does, the wedding, she shouldn't have gone to. But she does these book things I think is really important. Absolutely. That helps. That's good. Yeah. I would have thought her voice was generated by AI. I wouldn't have believed it. Oh, my God. I still think this is all a simulation. Yeah. The last six weeks. Well, that's your next book. It is all a simulation in case you're – it's some teenagers from the future who are playing a video game right now, and they're enjoying themselves. Anyway, much congratulations. This has been a fascinating conversation, and I really appreciate your book.

50:02Kara Swisher:It's great. I'm recommending it to lots of, you know, people who don't really understand it. It's a really great way to understand. And you don't shy away. You don't stupidize these issues. Thank you so much. Anyway, thank you. Thank you. It's been a huge pleasure.

50:20Kara Swisher:On with Kara Swisher is produced by Christian Castor-Russell, Kateri Yoakum, Megan Burney, Allison Rogers, Lissa Soap, and Kaylin Lynch. Nishat Kirwa is Vox Media's executive producer of podcasts. special thanks to Rosemarie Ho our engineers are Rick Kwan and Fernando Arruda and our theme music is by Trackademics if you're already following the show you get an AI and you get an AI if not watch out for that jumbotron go wherever you listen to podcasts search for On with Kara Swisher and hit follow and don't forget to follow us on Instagram TikTok and YouTube at On with Kara Swisher Thanks for listening to On with Kara Swisher from New York Magazine, the Vox Media Podcast Network, and us.

51:06Kara Swisher:We'll be back on Monday with more.

51:30shelving or sharpening your hedge trimmer blade with an angle grinder. The Home Depot has the tools you need to check projects off your list. Shop Labor Day savings at the Home Depot and gear up for fall projects with the right tools to keep your projects moving.