Lessons from the very first chatbot

11 Aug 2026 · 41 min · 15 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

The Verge Cast episode examines ELIZA, the first famous chatbot (1960s), and what its “Eliza effect” teaches about why people believe chat systems and how that relates to today’s LLMs.

Guests (backgrounds)

David Berry, professor at the University of Sussex; Mark Marino, professor at the University of Southern California. They co-authored Inventing ELIZA, a book based on excavated ELIZA source code.

Key claims

ELIZA’s success came from simple conversational tricks (keyword matching, asking disarming questions, keeping the dialogue moving), not real understanding. Weizenbaum feared the “magic trick” would be harmful when people treat computers as human-like judgment-makers. He distinguished calculation from judgment (inspired by Hannah Arendt) and later wrote Computer Power and Human Reason (1976), becoming a “heretic.”

Notable examples

ELIZA’s “doctor” script/therapist-style dialogue; the “Wizard of Oz” con analogy; Weizenbaum’s MIT concerns about dehumanizing language; modern parallels like AI boyfriends and mental-health commercialization.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

The Origins of Eliza

0:45 to 4:14

A discussion on the first chatbot, Eliza, its functionality, and impact.

“It was so effective that it convinced a lot of people that there had to be a human on the other end.”

Joseph Weizenbaum: The Mind Behind Eliza

4:14 to 9:47

Exploration of Joseph Weizenbaum's background and his creation of Eliza.

“I am joined now by two of the co-authors of Inventing Eliza.”

The Philosophy of Eliza

9:47 to 14:01

Discussion on the deeper implications and philosophical questions raised by Eliza.

“Mark, this is an interesting moment in time that you guys reckon with a bunch in the book is kind of this question of what was Joseph Weizenbaum building when he built Eliza.”

The Evolution of Human-Computer Interaction

14:01 to 16:30

Explore how early chatbots like Eliza shaped our understanding of AI and interaction.

“the interface experience that most people would have would be with a tele typewriter um so it was on paper with this clattering, incredibly loud device that you would type your request into the computer.”

The Magic of Believing in Technology

16:30 to 21:12

Discuss the psychological aspects of believing chatbots are human and its implications.

“There are lots of people who cannot be convinced that there is not a person on the other end.”

Weizenbaum's Crisis of Faith

21:12 to 24:55

Learn about Joseph Weizenbaum's reflections on the implications of AI and human interaction.

“because it's not a single moment of conversion.”

Weizenbaum's Crisis of Faith

25:58 to 26:50

Learn about Joseph Weizenbaum's reflections on the implications of AI and human interaction.

“Business owners know that sometimes you encounter problems that can easily be solved with an extra set of hands.”

Political Context and the DSA

28:01 to 28:34

Discussion on political implications surrounding the DSA and historical references.

“I would be the greatest communist in history.”

Humans vs. Computers in Understanding

28:35 to 29:39

Exploration of the distinction between human understanding and computer calculation.

“I wrote down a line from your book that he drew a distinction between calculating and understanding.”

The Limits of Computational Trust

29:40 to 30:45

Analysis of the trust we place in computers versus human judgment.

“But he seemed to understand that computers were very good for certain things, but were not to be trusted or even asked to do some of the things that humans are supposed to be doing.”
Show all 15 chapters

Understanding the Human Condition

30:46 to 31:45

Discussion on how tech leaders leverage humanities education in AI.

“Computers don't, by their programming, just care for other computers.”

The Current AI Landscape

31:46 to 34:16

A deep dive into the present-day implications and financial aspects of AI advancements.

“I mean, the one thing just to mention that Weizenbaum wasn't doing that a contemporary chatbot does do is that he wasn't extracting data from his users.”

Concerns Over AI Dependency

34:17 to 36:36

Examining the societal impacts and mental health issues related to AI technology.

“Wait, are you saying winter is coming, David?”

Explaining AI Systems

36:37 to 37:58

Discussion on making understanding of AI systems more accessible and transparent.

“And then everything that Weizenbaum predicted, we see playing out in people being deluded, people taking their own lives, people out of consultation with these chatbots.”

Weizenbaum's Legacy and Modern Concerns

37:59 to 39:58

Speculation on how Weizenbaum would react to today's technology and its implications.

“But I do wonder if that's even possible at this moment.”
Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:02Hello and welcome to The Verge Cast, the flagship podcast of If Then Statements. I'm your friend David Pierce and today on the show we're talking about chatbots. Specifically the very first chatbot from the 1960s. It was called Eliza and the way to think about it is basically as a very simple back and forth experience that you might have with a computer. One of the first instantiations of Eliza was this thing called doctor which would play therapist for you. And it would ask you how you were doing and you would respond and you'd say, I'm having a bad day. And it would go through and look for specific keywords and ways that you responded and respond back to you.

0:36And it was able to carry on a pretty functional conversation. It remembered things about you. It could ask questions. It could move things along. It could make connections. It wasn't very sophisticated, just a few hundred lines of code, but it worked. It was so effective that it convinced a lot of people that there had to be a human on the other end. It was a fascinating product made by some fascinating people, and a group of academics and researchers and writers recently has excavated all of the source code of ELIZA, and they've written a book called Inventing ELIZA about where it came from, who created it, and the lessons it might have for us in the current world of AI.

1:14Because, spoiler alert, there are a lot of them. And the way that we think about computers really hasn't changed that much since the 1960s. It's going to be very fun. I'm really excited to dig into it. But first, here's everything else happening on The Verge today. This is 90 Seconds on The Verge for Tuesday, August 11th, 2026. Have you ever wondered why your Amazon emails are so unhelpful? They just say, like, your wireless accessory is confirmed, instead of giving you anything useful or specific? Well, The Verge's Miyasato wondered too, and she found out that it's all about AI data. Basically, if Amazon puts your orders in the email, suddenly Gmail knows what you're buying, which means Gemini can do a better job of steering you to stuff you like on Google Shopping.

1:54Amazon says it's also about privacy and convenience, but this is a company that is also fighting perplexity and others to keep AI agents from taking over any of your shopping experience. Because it's very important to Amazon that you go to Amazon.com and accidentally click on a bunch of barely labeled ads. That's the business. Meanwhile, David Ellison, the CEO of Paramount, has reportedly decided to move his company out of California if the state's attorney general refuses to settle the antitrust case challenging Paramount's acquisition of Warner Brothers Discovery. This is all obviously a threat.

2:26The case isn't set to go to court until next March, and Paramount will start owing shareholders about$7 million a day this October. Ellison would very much like to not spend that money. But if you're wondering how serious this plan actually is, Paramount reportedly hasn't even decided where its new headquarters would be, and it would be scheduled to move there in like six weeks. I'm not sure this is a real plan. At least not yet. Finally, we're only a day away from Google's next big launch event where we're expecting new Pixel phones, including maybe new foldable, a new Pixel Watcher 2, and maybe even an AirTag-style Pixel tracker.

2:59There's a big event tomorrow night hosted by Trevor Noah, but if it's anything like Jimmy Fallon's weird informer show last year, you can safely skip that one. You can read more about all this at TheVerge.com. That is 90 Seconds on The Verge for Tuesday, August 11th. Are you a pet owner? Every six seconds, a pet owner in the U.S. gets hit with a vet bill over$1 ,000. Always an unwelcome surprise. That's where Fetch comes in. Fetch is the most complete pet insurance for dogs and cats. According to consumeradvocate.org, you get paid back up to 90 % of vet bills at any vet in the U.S. and Canada, with claims paid back in as little as two days.

3:40Go to FetchPet.com slash save right now for your free quote. That's FetchPet.com slash save.

3:53Hey, it's Sue Bird. This week on Bird's Eye View, I'm joined by Kayla McBride. We get into her unstoppable three-point game and what getting Nafisa Collier back means for the Lynx playoff run. Plus, I take a look at why Kelsey Plum landed in Phoenix at the trade deadline. Check out Bird's Eye View on YouTube and your favorite podcast app. All right, let's talk chatbots. I am joined now by two of the co-authors of Inventing Eliza. David Berry, professor at the University of Sussex. Welcome to the show. Thank you. And Mark Marino, professor at the University of Southern California. Welcome to the show.

4:25Hello. Thanks for having us. You're the American one. David's the British one. If you're listening, that's a useful way to remember who's who on this one. Yeah, otherwise we're indistinguishable. That's right. You would never be able to tell the two of you apart, except for the accents. So it's good that we have those. I want to go back to the beginning here, and I want to talk about Joseph Weisenbaum a little bit, because I think a lot of the study that you guys have done is of Joseph Weisenbaum and his own feelings about the thing that he created. So give me just a flavor, David, of who Joseph Weizenbaum was.

4:58So Joseph Weizenbaum is quite an incredible character. I think that's fair to say. I mean, he was born in 1923 in Nazi Germany. He was Jewish, born to a kind of upper middle class family, actually. And of course, they start to suffer under the rise of the Nazis. and so his family emigrated to the US in 1936 he was 13 years old so that early on he was already living quite an incredible life he moves to Detroit and becomes Americanized though he never fully loses his German accent goes to Wayne University now Wayne State University I believe and there he studies mathematics because he doesn't feel his english language is strong enough for any other subject and he was a natural at mathematics and of course mathematics in the 1940s coming into the 1950s is really seething with the excitement over digital computation we've got a lot of interesting experiments taking place and wayne state is actually where one of the first digital computers is being made but being made by hand and weisenbaum is actually part of the team who are actually soldering together and trying to figure out how to to make this object and so he's very much um right there at the very beginning of the kind of turn to the digital he's living the things that we can only read about now and the excitement of that time but also he's a very fascinating character he's not a usual um technical person he's got a very well-rounded personality, which I think probably comes out of that German-Berlin sensibility, where culture is so important to one's life.

6:45Consequently, he has an interest in everything, and he's particularly fascinated by the American notion of the con. A wonderful book called The Con Documents is, for a Brit, actually quite a strange part of American culture, which is that There were entire societies set up just to con people who would build semi-film sets. This was in the early part of the 20th century, so 1910s, 1920s, where they would literally pull people in just to fleece them of their cash. And it was really quite a remarkable thing. And this book, The Con, documents it. And Weissberg was absolutely obsessed with this book.

7:24So this was the idea that one could build an artifice and the artifice behind it would hide all of the work that was being done to make this artifice work. It's very kind of dramatic, like a theatre, really. A theatre made to steal money from people. And so when he's working at Wayne State, he starts thinking about these questions and he goes to work. he works for General Electric who at the time had been tasked with producing a new computer system really to save the banking system in America because of the use of paper checks I don't know if your readers and watchers will ever remember those but essentially that's how we used to pay money between organizations and people and the quantity of checks was so huge that they couldn't be managed by human tellers anymore so they needed to computerize the system and Vitamount was very much part of that computerization process.

8:21And he almost certainly would have witnessed the very famous situation where General Electric were asked to give a demo to the Bank of America. The system didn't work. And so they had to do a con, which is that they set up a facade of the computer system. And in the back room, where the system wasn't finished yet or had too many bugs, people would literally fill in the bits to make it kind of work. the Wizard of Oz demo, right? This is the man behind the curtain in a very real way. Yeah. Essentially, Bank of America didn't get wind of it at all. And they gave the contract to General Electric and the rest is history.

8:57Eventually, luckily, I suppose, that system worked and was a great demonstrator of the power of digital systems. And from that, really, that I think inspired Weizenbaum to think about the difference between what we would now call the interface or the surface of computation and what's going on behind. They don't have to be the same thing. And so he starts to already early on start thinking about, you know, what does it mean to present something to a user? And he writes a very famous paper called How to Make a Computer Appear Intelligent in 1962. And he says, you know, it doesn't matter if the system behind isn't intelligent.

9:38What matters is that people think it is. And you can see already the bits of Eliza falling into place for him to go on to do it. Mark, this is an interesting moment in time that you guys reckon with a bunch in the book is kind of this question of what was Joseph Weizenbaum building when he built Eliza. And there is a sense that he made a therapist, which you prove pretty conclusively is not true, right? That's the demo everybody remembers where you're talking into a therapist. That was not the original or only conception of Eliza, that it's actually a collection of chatbots. But it also doesn't seem like he was building what he perceived to be some kind of breakthrough user product that someday somebody would sell and make lots of money on, that there was something more philosophically questioning about this thing that he was building.

10:29What do you think he was building when he was building Eliza in those early days? I mean, that's a great question, David. I think it's complex. So Jeff Schrager, one of our collaborators, he's fond of just pointing to the title of that initial paper in 1966, ELISA, A Computer Program for the Study of Natural Language Communication Between Man and Machine. So there's one version of this where he's doing an experiment to see what happens when people interact with a computer. I don't think it's as simple as that, I guess. And this is maybe this is why it's wonderful to have eight contributors to a book all in dialogue over the course of four years.

11:07Yeah. Because on the one hand, you know, again, Peggy Weil, another one of our contributors, loves to quote the first line of that article, which is, it is said that to explain is to explain away. So in other words, here's my clever little chatbot. Now I'm going to explain it to you. It's like one of those Penn & Teller's, you know, magic tricks, right? They do the magic trick. Then they show you how they do the magic trick. Right. And then that's also a bit of a performance. I think where things get really complicated for me is, and then he includes this dialogue called Men Are All Alike, or that's how we refer to it, right?

11:42And it seems to be between a young woman and this doctor script. It's so convincing. It's so interesting. And it ends in this mic drop moment, right? Where the therapist seems to draw back around the conversation about the father and the conversation about the boyfriend. And it's imagine, you know, at this time where, you know, psychotherapy is popular in the consciousness of America, right? And so also some of our efforts were in trying to figure out how he was the only person who could have, Weisenbaum himself could have known that that conversation, if you said it in a certain way, could have led to that particular moment.

12:23But I guess the reason why I'm calling our attention to that dialogue is that that dialogue to me says that he was also invested in Eliza and Dr. Seeming like it really did what, you know, it could converse, right? And he continues to tell the story about his secretary closing the door, you know, and having a private conversation with it. And so it's like these dueling impulses. On the one hand, I'm going to explain everything to you. On the other hand, I'm going to do the magic trick. And it is a good magic trick. Actually, this might be a useful thought exercise. I think a lot of people watching and listening to this will have had lots of experiences with chatbots.

13:05In the 1960s, just put me in the frame of mind of somebody sitting down in front of Eliza. So to give you an idea of what computation was like, because already the computers were becoming commoditized, right? So DEC had produced the PDP-1. and in fact um mit received one of the first pdp ones as a gift uh from deck and so already people were moving away from the idea that computers were hand soldered together they would purchase them sort of pre-made as it were but the pdp one when it was delivered had no operating system right the first thing you would have to do is write your own operating system to use this computer and then you'd have to write all your tools and then you might be able to have enough time to write some actual software that did something that was quite interesting so it was quite an uphill battle to get these machines working secondly i would say that um they were fairly primitive um so the idea of a screen in of itself um was a really new idea and and in fact most of the interface experience that most people would have would be with a tele typewriter um so it was on paper with this clattering, incredibly loud device that you would type your request into the computer.

14:21It would clatter away and then give you an answer on paper. So it was very clunky, very slow, very loud, very large, actually, and quite hot because these machines were obviously burning a lot of electricity. But what I would say to you is there was also a kind of imaginary there as well. I mean, Vannevar Bush's Memex, Linklider, and lots of other people were already trying to think beyond the existing restrictions of the hardware of the time and dreaming of new ways of seeing what human-machine interaction could be. And I think that Eliza certainly was part of that because it was part of the project Mac project at MIT, which was explicitly about how can we use computers in a different way?

15:14I always like to muse on this question, how could people have thought that this chatbot was a person? It's so rudimentary, although of course, as we've already mentioned, so clever. I think what David just described of not having experience of anything happening between you and a computer, perhaps for a lot of these people. And then least of all, instead of having a screen interfacing something just by, you know, we're a lot closer to the Turing test imitation game, passing notes with someone who's behind a curtain than we are anything like we're experiencing right now. So if you get, if text goes in and text comes out, You know, I mean, I hate to hearken us back to like a telegram, but I mean, communication is happening pretty much the same way it could happen with another human.

16:07And you had no experience of communicating with another human through a computer at all anyway. Right. Well, OK, this is actually the reason I bring this up is because one of the things I have been trying to reckon with for a long time and was thinking about this a lot reading your book is overwhelmingly people who sit down and experience this very simple, very basic chatbot, Eliza, fall for it. They believe in the thing. There are lots of people who cannot be convinced that there is not a person on the other end. The experience works. Like you said, Mark, the magic trick is successful. and I've been trying to decide is there something about that that is fundamentally about inexperience with computers and if you just gave that to a bunch of people who had used this kind of technology before if it would have been less new and less surprising and less magical or if there is something deeply human inside of us that just desperately wants that magic trick to work and my sense is it's probably a little bit of both but I know you've both spent a lot of time thinking about this David, where do you land on this?

17:11So, you know, this is a fundamental question. I would take you back to the early part of the 20th century. When you read the cons that were taking place, you know, you read them with a sense of kind of disbelief that people were taken in. It seems utterly, to me anyway, that American, rich Americans would visit this fake bank and would be wowed by the fake technology that they were showing and the fake tellers and the fake stock exchange and then would hand over hundreds of thousands of dollars, millions today. And then when the con people ran away with the money, which is what they did, they ran, got to the train station and they'd be gone, the people would pull back the screens and it would be paper mache, balsa wood and why would they believe it?

18:03I think, as you said, there seems to be a deep kind of desire for us to want to believe. And maybe we believe because it seems like such a good offer. And Eliza seems like such a good offer, right? You had a therapy time, which was free. I mean, that's a good starting point. And it listened without putting you down or attacking you. It was a very nice experience. And it's also novel, right? kind of interesting to chat to a computer and and so on so forth so i think many of things play together in the same way that today you you hear about people having ai uh boyfriends and girlfriends and you kind of probably can't believe and there's a wonderful interview in the new york times actually with somebody who had an ai boyfriend and they said you do realize it's a machine she's like yeah yeah but that's what makes it so good i can turn it off when i have enough so that we're able to do, and this is the theater, isn't it?

19:02We walk into a theater. We know the theater is a theater. We know they're actors, and yet we can allow ourselves to just become and listen to and experience the play. I mean, I threw two more things into that bucket. One is, it's asking you how you're doing, right? Which is a very disarming question and puts us maybe in a bit of a vulnerable state, especially if we need to express that as probably most humans need to do, right? So that's step number one. And I think we see that in a lot of chatbots. The second, and of course, the design of it is brilliant that it keeps asking questions. Of course, the, you know, ChatGPT and all those other frontier models have figured out that trick.

19:40They end every sentence in a question. Again, drawing from the expertise of our members, Jeff Schrager is quick to talk about the kind of repair we do conversationally, even to have a conversation with another human. And then maybe a good example might be when you receive text messages from someone who's using speech to text and they're kind of terrible, right? But then you do all this repair work to put it together. The reckoning with all of this strikes me as the thing that then Joe Weizenbaum goes and does, right? Like he has this real, I think I get the sense he was surprised by how well Eliza worked that he's like, I built this thing that seems like an interesting way to communicate with computers.

20:24And then he's like, oh my God, people are pouring their heart and soul into this thing thinking that the computer is a person. And he seems to, rather than do what it seems like everyone else did at the time and what certainly everybody's doing right now, which is say, you know, oh my God, I just got rich. This is going to be incredible and technology is going to take over. There's Scourge McDucking rubbing their hands together. He has a real crisis of faith in all of this. And before we skip to now, I want to get to kind of where Joseph Weizenbaum got to at the end of all of this process, at the end of all of this thinking.

20:58He seems to have a real kind of, you know, Oppenheimer, what have I done kind of moment at the end of all of this. David, where does he net out after seeing what Eliza accomplishes and does to people? So I think that's quite a complicated question, actually, because it's not a single moment of conversion. Remembering his kind of experience with the Khan and the reveal of the Khan, he loved this idea of doing the reveal. And he used to play with his children. He would write little programs for them, and then he would reveal that it was actually him that was the Wizard of Oz that was making the system work.

21:38And I think firstly what worried him was when he did the reveal, you know uh people like yes joe could you leave the room well stop stop ruining the experience so that that's the first thing that's kind of interesting the second thing is is that people don't want to have the magic revealed to them i think this becomes a theme that i want to i want to stay on here but yeah but i but i think that's right people even if he says look it's a trick i did it everybody says no no no go away it's more fun to leave in the trick right yeah the worst cut the worst kind of person is you're in the the theater watching a play and they're like you know this is a play this isn't real i'm like thanks but um so that was the first thing that that that question but i think the second thing was was that he began to notice that people in mit were talking about humans as if humans were sort of robots or disposable and this really concerned him because, of course, it reminded him of the language of Nazi Germany and the language of how the Jews were described by the Nazis as non-human, not important.

22:47And Minsky talked about humans as meat machines. And he thought this was utterly unacceptable, actually. And this is a time, of course, with the rising political crises in America. You've got the civil rights movement. You've also got the Vietnam War. You've got the use of particular kinds of rational management in government to run the Vietnam War with little understanding of human context and human nuance. And all of this surrounds him together with Hannah Arendt's work that he was a big fan of and deeply read her work. And also, of course, Mumford, Lewis Mumford, another very important influence on him.

23:33And the context of the time is so important. There was this concern about what was happening in America. What is happening to us? Where are we going wrong? And so I think all this adds up to a kind of a tipping point, really, where he says to himself, I'm actually not equipped to deal with where I am, this place at MIT. I'd wanted to get here, the pinnacle of my career. and yet now I'm here, I'm seriously doubting my place. And so very strangely, particularly for a computer scientist, he applies to do a kind of humanities fellowship, actually, where he goes off and studies humanities texts, a whole range of texts, philosophical, sociological, and political, and eventually tries to put this all together into a book called Computer Power and Human Reason, which is, of course, that really is the turning point in 1976, which says computing, and particularly artificial intelligence, is going wrong.

24:43There is something very wrong with the way we are approaching this topic. And I will just add that made him a lot of enemies, both at MIT and across the field. He became a heretic. Support for the show comes from Superhuman. It's summer, which means a lot of us have vacations planned. But there's one part of going on vacation that's not all that fun. Coming back to work. Your inbox is full, tasks have piled up, and it feels impossible to stay on top of everything. Superhuman Go is an AI chat that's always there when you need it, already aware of what you're doing and doesn't ask you to start from zero.

25:20Go, from the makers of Grammarly, works inside the tools and sites you already use. meaning you can stay focused on the most important parts of your job. No new tabs, no switching apps, no losing your place. Say you've just gotten back from that vacation and you're trying to figure out where to start. You can use Go's AI chat to summarize a long email thread that started while you were out. Or maybe you have a day of back-to-back meetings. Superhuman Go can help you prep in seconds. This is AI that works for you, not on top of you. Check it out. Find out more at superhuman.com. Support for the show comes from Upwork.

26:00Business owners know that sometimes you encounter problems that can easily be solved with an extra set of hands. But they also know how complicated it could be to find that extra set of hands. Upwork exists to streamline the process. Upwork is a one-stop platform to find, hire, and pay expert freelancers. You get fast access to specialized talent across more than 125 categories. On Upwork, you can browse profiles, review past work, and get help scoping the role so you can hire with confidence and get started quickly. You also don't need to let the operational tedium get in your way, since Upwork handles things like contracts and payments.

26:38They also have Business Plus, which gives you access to the top 1 % of talent on the platform. With AI-powered shortlisting, you'll get matched to the right freelancer in under six hours. No endless searching required. Visit Upwork.com right now and post your job for free. That is Upwork.com to connect with top talent ready to help your business grow. That's U-P-W-O-R-K.com. Upwork.com.

27:09Are you a pet owner? Every six seconds, a pet owner in the U.S. gets hit with a vet bill over$1 ,000. Always an unwelcome surprise. That's where Fetch comes in. Fetch is the most complete pet insurance for dogs and cats. According to consumeradvocate.org, you get paid back up to 90 % of vet bills at any vet in the U.S. and Canada with claims paid back in as little as two days. Go to FetchPet.com slash save right now for your free quote. That's FetchPet.com slash save.

27:49Lenin. Democratic socialists keep winning primary races. In Michigan, Abdel al-Sayed, not in the DSA, but endorsed by them, just won a fiercely fought Senate primary. President Trump appears worried enough about the DSA to have started conflating some things. I would be the greatest communist in history. I'd be right up. Free rent for the rest of your life. What they don't say is that you'll be living in squalor in 12 months. The Trump administration has raised the specter of the far left in various ways during Trump's second term. But the recent arrest of an American communist in Spain, what year is it, at the request of the American government is a more concrete step.

28:29Coming up on Today Explained from Vox, real communist threat or new red scare? Today Explained drops every weekday.

28:49I wrote down a line from your book that he drew a distinction between calculating and understanding. And there's a real thread in all of this that I had not really thought of in this way, but is the sort of turning idea that humans are complicated and irrational and messy and impossible to understand. And by comparison, computers and math are perfect and rational and thus better. And I think we are in a very real way living in that moment right now where there is a belief that computers are better than humans, that they are more predictable, that they are more understandable, and that those are good things and we should thus trust them more than we trust humans.

Read the full transcript

29:29I wonder if you can just pull apart the way that Weizenbaum in particular was thinking about this, that we have computers that are very good. I don't think Weizenbaum ever turned against technology. He was never like a, you know, we have to unplug everything and run back into the woods. But he seemed to understand that computers were very good for certain things, but were not to be trusted or even asked to do some of the things that humans are supposed to be doing. How would you say he drew those lines? Yeah, I mean, again, the context is really important, right? So we have Robert McNamara in the US government who essentially thought you could just import corporate management around making various kinds of objects into the Vietnam War.

30:13that he saw the Vietnam War, the problem being bad management. And so they try to implement these very instrumental, these very rational systems into government, and they make it worse. They make the Vietnam War a lot worse. And so the line about calculation and judgment is actually inspired by Hannah Arendt, who wrote a book that was highly critical of this turn in the US government. And she said they calculate, they do not judge. And of course, this was obviously very inspirational for Weizenbaum. And that distinction, that really important distinction between calculation and judgment, it's really a humanistic one as well, that there is a specialness to humans in that the way in which we approach something is with a lived experience, that we actually care about the world and we care for each other and we care for the environment.

31:10Computers don't, by their programming, just care for other computers. They don't care for the users that interact with them, and they have no care for anything else either. So I think that's a very important insight he found, and a very convincing one as to why we should be very careful about using computers in all aspects of our life. I mean, I think this jumps ahead a little bit, but I think it's interesting to think about the way the Silicon Valley oligarchs cite their humanities background as part of their resumes. At the same time that that's the stuff that's used for the trick and the deception.

31:50I mean, the one thing just to mention that Weizenbaum wasn't doing that a contemporary chatbot does do is that he wasn't extracting data from his users. And, of course, building mental dependency on them the way they are today. So there's, again, Wysimam may have created one trick, but compared to the number of tricks that are being played on people, again, by, what would you say, weaponizing the Eliza effect? Well, yeah, let's talk about where we are right now, because I think I mean, obviously, a lot of the stuff that you guys have been describing is very familiar right now. We are in this moment right now where the novelty effect of LLMs has been so intense that people have had genuine good faith discussions about whether these models are alive, which just to be clear, they're not like they aren't.

32:41But I think that the fact that we are back in this 60 years later, and again, with a huge amount of computer fluency and history with this stuff that suggests that we should be better at this, we're not. And I think a lot of the stuff that you guys have been describing still applies. This sort of deeply human idea that we want to be talking to other humans. We want to be heard. We want to talk about all this stuff. Are we just doing this again? Does it really feel to you guys like looking at the current moment that we're in, that we are just reliving a full circle version of what we did in the late 60s and early 70s?

33:17Or is there something different about the LLM moment as opposed to the ELISA moment? Well, if I could pick that up, I think there is something different. I mean, I don't think history repeats itself or certainly doesn't repeat itself exactly. There's no doubt about it that the technologies in LLMs and other kinds of diffusion architectures are novel and remarkable. And to me, astonishing that the scaling effect seems to be paying off in terms of new models and new systems. I think that's not debatable at all, really. But where we are in a similar situation is these kind of bubbles of excitement around a technology.

34:00And AI has been through, this is now the third kind of AI summer, if you like, before the usually inevitable AI winter follows. And the problem today, of course, is that the AI systems and the AI companies are linked heavily to huge debt exposure. Wait, are you saying winter is coming, David? Is that what you're saying? Are you saying winter is coming? And it's certainly the case that the finance is out of control. And of course, there's an incentive on the companies to continue to push it because both OpenAI and Anthropic won to IPO hopefully this year. And to do so, they need the excitement behind the AI bubble.

34:41But nonetheless, we do have to take our hat off to them and say that the technologies they've developed are really rather remarkable. Anybody who's used either of the models of ChatGPT or Claude to, for example, program systems, it's quite remarkable. And there's no doubt that computing and computer programming, particularly software development, is going to be under a paradigm shift, I think. I mean, it's such a fascinating question because, you know, on the one hand, we've had so much. So everything has accelerated and all the vectors coming off of this. So we talked about the beginning about people having no experience interacting with a computer or even other humans through digital means or computational means.

35:29Well, now we all have that 24-7, right? Also, I think there's an increase of isolation and loneliness that's going on, right? There's more awareness of mental illness. These companies are so much more wealthy than probably even the U.S. government was at the time when it was funding these operations. And so their incentives are so high. You read a book like Karen Howell's book about open AI, and you see terrifying things. So a couple of things just to mention. And Weizenbaum was hand programming all of these effects. ChatGPT's ability to speak is like from where I sit. Again, I'm not an expert in this, but an epiphenomenon.

36:20Like, oh, look what happens when we increase the parameters. All of a sudden, it can do this thing. And then there are a few safety people who say, and by the way, let's not do that because terrible things are going to happen, right? Yes. And so think about some of the people who started Anthropic and things like this and the security people and all the concerns they've said. And then everything that Weizenbaum predicted, we see playing out in people being deluded, people taking their own lives, people out of consultation with these chatbots. So I don't know. I mean, in some ways, again, that's I feel like you read computing, computer power and human reasoning and you've you've got you've got your marching orders, which is, you know, maybe maybe a little regulation at this point would be sane if if a society wants to stay stay healthy.

37:14one of the real prescriptions for how to solve a lot of this is better understanding of how the trick works, right? And this goes all the way back to the cons. Even a big part of your project has been we want to get into the actual source code of understanding how Eliza worked because if we can understand how it worked, we can make sense of it better. It actually becomes very important to see the thing. And Mark, I know this is a thing you've been working on a lot is like how do we make code explainable? I'm curious how you point that thinking at something like the LLM phenomenon, which is kind of by design an unknowable black box.

37:52A lot of the people who work with and build and operate this technology profess to not know exactly how it works. And I think I agree with the principle that we should all understand how our computers and how our systems and our platforms work much more clearly. But I do wonder if that's even possible at this moment. Yeah, I think so. And I'm going to point this back to David in a second, but I'll mention, so we've been building some tools lately to help visualize this, to help break the spell, I guess you could say. One tool I've been working on, and admittedly I did vibe-coded, which maybe invalidates it, but is after someone very close to me was telling me they were consulting their computer on personal matters, I built a little web app that's called the Oracle where you can ask this app questions about your future, what you should do, making certain binary choices between, you know, should I keep my love for the theater or get a practical job?

38:49And it'll answer you. But then you change the temperature or you change some of the other weights and you get a very different answer. So just to sort of help people think about, okay, this is a mathematical, this is a stochastic process that's getting us to some sort of answer. My first kind of reflection on this is that Vico, the theologian who was writing in the, I think it was the 1600s, he wrote very famously that if God made it, then it's impossible for us to understand it. Because obviously God is an omnipotent being. But something that humans make, we can understand, right? And I think that principle has held pretty well for 400 years.

39:32And it still holds today. We humans have made these machines. We can work out how to understand them. Do you think Joseph Weizenbaum, who didn't live to see the Transformers era of LLMs and all the stuff we're going through right now, Do you think he would have used ChatGPT and been sort of thrilled by how successfully it pulled off the game that he was trying to play 60 years ago? Or would he be running around screaming to everybody about how the apocalypse has come and all of the things he's been warning about losing our humanity has all come to pass? What do you think he would make of the LLM moment in technology right now?

40:10It depends how old he is, I think, when he encounters the technology, right? Right. Because like, again, I hate to put him totally in the sort of chicken little camp, not the chicken little camp, but the the, you know, person who's trying to tell us that there's a gremlin on the wing or, you know, like I hate to put him totally in that camp because he I think he would. There would have been a moment where he would have been fascinated by by the possibilities and would have explored and would have pushed boundaries and would have and would have maybe even extended the technology. like so many people who are working for these companies right now.

40:44But I think the horror would set it pretty relatively quickly. I don't know, David, what do you think? I think that's a brilliant answer, Mark, to be honest. I think you're absolutely right. The very early Weizenbaum would have loved this technology. I mean, it's the ultimate con. I mean, it really is. And there is the potential to reveal the con, to reveal the trick, to reveal the illusion. and the the mid to late weisenbaum would be very concerned i think at the commercialization of these technologies remember he was working on university research projects to see it used in the ways in which it is being done for ai boyfriends and girlfriends commercializing mental health education so so quickly with very little regard to thinking about that distinction between calculation and judgment, I think he would be very, very worried.

41:35But I think his book, Computer Power and Human Reason, is still very valid today. The discussions, although the technologies may have shifted, the problems that he identifies and the things that we as humans have to take account of are still very, very live. True. All right. Thank you both so much for being here. This was great. I really appreciate it. Thanks for having us. Thank you. All right. That's it for the show. thank you to David and Mark for being here. And thank you as always for watching and listening. If you have thoughts, feedback, questions, any of that, I want to hear all of it.

42:08I think 1960s technology speaks to us today in really interesting ways. And I want to know how it made you feel. Send us an email, vergecasttheverge.com, call the hotline 866-VERGE-11. And by the way, we'll put a link in the show notes to all of the Eliza archaeology project that these guys have been talking about, including the recreated interactive Eliza. It's fascinating. And I think your mind might be as blown as my was by how well the thing does its job. As always, if you want to support everything that we're up to, the best thing you can do is subscribe to The Verge, theverge.com slash subscribe.

42:40It gets you all of our podcasts, including this one ad free, gets you all of our exclusive newsletters, gets you all of our coverage of AI and everything else. Theverge.com slash subscribe. Thank you in advance. The Verge Cast is a Verge production and part of the Vox Media Podcast Network. The show is produced by Josh Cajas, Eric Gomez, Brandon Kiefer, and Travis Larchuk. We'll see you tomorrow. Rock and roll.

From the publisher

Before Alexa and Siri, there was ELIZA. The first chatbot ever was the creation of a man named Joseph Weizenbaum, who built the bot almost as a magic trick — and then couldn't believe how many people fell for it. David Berry and Mark Marino, two members of a team dedicated to restoring and understanding ELIZA, explain how Weizenbaum created the virtual conversant, why he came to regret it, and what ELIZA can teach us about life in the AI age.

Further reading:

Why your Amazon order confirmation emails have become so unhelpful

David Ellison’s ready to pull Paramount out of California

What to expect from Google’s 2026 Pixel hardware launch event

The ELIZA Archaeology Project

Inventing ELIZA: How the First Chatbot Shaped the Future of AI

From Eliza to ChatGPT: the 60-year history of chatbots

Subscribe to The Verge for unlimited access to theverge.com, subscriber-exclusive newsletters, and our ad-free podcast feed.

We love hearing from you! Email your questions and thoughts to vergecast@theverge.com or call us at 866-VERGE11.
Learn more about your ad choices. Visit podcastchoices.com/adchoices

More from The Vergecast

All 445 episodes
Lessons from the very first chatbotThe Vergecast · 41 min
Listen in VO