TECH011: The History of AI and Chatbots w/ Dr. Richard Wallace (Tech Podcast)

0:00You're listening to TIP. Hey, everyone. Welcome to this Wednesday's release of Infinite Tech. Today's episode is a deep dive into the early foundations of conversational AI and what they reveal about today's language models. My guest is Dr. Richard Wallace, a pioneering chatbot creator and three-time Lobner Prize winner best known for building ALIS and the AIML language that powered early conversational systems. The Lobner Prize was an annual competition designed to implement Alan Turing's imitation game, awarding the chatbot that could most convincingly carry on a human-like text conversation with judges, just as an FYI.

0:39So during the show, we talk about why simplicity beats scale in the early AI race, how supervised rule-based systems differ from modern LLMs, what the Turing test actually misses, and why combining symbolic reasoning with neural networks may matter more than raw model size. This is surely an episode you will not want to miss. So without further ado, let's jump right into the conversation.

1:22shaping the next decade and beyond, empowering you to harness the future today. And now, here's your host, Preston Pysh.

1:39Hey, everyone. Welcome to the show. I'm here with Richard Wallace. And wow, this is really exciting for me to talk to such a pioneer in this space, in the chatbot AI space. And first of all, Welcome to the show. Excited to have you here. Thank you. Thank you, Preston. It's a pleasure to be here as well. So where I want to start is I'm just super curious how people kind of fall into their field of expertise. And when I look at what you accomplished very early on back in the 1990s, I'm curious what drove you or motivated you to be paying attention to chatbots and the Turing test and all of that.

2:20it's such an early phase. Because I think for most of the listeners, they know that all this stuff has really come to fruition in the last five or 10 years. It's gone on everybody's radar. But you were doing this literally decades before anybody was even aware of these ideas of chat bots and whatnot. So what was your initial motivation to get into this kind of stuff? That's absolutely right. I like to say that nobody knew what artificial intelligence was until a couple of years ago. And now I'll be sitting in a restaurant somewhere and I'll hear a conversation at the table next to me and they're talking about AI.

2:53Well, anyway, there are several threads that came together that inspired me to work on the chatbot Alice. And I'll just pull on a couple of those threads here. One is that in 1990, I read an article in the New York Times about the first Loebner Prize contest. Now, the Loebner Prize was an annual Turing test, an annual contest based on the Turing test, funded by a rather eccentric philanthropist, Hugh Loebner. And the story with the very first contest was that none of the programs competing came close to passing the Turing test. They were all just terrible chatbots. But Loebner awarded a bronze medal every year to the chatbot that was ranked highest by the judges in terms of being the most human.

3:44And that first year, the bot that won was simply based on the old Eliza Psychiatrist Program, which if you're familiar with that, was a very primitive chatbot developed by Joseph Weizenbaum in 1966. And it had very few responses, but it had some clever tricks to it. It could sort of match keywords in the input and had canned responses associated with those keywords. words, it could invert prepositions. So if I said, I came here to talk to you, then it would repeat back, you came here to talk to me. So it did that sort of pronoun swapping trick. But when I was in graduate school in the 1980s, the Selysa program was basically considered kind of a dead end or at best kind of a hoax in AI.

4:37And not only that, the inventor, Joseph Weisenbaum, ended up pulling the plug on it because he thought it was too dangerous. He thought that people were reading too much into it and was actually there. It was a psychiatrist program, so people were trusting it with their personal issues and problems. They were surprised to find out that Weizenbaum could read all the transcripts of their conversations. Wow. And so he wrote a whole book after that, Computer Power and Human Reason, where he criticized the whole field of AI and his ELISA program in particular. It's really hard to imagine this now that someone would come up with a new AI application that's very engaging and popular, people are using it, then they would say, oh, no, this is too dangerous.

5:25We have to put the genie back in the bottle. I think most people would have run out and try to find venture capital to start a company to commercialize. True. Very true. But isn't this fascinating that the thing that he discovered very early on in the 90s, he started playing around this in the 60s, which is mind-blowing to me. But what he found in the 90s was that there was a huge centralization concern with privacy and what people were putting into these discussions, which is now a major talking point with AI. And it doesn't seem that, I know I'm generalizing here, but it doesn't seem like the population really cares too much or even thinks about these issues that caused him to shut down his entire effort behind this.

6:15I don't know. I find that really fascinating that he discovered this, what, four decades before it became like, or three decades before it became something that the rest of the world should be very concerned about. Well, he really discovered it in the 1960s. Wow. When he first created the program. The other ironic thing about Eliza was that up until very recently, I would say, well, let's say 20 years ago, Eliza was by far the most widely distributed, popular, and well-known AI application. If you knew anything about AI up until maybe the year 2000, then you would know about ELISA. I'm curious, when you read this, I think you said New York Times article in 1990, did you ever think that you would be the winner of this Lobner Prize a decade later?

7:06Well, that planted a seed in my mind, and I didn't really do anything about it for about five years. So another thread that led to the development of Alice or the inspiration for Alice was around that time in the early 90s, it was the end of the Cold War. And so there was decreased amount of government funding available for AI and robotics research compared to the 1980s. And so a number of us in the robotics field, I was working in robotics at the time, got interested in the idea of minimalism, robot minimalism. And basically, that was the idea that we could build robots with very simple, inexpensive sensors and actuators, very commodity microprocessors.

7:52And as a result of that, you could actually get more lifelike behavior out of these robots than you could with approaches people had tried in the past with much larger computers and so forth. One of the interesting inventions that came out of that period was the Roomba. So if you think of the Roomba rolling around and bumping into things and changing its direction, it's all basically just a stimulus response application. We call that stateless. So it's sensing something and then taking an action based on what it's sensing, you know, changing direction, for example. So that whole approach of minimalism, you know, was also in my mind at the time.

8:34And that kind of dovetailed with the very simple approach of the ELISA program, which was also kind of a stimulus response. You know, it was so simple that it could respond very quickly. It didn't have to go and do a lot of computations to come up with the responses. What was your inspiration for thinking that simplicity was going to lead you to better results? Was there something in your life or something that you were reading at the time or what drove you to that intuition? Like I said, we were working on the minimalist philosophy of robotics. Oh, okay. Yeah. At that time, I was working on the development of a robot eye.

9:15And by that, I mean a visual sensor that's based on the architecture of the human eye. So the human eye differs from a TV camera in the sense that a TV camera is basically a square grid of square pixels, but the human eye is more like concentric rings of pixels with higher and higher resolution towards the center. We call that a log map. And so we developed a sensor that had that log map, log map pixel organization. In order to use a camera like that effectively, you have to be able to point it. So we developed a little motor, high-speed pointing motor, based on a direct drive design. And that motor could point the camera, the eye camera, in pan and tilt directions very, very quickly.

10:03And again, it was a very simple kind of actuator, simple sensors, and it could move very quickly, could move actually faster than the human eye. So you'd sort of see this thing whipping around and looking at different things. And it was very lifelike. So just for the audience to understand, so in 2000, I believe 2001 and 2004, Dr. Wallace won the Lobner Prize, which is this Turing test with his ALICE protocol or chatbot that he had created. And I guess for me, what was the major insight that you think that you had back then? You talk about this idea of simplicity, but what would you say was the major insight that you had to outperform everybody else that was competing on what is, I mean, for anybody listening, the most complex, challenging problem you could ever try to go after, right?

10:56What would you say was your keen insight that you had that allowed you to do this? Well, it was basically the idea that I could build on the ELISA program. So the ELISA program had about 200 rules, 200 stimulus response rules, and you could think of that as a pattern and a response. And my idea was to build kind of a super ELISA, where instead of 200 rules, you had thousands and thousands of rules. And in fact, by the time I was entering those contests, I got Alice up to about 50 ,000 patterns and responses. Wow. Amazing. So Richard, one of the things that I found really fascinating about you back at this time was that you came up with this artificial intelligence markup language.

11:43You effectively, for all intents and purposes, and correct me if I'm mischaracterizing this, you had to come up with your own language in order to kind of build efficiency into how this chatbot was working, which is, as a person who's not very good with languages, I'm much more of a math person. I'm reading this and I'm thinking this is mind-blowing. So talk to us about this and what was this insight that you had to come up with the AI ML, artificial intelligence markup language at the time that you did this? Well, AIML is based on XML, and XML was very popular at the time. One thing that appealed to me about XML for the purpose of writing chatbots was that I always say XML has an implicit print statement.

12:28So when you write the responses, you don't have to put in an expression that says print blah, blah, blah, something between the parentheses because the XML already just provides the text inside the markup. So the response is just the text inside the markup. And then basic unit of knowledge in AIML, I call the category, which is like the rules I was talking about a second ago. So the category consists of a pattern that matches some input, some natural language input, and then a response called the template. The reason it's called a template is because it's not exactly the answer, but it's a template for the answer that can be populated with various other things.

13:12And then there was also a recursive element to it where the response could actually simplify the input into a kind of simpler input. So the example of that is, I want you to tell me who you are right now. So you can reduce that by removing the right now. So I want you to tell me who you are. And then you can remove the I want you just so it reduces to just tell me who you are. And then that reduces to who are you. So there was that recursive element built into the responses as well. So in general, you're just, you were taking language and you were making it way more efficient. And they're like, where do you even start with something like that?

13:54I mean, you literally have to go through, there's just so many different variations of language. And I think of the complexity of this, I wouldn't even know where to begin to start writing something that makes it more efficient. Like, yeah. So how did you, how did you think about solving that problem? Well, it all goes back to the conversation logs. So just like Weisenbaum, I can read the transcripts of conversations people were having. By the way, this would have never worked without the internet, without the World Wide Web. Because with the World Wide Web, I could start to accumulate conversations from a very large audience of people.

14:32And by looking at the transcripts of those conversations, I could basically program responses to the things people were saying. Later on, I realized that there was kind of zip distribution over the things people were saying. So, you know, there's kind of a most common thing people say, which is hello, and then who are you and how are you, and I like something. So you can create the responses in order of how frequently people say particular things. Let's take a quick break and hear from today's sponsors. All right. I want you guys to imagine spending three days in Oslo at the height of the summer.

15:14You got long days of daylight, incredible food, floating saunas on the Oslo fjord, and every conversation you have is with people who are actually shaping the future. That's what the Oslo Freedom Forum is. From June 1st through the 3rd, 2026, the Oslo Freedom Forum is entering its 18th year, bringing together activists, technologists, journalists, investors, and builders from all over the world, many of them operating on the front lines of history. This is where you hear firsthand stories from people using Bitcoin to survive currency collapse, using AI to expose human rights abuses, and building technology under censorship and authoritarian pressures.

15:53These aren't abstract ideas. These are tools real people are using right now. You'll be in the room with about 2 ,000 extraordinary individuals, dissidents, founders, philanthropists, policymakers, the kind of people you don't just listen to, but end up having dinner with. Over three days, you'll experience powerful main stage talks, hands-on workshops on freedom tech and financial sovereignty, immersive art installations, and conversations that continue long after the sessions end. And it's all happening in Oslo in June. If this sounds like your kind of room, well, you're in luck because you can attend in person.

16:29Standard and patron passes are available at OsloFreedomForum.com with patron passes offering deep access, private events, and small group time with the speakers. The Oslo Freedom Forum isn't just a conference. It's a place where ideas meet reality and where the future is being built by people living it. Ever wanted to explore the world of online trading but haven't dared try? The futures market is more active now than ever before, and Plus500 Futures is the perfect place to start. Plus500 gives you access to a wide range of instruments, the S &P 500, NASDAQ, Bitcoin, gas, and much more. Explore equity indices, energy, metals, forex, crypto, and beyond.

17:13With a simple and intuitive platform, you can trade from anywhere, right from your phone. Deposit with a minimum of$100 and experience the fast, accessible futures trading you've been waiting for. See a trading opportunity? You'll be able to trade it in just two clicks once your account is open. Not sure if you're ready? Not a problem. Plus 500 gives you an unlimited, risk-free demo account with charts and analytic tools for you to practice on. With over 20 years of experience, Plus 500 is your gateway to the markets. Visit plus500.com to learn more. Trading in futures involves risk of loss and is not suitable for everyone.

17:53Not all applicants will qualify. Plus 500, it's trading with a plus. Every business is asking the same question right now. How do we make AI actually work for us? The possibilities are endless. Guessing is too risky, but sitting on the sidelines, that's not an option either because your competitors are already making their move. With NetSuite by Oracle, you can put AI to work today. NetSuite is the number one AI cloud ERP trusted by over 43 ,000 businesses. It brings your financials, inventory, commerce, HR, and CRM into one single source of truth. And that connected data is what makes your AI smarter.

18:31It doesn't guess, it knows. It automates routine tasks, delivers real insights, and helps you cut costs and make decisions with confidence. And now with NetSuite's AI connector, you can use the AI of your choice plugged into your actual business data. Ask any question you've ever had, key customers, cash on hand, inventory trends. Let's see your competitors do that. I use this and you should too. If your revenues are at least in the seven figures, get the free business guide, Demystifying AI at netsuite.com slash TIP. The guide is free to you at netsuite.com slash TIP. All right, back to the show.

19:08My question for you is, and I'm struggling to find a way to frame this, but you write these thousands of rules and you're also working on a way to compress or make the English language more efficient. What did you fundamentally learn through the experience of writing these thousands of rules and rules of thumb of compression? Because when I think about it, we look at these LLMs and machines are doing all of this really hard and complex work. But I would imagine what you were doing there in the 90s and early 2000s was exactly what all these LLMs are doing today, but you were doing it manually. And so I guess it's almost, I hear these people that say, well, we have no idea what's behind these ones and zeros in all these LLMs, which I guess is a true statement, right?

20:01But if a human was going to maybe be able to understand what it is that it's doing, I think you would be one of the very few people on the planet that could maybe help us understand what that is because you did this manually for so many years. Right. Well, there's so many things wrapped up in that question. So let me see if I can pull it apart. Yeah. So there's always been a kind of tension in the history of artificial intelligence between, let's say, supervised learning and unsupervised learning. So what I was doing was what we call supervised learning because I was playing the role of a teacher or, you know, a guide.

20:41So whenever I added a new response, it was manually added, as you're saying, driven by a particular input that I saw in the conversation logs. And so the way that I'm teaching the robot is by acting as its teacher, basically, and saying, And when you see this, you should say that. And that's in contrast to unsupervised learning, which is what these LLMs are doing. They're basically trying to accumulate a lot of inputs and find the neural network weights that match it to particular outputs. And so with that technique, you can get phenomenal results, obviously. But as you're saying, it's difficult to know how the LLM came up with particular responses.

21:30Whereas in the supervised learning case, where it's all a symbolic process, it's very easy to trace back through the logic of the program and see what caused a particular response to be generated. And I always say that people who do supervised learning approaches spend all of their time doing creative writing, which is what I was doing with the Alice bot. But people who do unsupervised learning spend all of their time deleting crap from the database. And that's sort of what's going on with the LLMs now is, you know, they're having to put a lot of work into filtering to make sure they don't say anything inappropriate or offensive or political.

22:12And, you know, that ends up being a lot of manual work as well. Yeah. And I guess my understanding is that everybody that's on the cutting edge of AI today, like that's the holy grail for them is to get the human out of the loop and for it to be completely AI generated and filtered and just like there's no humans there. As a person who deeply understands this and the way that you frame that is this back and forth and there's consequences to one side and the other, is there a moment where you think that they will be able to get away from complete removing the human out of the loop and it progressing in a way that's actually beneficial?

22:52Or do you think that the more that they lean into removing the human out of the loop, that they actually are setting themselves up for a systemic failure because it's going to spiral into this AI slop, if you will, or it's creating and generating content in a direction that's so fast and so extreme that they get away from human filtering altogether and it just kind of turns into this almost like a runaway virus, if you will. Is that how you kind of see this, that it needs to be balanced? Or is it even possible for it to go in that direction without humans? Well, it's so hard to predict the future.

23:27I would have never expected this whole LLM development to come along in the first place. But I always think of a child learning language, and there are big differences here between a child learning language and an LLM. A kid doesn't have to scan the whole internet to learn how to speak a language. In fact, they're pretty good at what we call one-shot learning. If you say to a kid, this is a dog, then they can instantly recognize every dog in the world as a dog. But what also comes into play here is the supervised-unsupervised learning dichotomy, which is if you are a kid and you have a good teacher and good parents, you'll learn to speak very well.

24:11But if you're a kid who has to pick up language on the street without any supervision, then your language learning won't be nearly as good. And so the LLM is more like the kid out on the street learning language without any supervision. And that's why they learn so much inappropriate and offensive material and so on. What did the wins teach you back in the day when you were winning this about how humans judge intelligence? Well, you know, I can say the same thing about LLMs now that I said about my chatbot back then, which is that people say, well, these chatbots are becoming more and more like humans.

24:49And, you know, I have a different opinion about that, which is that what it's really showing us is that people are more like robots than we would like to think we are. Because it's not that they're robots becoming more like a human, it's that it's revealing to us how robotic we are. And back in the early days of working on Alice, I came to realize that most people most of the time are saying things that they themselves have said before, or that they've heard other people say before. And even when they're writing, they're basically synthesizing thoughts and ideas that are not necessarily original.

25:26And all of these chatbots work because language is predictable, and predictable means robotic. So I always say that if we were all William Shakespeare's uttering an original line of poetry with every sentence we spoke, then these chatbots would never work because they're based on language being predictable, not original reutterance. Yeah. Is it fair to say that you would suggest that humans judge intelligence by their flow or by this response of like most people are looking at that and they're saying, oh, that's intelligence. But then you're looking at it and you're saying that's not intelligence.

26:05It's just repetition. I think that's kind of what you're getting at. Yeah. Repetition, but robotic predictable. Yeah. You know, it's interesting. I just read something, it was like last week, and I think Google came out with this many months ago, but for them to do this long-term learning where it has much more of a memory, it's highly based on whether something's novel or not relative to its index of everything that it's been trained on. And when it sees this novel thing that it wasn't predicting or expecting to come next, that it then stores that in its long-term memory, or I apologize for the terminology here, Dr.

26:44Wallace, but it flags it as something that is worthy of being remembered because it's novel and so different and outside of what would have predicted to be the next thing. And it's interesting that it's in keeping with Claude Shannon's information theory and how it's all aligned. I'm curious if you have any opinions on that in particular, and whether you think that that has a key component to intelligence or how new things are discovered in knowledge in general. Well, that really gets to the heart of what I think the difference is between humans and robots, robots, which is that, like I said, I think most people most of the time are acting like robots.

27:26They're just acting in kind of a stimulus response fashion. Just as an aside, I always used to say that most human conversation is stateless, meaning that what I'm saying to you right now only depends on the question that you just asked me, and we can forget the whole history of our conversation up to this point. You know, one of the pieces of evidence for that is, you know, if you can imagine yourself having a casual conversation with someone at a party, say, and then you say, oh, where did you go to college? And they say, oh, I went to Harvard. I already told you that. You kind of forgot that you had already talked about college earlier in the conversation.