Decoding the AI Dilemma

0:07Hello, and welcome to another episode of the duct tape marketing podcast. This is John Jantz. My guest today is Kenneth Wenger. He's an author, research scholar at Toronto Metropolitan University and CTO of Squint AI Inc. His research interests lie in the intersection of humans and machines, ensuring that we build a future based on the responsible use of technology. We're gonna talk about his book today is The Algorithm Plotting Against Us, a lay person's guide to the concepts, math, and pitfalls of AI. Ken, welcome to the show. Hi, John. Thank you very much. Thank you for having me. So we are going to talk about the book, but I'm just curious, what does Squint AI do?

0:51That's a great question. So Squint AI is a company that we created to do some research and develop a platform that enables us to do AI in a more responsible way. okay so uh i'm sure we're going to get into this but i touch upon it uh in the book in many cases as well where we talk about uh ai ethical use of ai some of the downfalls of ai and so what we're doing with squint is we're trying to figure out you know how do we try to create a an environment that enables us to use ai in a way that lets us understand when these algorithms are not performing at their best when they're making mistakes and so on.

1:34Yeah. So, so the title of your book is the algorithm plotting against this is a bit of a provocative question. I mean, obviously I'm sure there are people out there that are saying no, and some are saying, well, absolutely. So, so why ask the question then? Well, because I actually feel like that's a question that's being asked by many different people with actually with different meaning, right? So So it's almost the same as the question of is AI posing an existential threat? It's a question that means different things to different people. So I wanted to get into that in the book and try to do two things.

2:12First, offer people the tools to be able to understand that question for themselves. And first, figure out where they stand in that debate. And then second, also provide my opinion along the way. Yeah. And I probably didn't ask that question as elegantly as I'd like to. I actually think it's great that you asked the question, because ultimately what we're trying to do is let people come to their own decisions rather than saying this is true of AI or this is not true of AI, right? That's right. That's right. And again, especially because it's a nuanced problem and it means different things to different people.

2:48So this is a really hard question, but I'm going to ask you, where are we really in the continuum of AI? I mean, people who have been on this topic for many years realize it's been built into many things that we use every day and take for granted. Obviously, ChatGPT brought on a whole nother spectrum of people that now, you know, at least have a talking vocabulary of what it is. But I remember, you know, I've been, I've been, I've had my own business 30 years. I mean, we didn't have the web. We didn't have websites. You know, we didn't have mobile devices that certainly now play a part. But I remember as each of those came along, people were like, oh, we're doomed.

3:24It's over. Right. So currently there's a lot of that type of language surrounding AI. But where do you think we really are in the continuum of the evolution? You know, that's a great question, because I think we're actually very early on. I think that, you know, we've made remarkable progress in a very short period of time. But I think it's still where the very early stages, you know, if you think of AI where we are right now, we were a decade ago. We've made some progress, but I think the fundamentally at a scientific level, we've only started to scratch the surface. I'll give you some examples.

4:00So initially, you know, the first models that were great at really giving us some proof that this new way of posing questions, you know, the neural networks, essentially, right? They're very complex equations. If you use GPUs to run these complex equations, then we can actually solve pretty complex problems. That's something we realized around 2012. And then after around 2017, so between 2012 and 2017, progress was very linear. You know, new models were created, new ideas were proposed, but things scaled and progressed very linearly. But after 2017, with the introduction of the model that's called the transformer, which is the base architecture behind ChatGPT and all these large language models, we had another kind of realization.

4:49That's when we realized that if you take those models and you scale them up and you scale them up in terms of the size of the model and the size of the data set that we used to train them, they get exponentially better. okay and that's when we got to the point where we are today where we realized that just by scaling them again we haven't done anything fundamentally different since 2017 all we've done is increase the size of the model increase the size of the data set and they're getting exponentially better so multiplication rather than addition well yes exactly yeah so so it isn't the progress has been exponential not on a linear trajectory but i think but again the fact that we haven't changed much fundamentally in these models, that's going to taper off very soon.

5:34It's my expectation. And now, where are we on the timeline, which was the original question? I think if you think about what the models are doing today, they're doing very simple statistics, essentially. The idea of these models being called artificial intelligence, I think it's a bit of a misnomer sometimes. I agree. And it leads to some of the questions that people have. because there isn't much like deep intelligence going on. It's just statistical modeling and very simple at that. And then where we are going from here and what I hope the future is, that's when we start. I think the things are going to change dramatically when we start getting models that are able, not just to, not just to do simple statistics, but are able to understand the context of what it is they're trying to achieve and are able to understand, you know, the right answer as well as the wrong answer.

6:32So for example, they're able to know when they're talking about things they know and when they're kind of skirting around this gray area of things they don't really know about. Does that make sense? Yeah, absolutely. I mean, I totally agree with you on artificial intelligence. I've actually been calling it IA. I think it's more of informed automation is kind of how I look at it, at least in my work. Do you see a day where, you know, prompts asking questions or you know that's kind of the the street use if you will of ai for a lot of people do you see a day where it starts asking you questions back like why would you want to know that or what are you trying to achieve uh by asking this question yeah so the simple answer is yes i definitely do and i think that's part of what what achieving a higher level intelligence would be like it's when they're not just doing your bidding it's not just a tool but they kind have their own purpose that they're trying to achieve.

7:25And so that's when you would see things like questions essentially arise from the system, right? It's when they have a goal they want to get at, and then they figure out a plan to get to that goal. That's when you can see emergence of things like questions to you. I don't think we're there yet, but I think it's certainly possible. But that's the sci-fi version too, right? I mean, where people start saying, you know, the movies, it was like, no, no, Ken, you don't get to know that information yet. I'll decide when you can know that. Well, you're right. I mean, so the question, the way you asked the question was more like, is it, is it possible in principle?

8:04I think absolutely yes. Do we want that? I mean, I don't know. I guess that's part of, it depends on what use case we're thinking about. But from a first principles perspective, yeah, it is certainly possible to get a model to do that. So I do think there are scores and scores of people. Their only understanding of AI is I go to this place where it has a box and I type in a question and it spits out an answer. Since you have both layperson and math in the title, could you give us sort of the layperson's version of how it does that? Yeah, absolutely. So, well, at least I'll try. Let me put it that way.

8:41A few moments ago, when I mentioned that these models, essentially what they are, they're very simple statistical models. That itself, that phrase itself, it's a little bit of, it's controversial because at the end of the day, we don't know what kind of intelligence we have, right? So if you think about our intelligence, we don't know whether at some level, we are also a statistical model, right? However, what I mean by AI today in large language models like ChatGPT, being simple statistical models, what I mean by that is that they're performing a very simple task. So if you think of ChatGPT, what they're doing is they are trying essentially to predict the next best word in a sequence.

9:27That's all they're doing. And the way they're doing that is that they calculate what are called probability distributions. So basically for any word in a prompt or in a corpus of text, they calculate the probability that word belongs in that sequence. And then they choose the next word with the highest probability of being correct there. Now that is a very simple model in the following sense. If you think about how we communicate, right? You know, we're having a conversation right now. I think when you ask me a question, I pause and I think about what I'm about to say, right? So I have a model of the world and I have a purpose in that conversation.

10:09I come up with the idea of what I want to respond. And then I use my ability to produce words and to sound them out to communicate that with you, right? It might be possible that I have a system in my brain that works very similar to a large language model in the sense that as soon as I start saying words, the next word that I'm about to say is one that is most likely to be correct, given the words that I just said. It's very possible that's true. However, what's different is that at least I already have a plan of what I'm about to say in some latent space. I have already encoded in some form what I want to get across.

10:51How I say it, the ability to produce those words might be very similar to a large language model. But the difference is that a large language model is trying to figure out what it's going to say, as well as coming up with those words at the same time. Right? Does that make sense? So it's a bit like they're rambling. And sometimes if they talk for too long, they ramble in a nonsense territory because they don't know what they're going to say until they say it. So that's a very fundamental difference. Yeah, I have certainly seen some output that is pretty interesting along those lines. But you know, as I heard you talk about that, I mean, in a lot of ways, that's what we're doing is we're querying a database of what we've been taught, or the words that we know, in addition to the concepts that we've studied, and are able to articulate.

11:41I mean, in some ways, we're querying that, to me prompting or me asking you a question as well. I mean, it works similar, would you say? the aspect of prompting a question and then answering it it's similar but what is different is the the concept that you're trying to describe so again when you ask me a question i think about it and i come up with so again i have a world model that works so far for me to get me through life right and that world model lets me understand different concepts in different ways and when i'm about to answer your question, I think about it, I formulate a response, and then I figure out a way to communicate that with you.

12:27Okay. That step is missing from what these language models are doing, right? They're getting a prompt, but there is no step in which they are formulating a response with some goal, right? Some purpose. They are essentially getting a text, and they're trying to generate a sequence of words that are being figured out as they're being produced, right? There's no alternate plan. So that's a very fundamental difference. I do want to come to like what the future holds, but I want to dwell on a couple things that you dive into in the book. What are the, you know, other than sort of the fear that the media spreads, what are the real, you know, and obvious pitfalls of relying on AI?

13:17I think the biggest issue and one of the, I mean, the, the real motivator for me when I started writing the book is that it is a powerful tool for two reasons. It's very easy to use seemingly, right? You can spend a weekend learning Python. You can write a few lines and you can transform, you can analyze, you can parse data that you couldn't before just by using a library so you don't really have to understand what you're doing and you can get some result that looks useful okay but hidden in that process right the fact that you can take data a lot a large amounts of data modify it in some way and get a response get some result without understanding what's happening in the middle has huge repercussions for misunderstanding the results that you're getting, right?

14:16And then if you're using these tools in the world, right, in a way that can affect other people. For example, you know, let's say you work in a financial institution and you've come up with a model to figure out who you should give some credit, you know, a proof for credit, for a credit line, and who you shouldn't. now right now banks have their own models but if you take the ai out of it traditionally those models are thought through by statisticians and they may get things wrong once in a while but at least they have a big picture of what it means to you know analyze data biasing the data right what are the repercussions of bias in the data how do you get rid of all these things are things that a good statistician should be trained to do but now if you remove the statisticians because anybody can use a model to analyze data and get some prediction, then what happens is you end up denying and approving credit lines for people with repercussions that could be driven by very negative bias in the data.

15:24It could affect a certain section of the population negatively. Maybe there's some people that can't get a credit line anymore just because they live in a particular neighborhood. There's many reasons why this could be a problem. But wasn't that a factor previously? I mean, certainly neighborhoods are considered, you know, as part of the, you know, even in the analog models, I think. Yeah, absolutely. So like I said, we always had a problem with bias, right, in the data. But traditionally, you would hope, so two things would happen. First, you would hope that whoever comes up with a model, just because it's a complex problem, they have to have some statistical training, right?

16:02And an ethical statistician would have to consider how to deal with the bias in the data, right? So that's number one. Number two, the problem that we have right now is that, first of all, you don't need to have that set decision. You can just use a model without understanding what's happening, right? And then what's worse is that with these models, we can't actually understand how the – or it's very difficult traditionally to understand how the model arrived at a prediction. So if you get denied either a credit line or as I talk about in the book, bail, for example, in a court case, it's very difficult to argue, well, why me?

16:40Why was I denied this thing? And then if you go through the process of auditing it, again, with the traditional approach where you have a statistician, you can always ask, so how did you model this? Why was this person denied this particular case in an audit? With a neural network, for example, that becomes a lot more complicated. So I mean, so what you're saying, one of the initial problems is that people are relying on the output, the data. I mean, even, you know, I use it in a very simple way. I run a marketing company and we use it a lot of times to give us copy ideas, give us headline ideas for things.

17:18So I don't really feel like there's any real danger in there other than maybe sounding like everybody else in your copy. But but you're saying that, you know, as people start relying on these to make decisions that are supposed to be informed, a lot of times predictions are wrong. Yes. And there's so the answer is yes. Now, there's two reasons for that. And by the way, let me just go back to say that there are use cases where, of course, you have to think about this as a spectrum, right? Like there are cases where the repercussions of getting something wrong is worse than other cases, right? So as you say, if you're trying to generate some copy and, you know, if it's nonsensical, then you just go ahead and change it.

18:00And at the end of the day, you're probably going to review it anyway. So that is a lower, probably a lower cost, the cost of a mistake that would be lower than in the case of, you know, using a model in a judicial process, for example. Right, right, right. Now, with respect to the fact that these models sometimes make mistakes, the reason for that is that the way these models actually work is that they, and the part that can be deceiving, is that they tend to work really well for areas in the data that they understand really well. So if you think of a data set, right, so they're trained using a data set.

18:39For most of the data in that data set, they're going to be able to model it really well. And so that's why you get models that perform, let's say, 90 % accurate on a particular data set. The problem is that for the 10 % where they're not able to model really well, the mistakes there are remarkable. And in a way that a human would not be able to make those mistakes. Yeah. So what happens in those cases that first of all, when we're training these models that we get, we say, well, you know, we get 10 % error rate in this particular dataset. Now, the one issue is that when you take that into production, you don't know that the incidence rate of those errors are going to be the same in the real world, right?

19:19You may end up being in a situation where you get those data points that lead to errors at a much higher rate than you did in your dataset. Just one problem. The second problem is that if you're in a, if your use case, if your production, you know, application, it's such where a mistake could be costly, like let's say in a medical use case or in self-driving. When you have to go back and explain why you got something wrong, why the model got something wrong, and it is just so bizarrely different from what a human would get wrong. That's one of the fundamental reasons why we don't have these systems being deployed across safety critical domains today.

19:59And by the way, that's one of the fundamental reasons why we created Splint is to tackle specifically those problems is to figure out how can we create a set of models or a system that's able to understand specifically when models are getting things right and when they're getting things wrong at runtime. Because I really think it's one of the fundamental reasons why we haven't advanced as much as we should have at this point. It's because when models work really well, when they're able to model the data well, then they work great. But for the cases where they can't model that section of the data, the mistakes are just unbelievable, right?

20:37It's things that humans would never make those kinds of mistakes. Yeah, yeah. And obviously, you know, that's certainly going to, that has to be solved before anybody's going to trust sending, you know, a manned spacecraft, you know, guided by AI or something, right? I mean, when human life is at risk, you've got to have trust. And so if you can't trust that decision-making, that's certainly going to keep people from employing the technology, I suppose. Right. Or using them, for example, to help in, as I was saying, in medical domains, for example, cancer diagnosis, right? If you want a model to be able to detect certain types of cancer, given, let's say, biopsy scans, you want to be able to trust the model.

21:18Now, anything, any model, essentially, you know, it's going to make mistakes. Nothing is ever perfect. But you want two things to happen. First, you want to be able to minimize the types of mistakes that the model can make. You need to have some indication that the quality of the prediction of the model isn't great. You don't have that. And second, once a mistake happens, you have to be able to defend that the reason the mistake happened is because the quality of the data was such that, you know, even a human couldn't do better. We can't have models make mistakes that a human doctor would look at and say, well, this is clearly incorrect.

21:54Yeah, yeah, absolutely. Well, Kenneth, I want to thank you for taking a moment to stop by the Duct Tape Marketing Podcast. You want to tell people where they can find, connect with you if you'd like, and then obviously where they can pick up a copy of, is the algorithm plotting against us? Absolutely. Thank you very much, first of all, for having me. It was a great conversation. So yeah, you can reach me on LinkedIn and for a copy of the book, you can get it both from Amazon as well as from our publisher website. It's called workingfires.org. Awesome. Again, thanks for stopping by. Great conversation.

22:25Hopefully, maybe we'll run into you one of these days out there on the road. Thank you.