The Logic of Language: Exploring ChatGPT's Reasoning Abilities

0:00What can 160 years of experience teach you about the future? When it comes to protecting what matters, Pacific Life provides life insurance, retirement income, and employee benefits for people and businesses building a more confident tomorrow. Strategies rooted in strength and backed by experience. Ask a financial professional how Pacific Life can help you today. Pacific Life Insurance Company, Omaha, Nebraska, and in New York. Pacific Life and Annuity, Phoenix, Arizona. We all know that ChatGPT is capable of coming up with some pretty complex pieces of content, right? It can write about virtually anything you ask it for, whether it's 100 % accurate or not.

0:37But the real question that a lot of people are asking is, does ChatGPT have a reasoning capability? Is this thing actually able to think? Can it sit there and compute what it's actually saying, or is it just kind of throwing out words? On the podcast today, we're going to dive into that. We're going to dive into how ChatGPT works and answer this important question. So first off, it's important to know that ChatGPT, the GPT stands for Generative Pre-Train Transformer. and then breaking that down even further, we're talking about the transformer. A transformer is something that was invented by a group of researchers at Google back in 2017.

1:12It's kind of funny that this technology was invented by Google. And the transformer is essentially a model that is able to accurately put out really powerful pieces of content that are it's able to write natural language and full sentences. is. So first, I think if we want to understand how this works, we need to kind of define what we mean by reasoning. So in general, reasoning refers to the ability to use logic and critical thinking when we're analyzing information. And then from that, to be able to actually draw conclusions or to, you know, like solve problems. So it involves making connections between different pieces of information.

1:51So essentially, you're identifying patterns and relationships, and then you're using these insights you get to make your decisions that should be informed, right? So reasoning is pretty critical. It's a pretty critical, obviously, aspect of human intelligence, and it's one of the main features that I would say distinguishes humans from animals, for example. So when it comes to ChatGPT, I feel like this question is pretty important, because the question of reasoning is, you know, its capability is a pretty complex topic. so on the one hand the model has obviously demonstrated a very impressive ability to generate text you know we all see that every day when we put questions in and get outputs and so that they appear to be reasoned and logical right like I mean I've asked it to write me an essay or an article and it it would look like it came out with a logically written article going from point a to point b and kind of coming up with arguments why x y and z is possible so for example when asked a question chat gpt can often provide a well-reasoned answer that takes into account relevant information and provides a clear explanation of its thinking right like what i was saying so that's the one side of it um and then on the other side you know there's a lot of researchers that have argued that chat gpt's apparent reasoning ability is largely a result of its statistical processing power rather than true reasoning so this is kind of getting down to the actual question.

3:22And it's kind of interesting because, you know, a while ago in the news, there was this Google researcher testing out Google's chat bot and he was saying, you know, it's sentient because it was, you know, saying all sorts of things to him. And, you know, using something like ChatGPT, we can kind of start to understand because back at the time the technology wasn't actually released to the public, we can sort of start to understand why he might say it's sentient, right? Like you can ask it to say anything or act any way and it can do it. but is it actually sentient? Is it actually reasoning? Those are bigger questions.

3:53So to answer the Monday, ChatGPT is essentially trained on a really vast amount of text data and it uses, you know, responses that are statistically likely to be accurate. So in other words, it's able to generate text that appears to be reasoned and logical because it's been trained on a large amount of text that has already been, you know, like someone wrote this, the text logically, and they were using reason when they did it. So it doesn't necessarily mean it's doing the thinking. It just means that it's able to, it was trained on, you know, data where someone had to do the thinking. And so now when it replicates that content or that data, it looks like it has been doing the thinking.

4:32So I guess to better understand the issue, I would say we could look at a couple examples of ChatGPT's reasoning capabilities. So one area where ChatGPT has been particularly, I would say impressive, is in answering trivia questions, right? This thing was trained on the whole internet, so shouldn't it know answers to all these trivia questions? So if you ask it, what's the capital of France, it's going to tell you Paris, along with some info about the city and history, that kind of stuff, you know. And also, if you ask it, you know, like, who invented the telephone, ChatGPT can provide you with a correct answer about that.

5:04So in these examples, it might seem like ChatGPT is demonstrating, like, reasoning ability, but all it's really doing is taking info about a topic, processing that information, and using it to generate a well-reasoned response, pretty much pulling it out of its database of, you know, it already knows the answer, it's getting the information, and it's able to produce that. So I would say, you know, if we're looking at some areas where ChatGPT falls short, you know, let's say we ask ChatGPT, if a train leaves the station A at 10 a.m. and it's traveling 60 miles an hour and a train leaves station B at 11pm is traveling seven miles an hour, right?

5:42Like those like mind bend and it's like, what time do the two trains meet? It's like a classical math problem that just essentially requires some logical thinking to solve. ChatGPT is likely to struggle pretty hard with this question. And a lot of people have, you know, shared screenshots about like, why is ChatGPT so bad at math, even some basic stuff. And it's because it has not specifically been trained on math solving problems. And so while it might be able to generate a response that contains some relevant information, you know, might talk about the distance or the speed of the two trains, it's pretty unlikely to be able to actually reason through the problem in a way that a human would do.

6:20So I would say all in all, you know, ChatGPT looks pretty impressive. It does not have reasoning capabilities. It is not actually thinking about, you know, how to solve different problems or things you're saying. It's essentially just kind of giving you back information it's already been trained on. And that's not to say that in the future, people aren't going to, you know, integrate complex math solving problem models into this or other models into this that help do that. I think when we're looking at Bing GPT, you know, or Bing's chatbot that's coming out soon, mixing chat GPT. And I think we're going to see maybe a little bit more of a hybrid between, I wouldn't even call it reasoning, but it's able to do a couple more complex things.

7:02It's able to search the internet Live and do some things like that. I think we'll start moving in a direction of these tools actually doing computational like reasoning, thinking about things, but at the moment it does not do it and anyone that says that you know ChatGPT is, I would say is probably misguided at this point.