Unveiling the Boundaries: Understanding Chat GPT's Limitations

0:00On the podcast today we're going to talk about some of the limitations of ChatGPT. and this is going to be kind of an interesting thing because a lot of people have been asking like people may have noticed if you try to put a massive document into chat chippity and ask it a question it is unable to do that if the content is too big and it also as you know will only give you responses that are kind of bite-sized it has kind of a max length of response and sometimes it cuts itself off before it even finishes a sentence and you got to say continue to get it to continue generating the full response that you're working on.

0:35So on the podcast today, we're going to talk about these limitations, what causes them, and I'll go over really briefly how this works from a technical perspective. So it's very interesting insights are going to get into ChatGPT today. So let's dive in. First is at a high level, the way ChatGPT is working, essentially, it's got an input from the user, that's you giving it a text question, a prompt, it does what it's called encoding. So the input text that you give it is converted into numerical representations that the model, ChatGPT, can understand. So it's not really reading the words that you're saying and understanding them.

1:12It has meaning assigned to every word and every, based off of all the content it was trained on, it has numerical values that it assigns to different words and phrases and sentences so it can really understand their meaning. That's part of like the real power of chat gpt so that is in the encoding process this is done using a process called tokenization which essentially just splits the text into individual words or sub words and maps them to unique numerical ids after that's done we go to processing so essentially the encoded input is fed into the transformer or the neural network which processes it to generate a probable distribution over the possible next words essentially based off of this you know what it would it guess the next word should be what would it guess the next response should be so this involves many different layers of computation each of which applies a different transformation to the input so after that we get the output the model outputs a ranked list of the most likely next words based on the probabilities generated in the previous step so all of that context is to get to our main point which is why does chat gpt have a maximum capacity when it comes to putting inputs in and receiving them out why is there length limits and kind of what is that all about so as impressive as chat gpt is it does have its limits one of the most significant limitations is what's known as max capacity so this refers to the maximum amount of text the model can process in a single interaction so once the input text exceeds this limit the model starts to break down and it produces significantly less accurate or even nonsensical outputs and that's just to do with you know how big a big question or an ask or how much content you're giving it and the reason for this limitation has to do with the way the neural network that ChaiGTBT is running on is designed so neural networks have a fixed amount of memory which is used to store the parameters or weights that determine the neural network's process information so these parameters are learned during the training process and are used to make predictions on new data.

3:22So the amount of memory available in a neural network is limited by the hardware it's running on. So for example, if ChatGPT is running on a, you know, server with 16 gigs of RAM, it can only store 16 gigs worth of parameters at any given time. So if the input text is too long, it's going to require more memory than the model actually has available, which can cause, you know, errors or crashes, which we see at, you know, as chat gpt is kind of new and they you know they can't give this thing unlimited ram for every single person using this because obviously as a free tool this thing's incredibly expensive it's costing them millions of dollars just to run the computational power on this thing and so they do have limitations which we see sometimes when our chats crash or when it has limitations on how much you can put into it another factor that contributes to chat gpt's kind of reaching its limits is the complexity of the input so the model is designed to process natural language and which can be highly variable and nuanced so if the input text contains many unusual or complex words phrases sentence structures and it's super long the model is going to struggle to make an accurate prediction because it's just not used to so much and like we talked about earlier it has limitations in its compute power so what happens when chat gpt reaches its maximum capacity or it's you know hits its max on on a prompt we've seen it before sometimes it will crash sometimes it will say error try again um sometimes it you know you got to rephrase stuff uh when it reaches that the model is going to also start to produce less relevant um and less accurate responses so that's just something to be aware of so to prevent this from happening developers need to be aware of chat gpt's maximum or it's pretty much as limits and design their applications accordingly so this might involve limiting the length of the input text the pre-processing the text to remove um so they could use pre-processing essentially like a developer if they're integrating this into an application they're making they could create pre-processing um so that when their users are inputting text it's pre-processed and the text removes complex or irrelevant information right so like essentially developers using this for their own tools can make additional layers before it hits the chat gpt api um to make it more likely to give a good response.

5:45So I believe that as artificial intelligence continues to evolve and improve, it's likely that we're going to see even more advanced language models in the future. However, at the moment, the models that we have do have their own set of limitations and challenges. So it's just important to understand that. And it's going to be pretty cool to see how this evolves in the future.