Google's AI Revolution: Major Bard Update Solves Chatbot Math and Reasoning

0:00GoogleBard recently made a major update that is making GoogleBard better at logic and reasoning and therefore better at math and code, something that ChatGPT and other AI models have notoriously been bad at. And I wanted to cover this on the podcast today because it is really interesting the way that they are doing this and their reasoning for it and how they kind of go about this problem. So I think this is interesting because obviously this is a problem that a lot of different AI models have had. And Google traditionally has been quite like as far as the research goes, Google is really ahead of their time.

0:33They came up with the transformer model or some researchers at Google did. And the way that they think at things and look at these problems, you know, they really look at psychology and a lot of different, you know, non computational areas to get the models for what they're doing. And once again, they're doing that with this kind of advanced reasoning thing. So I wanted to cover them in the podcast today, because I think it's really impressive the way they are going about this. So the two big improvements that they are launching is number one, BART is going to be getting better at math tasks, coding questions and string manipulation and it also has a new export action to google sheets so when bard generates a table in its response like if you ask it to create a table for a volunteer sign up for your animal shelter whatever you're planning on using it for you can actually export that right to google sheets i actually think this is a very useful feature because on chat gpt i've asked it for things before it's come up with like a sheet there and you kind of have to try to copy and paste it and this sells somewhere else and the formatting doesn't always work very good.

1:33So I think this is actually a really good use. I think Google is going to have a lot of advantages in a way that they can export what Google Bart is generating into Google Docs and all of the other all the other Google suite of things. And that being said, I think OpenAI is going to have that advantage as soon as they start integrating more heavily with Microsoft Word and the Microsoft Suite because of their partnership there. So the the big thing that they're doing right now is better responses for advanced reasoning and math problems. So a new technique called implicit code execution, that's what Google's calling it.

2:08And this helps bar to detect computational prompts and run code in the background. So as a result, it can respond a lot more accurately to math tasks, coding questions, string manipulation prompts. And it's getting a lot better. Here's a couple examples of some prompts that Bart is now going to be a lot better at doing after this big update. Number one is if you ask it something, you know, like what is a prime factor of 1568, if you ask it to calculate the growth rate of your savings, or if you ask it to reverse the word lollipop. And that last one's kind of funny, but they did have like an example of, you know, asking it to, you know, what is the reverse of the word lollipop?

2:52And it, you know, it did it, which typically Chai Chibut and other models have a hard time doing something like that. But what's interesting is they said, you know, here, here's what it is, here's the Python code I use to do this. And so how they actually did that is by writing Python code to solve that problem. And so it's actually writing code in the back end to do a lot of problems for it, versus, you know, like, it isn't just something it was trained to do in the back end, it knows it needs help with this problem so it can go and actually write a code that helps it solve that problem which i thought was super interesting but digging a little bit deeper into some of its newer capabilities and how it's helping bard to improve its responses um the second big thing is the whole improved logic and reasoning skill side traditionally large language models or llms are prediction engines right there and it's not even that they're predicting the next word they're predicting the next like four letters and then they just kind of string that together and out pops all of these things that it's generating so when it's given a prompt they generate a response um by doing that and as a result they've been really capable on language and creative tasks right so um things that you would i don't know feel like it's more like intuition or creativity creative writing that kind of stuff but they are definitely weaker in areas with reasoning and math that we've seen this a lot with ShadGPT.

4:12And so in order to help solve this really complex problem with advanced reasoning and logic, relying solely on LLM outputs isn't really gonna be enough, Google says. They don't believe that this is actually feasible. And so their new method or their new approach, which I think is absolutely genius, allows BARD to generate and execute code to boost its reasoning and math abilities, like what it did with reversing the word at Lollipop. so instead of just you know using a llm to spit out the words it believes it essentially can detect when something is uh they have a categorized so there's different areas where it needs advanced reasoning logic or math and it will write code um which i think is so interesting because you know in the past i was like oh man well they're just going to integrate it with a calculator and they're going to integrate with plugins kind of like what chat gpt is doing but this approach is really interesting because it's still using generative AI, but essentially it's Google generating code to solve its problems without having to do these third-party app integrations, which I think is really clever, really smart.

5:19Now, this is what I think is really interesting. They said that this approach takes inspiration from a well-studied dichotomy in human intelligence. So they say that it is there's a book by Daniel Common called Thinking Fast and Slow. And in this book, there's the separation of system one and system two thinking. So system one thinking is fast, intuitive and effortless. So for example, when a musician improvises on the spot, or a touch typer thinks about a word, and all of a sudden they watch it just appear on the screen, you're using system one thinking it's more intuitive but system two thinking is a lot more slow a lot more deliberate and a lot more you have to expend a lot more effort to do it so when you're carrying out you know long division or when you're learning how to play a new instrument you're using your system two so in that analogy large language models you can think of them kind of as operating exclusively under system one, right?

6:21These things, they just spit stuff out really fast, it's more intuitive. They're producing text really quick, but there's not a lot of deep thought into it. And so this actually leads to some really amazing capabilities that we've seen out of ChatGP so far, but they definitely fall short and they're not perfect. And it can be surprising in a lot of ways, you know, when you ask it to solve a math problem, that seems fairly simple and it just kind of bombs but at the same time if you tried imagining yourself solving a math problem using your system one uh alone you wouldn't be able to do it right like you can't you wouldn't be able to actually stop and do the arithmetic you just have to spit out the first thing that comes into your mind and that obviously wouldn't be very successful for most people so i think a really kind of like traditional computation really closely aligns with this sort of system two thinking.

7:12There's formulas, it's not very flexible, and you have to follow a very specific sequence of steps that can produce really impressive results, but you have to be very deliberate with it. And so you can, you know, you can solve long division and all sorts of things, but this isn't the intuitive system one that we're used to and that we see out of chat GPT. So with this latest update to Google Bard, they've combined essentially the capabilities of system one and system two the large language models um and traditional code and they're doing this to really improve the accuracy of bard's responses um and and they're doing it by allowing it to um do this kind of implicit code execution so bard essentially identifies prompts that might be from the logical code from like the logical side um it writes this code under the hood and executes it and the results um have been really positive so far they did a bunch of you know internal testing and based off of their own testing the improvement in accuracy was quite impressive they said that in response to kind of adding this computational based word a math problem you know system to thinking to it it improved on their data set it improved the responses the correct responses by around 30 percent so this is a pretty significant change Of course, this isn't perfect.

8:38Of course, Google bar doesn't get everything right. It doesn't. I think for one, they, you know, they noted that it doesn't generate the code correctly every time. So it still has issues with coding. It's not perfect at that. I think it's going to improve, right? If they're making a big push like this, this isn't very far after Google bar launch, and it's already getting 30 % better at some certain problems. I think we're going to see, we're going to continue to see them working and improving a lot of these issues in really impressive ways. So this is going to be a very interesting area to follow and watch to see if people like Anthropic and ChatGPT integrate this.

9:12And that being said, Google is coming right out there and explaining exactly how they're doing this. And so, you know, it's not like they have some secret code or there's some secret, you know, strategies they're trying to hold back. They really are sharing this with everyone, which I think is awesome is because it's, you know, going to push AI forward to be more effective and to be more accurate. So I do give them kudos for that. So this is gonna be very interesting to see how fast some of this technology continues to improve into the future.