Google DeepMind's AI Triumph: Defeating AlphaZero in Chess

0:00The first thing I want to talk about here is the fact that we know that Google Deepmind has taken some significant leaps in advancing artificial intelligence, right? DeepMind is the company that Google initially bought out of the UK, a really big AI kind of company, and really has, you know, increased in Google's chops when it comes to AI. I personally have spoke with some really incredible folks at Google DeepMind when I was at the AI4 conference, talking about some really incredible innovations that they were developing. And there's so many different areas we don't even see, right? Like what's reported in the news or what I talk about here is really scratching the surface to what Google and Google DeepMind are currently developing.

0:36A lot of really novel approaches, really, really impressive use cases that aren't all exactly being integrated into all of the Google suite of products, right? So we see a lot of like Gmail integrations with AI and, you know, Google Workspace integrations with AI. DeepMind is working on things that are a little bit more in-depth and sometimes touch more of the real world. But in any case, today we want to talk about what is happening with their current chess engine that they've just created. So it's the latest innovation is called AZDB. And this, you know, symbol unites several AlphaZero agents into what is called a league.

1:12And the idea here is that they're aiming to enhance the AI's performance in chess and potentially other areas. So this new development comes on the heels of AlphaZero's previous achievements, which set a new bar in AI research by essentially mastering chess through self-play reinforcement learning. However, despite its capabilities, AlphaZero still found certain complex chess puzzles really challenging, which indicated room for further development. And of course, if there's room for development, the DeepMind team is going to take it on. So in crafting this new AZDB, Google DeepMind employed techniques called behavioral diversity and response diversity, Essentially, these methods train the agents to approach chess differently, maximizing the variations in average piece position among agents and exposing them to a broad spectrum of opponents.

2:03This diversification essentially broadens the agent's experience, allowing the system to better adapt to new and unseen positions. The idea for this approach really kind of draws inspiration from human interactions in the world of chess, such as famous collaborations like Kasparov versus the world. Chess grandmasters have long prepared for crucial matches by essentially, you know, sparring with a team of strong players, each with a unique style. but essentially subsequent tests have affirmed that this diversity is not just theoretical so azdb agents actually evolved to have distinct playing styles they begin to favor different styles of chest openings pawn structures and even demonstrate varying rates of peace survivability this is so interesting right like in the grand scheme of things it's kind of funny because on the one hand i'm sure a lot of people are like okay what's the big deal right this is just uh This is just like a computer playing chess.

3:03And on the one hand, right, like I've played computer against a chess since I was a kid on my Windows device. But this is a whole nother level. And it's really impressive here. What's going on with this AI is essentially training different agents of AI to have different styles. And then fighting all of those different agents with all those different styles, combining them or, you know, facing them against different agents really gives these AI models like a very wide breadth of understanding. understanding. Essentially, we're using AI models to increase the ability of AI models, right? This isn't like we're training this because a human played an AI.

3:39It's like essentially we've trained AIs to be really good. Then we've trained them to have a bunch of different styles and do it in a bunch of different ways. And then we're training, you know, to have this kind of grandmaster super chess AI version because it can, it's going against all these different models. Now, the reason I bring this up and the reason that I think this is really important is because I think this same approach is going to apply to a lot of different areas. When it comes to the point where we are tackling, you know, using AI that can become an electrician or a plumber or all sorts of things, it's not just going to be one specific style that is going to win or an AI that's, you know, good at forecasting the stock market or other areas.

4:16It's not just one AI model that's going to win, but it would appear creating, you know, 16 different AI models with different unique strengths and weaknesses and using all of those to train the next variation of, let's say, an auto AI stock trader is an incredibly powerful strategy. And I think this also comes back to OpenAI and ChatGPT because with ChatGPT, it's actually funny. I believe 16 is the same number, but when the model weights were leaked for GPT-4, we found that GPT-4 is built out of like quote unquote 16 experts. So when you're asking it a question, they kind of have these 16 different layers or experts that your question goes through.

4:57It determines which one would be the best at responding. And it goes based off of that. Some of them are better at logic. Some of them are better at reasoning or math. Some of them are better at creative writing or other areas. And so it would appear that kind of creating multiple agents is the next best, is the next way that we improve these AI models. And right now you can see ChatGPT 4 has like 16 and this chess, it looks like they're using 16 different ones. But imagine if for chess or for ChatGPT, imagine if ChatGPT had like a thousand different experts or agents within it. When you ask a question, it has the response from a thousand of these as compute gets cheaper, as energy gets cheaper, as we move further down this line.

5:40And I think especially as, you know, we get these tensor processing units and all this kind of stuff from Google that's coming out, that's making this stuff a lot more efficient TPUs, then I think that we're going to get to a point where we have the bandwidth and the capability to essentially get much, much better results. And the way that we might achieve that is by creating way more AI models and having all of them kind of cross-reference and determine which of the responses is the absolute best. Really cool concept, but yeah, I just wanted to bring that concept to the forefront. In any case, going back to Google and what they're doing with this chess thing, researchers essentially found that when faced with challenging chess puzzles, some specially designed to trip up AI, for example, AZDB substantially outperformed individual alpha zero agents.

6:28And given enough time to process, the Ensemble system solved twice as many intricate puzzles, displaying a level of creativity attributed to the diverse perspectives within its agent league. So one of the standout features of AZDB is the use of sub-additive planning. So in this method, the system strategically selects its best suited agents for each specific chess opening when pitted against AlphaZero. This strategy yielded a noteworthy result, which was a 50 ELO rating improvement over the individual AlphaZero agents. So when artificial intelligence has yet to fully replicate the complexities of human thought.

7:07This new research, I think, brings us one step closer to that goal. The team at Google's DeepMind believes that AZDB's incorporation of human-like diversity and creativity significantly boosts its ability to generalize, narrowing the existing gap between machine and human cognition.