The God Machine: Demis Hassabis and the Quest for Superintelligence | Sebastian Mallaby

23:55Sebastian Mallaby:And I think Dermis is drawn to that sweetness very powerfully, so much so that he sometimes describes it in quasi-spiritual terms. And Jeffrey Hinton famously resigned from Google in order to speak out against his concerns around AI risk. And that's something we're definitely going to have a chance to talk about. What is most important for us to understand about Hasabis' life, whether that be his early experiences as a chess prodigy, his time at Cambridge and University College London, and the importance he ascribes to certain movies and novels like Ender's Game or books like Hofstetter's Gödel, Escher, Bach?

24:31Sebastian Mallaby:Well, Demis' life is extraordinary from start to finish. I mean, it really is one of the most amazing characters I've ever met in my long career of writing about people. And I mean, clearly the smartest as well. And so I think if we just start with the early life, one of the questions we have to ponder is why did somebody living in London of all places, not even in Silicon Valley, developed this conviction that he was going to devote his life to building super powerful AI. He has this conviction starting around 1993. This is 15 years before AI can even recognize the photograph of a cat. AI could do nothing in 1993.

25:12Sebastian Mallaby:Here is this kid, 17 years old, absolutely convinced that he's going to devote his life to building AI. Where does that come from? where's the conviction come from? And I think you can only understand it if you understand that he was the kid of immigrants. His mother was Chinese Singaporean, living in North London. His dad was of Greek extraction. And he was just outlandishly clever. And he would drop out of school for a year at a time because he was the captain of the junior chess team in Britain. And he was focusing on chess and just sort of studying by himself in his bedroom. And so he was a total outsider.

25:48Sebastian Mallaby:And he was weird to his parents because he was so extraordinarily precocious with his brains. He was weird to his schoolmates when he even went to school because he's this Chinese looking kid who's like insanely clever and disappears and makes a bunch of money by winning chess tournaments around the place. So it came naturally that he developed an ambition, which for anybody in 1997 was crazy. If for somebody in London was completely crazy, right? I want to build powerful AI. He did it by his chess led to also, he was very fascinated by coding. He got hired by a games company building early video games.

26:27Sebastian Mallaby:And he went off to stay in the house of the entrepreneur who had set up this gaming company. And this guy used to talk about AI and kind of slightly gave Demi this bug, this ambition, but it came from a very unusual place. And so I think understanding both how unusual it was his childhood, but also how driven it made you because chess was crazy competitive. They used to have these wooden planks underneath the tables because otherwise the contestants would kick each other in the shins to distract the other guy. It was vicious and people would break down, there would be tears all over the place, people lost games.

27:00Sebastian Mallaby:And it was brutal competition, but also very conducive to developing a kind of autodidact individualism. Yeah. His early experience as a chess prodigy was informative, not just because he came to determination at some point that it wasn't enough and that it was a waste of his brain power. But also there's this great part of the book where he recounts his father telling him that he had to basically give it his all and that would be enough. And what he heard was basically push yourself to near death. And that's the way you know for sure that you've given it your all. How important was that period as a kid, both the competitive aspects of chess, but also what he learned about himself and what mattered to him, what was important?

27:45Sebastian Mallaby:Well, I mean, I thought this story that his father said, hey, what matters when you play a game is just try your best. When I say that to my son, I kind of mean it doesn't matter if you lose. What Demis heard from his dad was, I've got to try so hard, as you said, I'm going to exhaust myself. I'm going to kill myself. I'm going to just everything. I have to give everything. And I heard this story initially from one of Demis' co-founders, Shane Legg. And I said, is that really right? And he said, oh yeah, there's no 50 % mode in Demis. There's no 99 % mode in Demis. It's always more than 100 % all of the time.

28:17Sebastian Mallaby:And I said, what, you mean he has no hobbies? And he goes, well, maybe he might go to a game of the British soccer team, Liverpool, once in a while, but that's it. That's it. Otherwise it's just the mission, build AI. and so yeah i think chess created this incredibly competitive persona maybe also and i kind of play with this idea a little bit there were other reasons too some people if they can make a lot of friends they don't need to prove themselves so much by winning competitions and because he was weird in his school, right? Too clever, too Asian looking, just different, very small physique.

29:00Sebastian Mallaby:So he wasn't going to impress people, overpower them through strength. He needed to assert who he was in a different way. And that was by winning games. And he won not only actually chess, he started to compete in other games and was the five times winner of something called the Mind Sports Olympiad. And he took it incredibly seriously. I'd never heard of it before I met him, But anyway, it was a big part of his life for a while, just beating everybody at all these different games, Backgammon, Shogi, Go, Chess. There's a game called, I guess, Risk. Just lots of games. Diplomacy. So I mentioned Ender's Game and Hofstetter's Gödel, Escher Bach, because, well, for two reasons.

29:42One is that in the book you write about how Hassabis was very drawn to stories. He was a great storyteller. And a lot of his motivations and the way that he tells the story of what drives him are told in some ways indexed to certain stories or certain novels. And also, his objectives in AI are not what you would call traditional entrepreneurial objectives. So philosophy and eschatology and epistemology play a very big role in his motivations. What can you tell us about what motivates him on a deeper level and how this almost feels kind of like a spiritual or ontological pursuit for him? Yeah.

30:21Sebastian Mallaby:Well, I mean, he told me that he would stay up late, always get a bed around 4 AM. and when he was sitting by himself reading, studying scientific papers, thinking about things at 2am, he would feel this sort of reality staring him in the face and saying, I need to be understood. He had this sort of sense that to understand nature as a scientist was to draw closer to some maybe divine force that had created reality. He would say to me, why is it that things are set up in the world in a way that we can study? Isn't it a coincidence? And then he would sort of slam the table with his fist. Isn't it a coincidence that this table is made of atoms jumping around, electrons, and yet it's solid, and you can put your laptop on it, and that laptop is made of bits of silicon, and yet with electricity passing through, it can think.

31:16Sebastian Mallaby:I mean, what's going on here, Sebastian? Why is the world set up like this for us to work, for us to study. I need to understand why. Yeah, I need before I die. I've got to understand this. And so that is a really intense feeling for him that he has to understand. And I think that's why he fundamentally is building AI. And by the way, initially I thought, wow, this is crazy. I've never met somebody so intense before. But then as I thought about it further over the next three years of talking to him, I realized that we all have an element where we look at technology, we love it, we want it, but at the same time, we're frightened about what it's going to do to our society.

31:56Sebastian Mallaby:And yet we move ahead. If we didn't take that trade of embracing a technology that's both exciting and scary, we would all still be living in caves. So in some sense, Demis is just like an enlargement of the human condition, that desire to understand. It's not just, I think, therefore I am. I invent, therefore I am. And I think we all have some of that. It's just that Demis has it in very large scale. Well, there are a number of threads that I extracted from the book that I wrote down as notes as I was reading through it. And one of those threads had to do with the competitive dynamics of the industry.

32:31So you go to great lengths to show how these aren't decisions that are taken in isolation. It's not like Hassabis is sitting in a room and saying, I have a choice to hit a button here. And if I don't hit the button, nothing will happen. We won't get any progress in artificial intelligence. And if I do, we might get a 50-50 existential extinction event. It's that everyone is competing with everyone else. And so the competitive dynamics almost drive inexorably toward innovation and whatever this outcome is going to be, whether it is cataclysmic or beneficial for humanity. Was that a dynamic that you feel like you were aware of early on and that maybe you gained more appreciation for as you worked on the book, or did it really emerge throughout the course of writing it?

33:18Sebastian Mallaby:I understood that there was a race dynamic the moment Chachi Piti came out. And indeed, I remember going to see Demis right after it came out and he said, oh, they've parked the tanks on our front garden, on the front lawn. This is war. There was a sense of betrayal too. Wasn't there a sense of sort of communal betrayal against what was unofficially agreed upon consensus? Well, I mean, I'd say at least it's true that both DeepMind and also Anthropoc had been working on a chatbot a bit like ChatTPT, and they had been reluctant to release it because they weren't sure it wouldn't be toxic. They're worried about hallucination and all that stuff.

33:57Sebastian Mallaby:and ChatGPT came out because OpenAI was less inhibited about those safety considerations. And so there was a sense that this wasn't a fair race. It was more like one side had just been optimistic and unscrupulous in releasing ChatGPT. So there was that. And then the other thing is that Microsoft, which of course was the major backer of OpenAI, didn't make any effort to conceal its glee that Chachi PT had gotten ahead of its rival Google DeepMind. And Satya Nadella famously came out and said, we've now overtaken them and they're going to have to show us if they can dance because can they dance was the challenge, which was, I know from my friends at Google, just super insulting and infuriating.

34:44Sebastian Mallaby:And so this was a take the gloves off, bend, knuckle fight kind of moment. And so that's when the race became totally clear. And that was at the beginning of my writing. So that was not a surprise to me. I think what was a surprise was understanding the backstory of how much Demis had tried to avoid this dynamic, this race dynamic, this race to the bottom. Because he could only justify this quest for superhuman intelligence by telling himself that he would make it safe. It would be fantastic for advancing science, but it would also be safe. And so we had a number of theories about how to make it safe.

35:21Sebastian Mallaby:And one of them was there would be a singleton scenario. In other words, only one lab would develop AI on behalf of all humanity and everybody would club together and make sure it was safe before it was released. And the moment when OpenAI was founded in 2015, which was five years after DeepMind, it was evident that Singleton scenario wasn't going to happen. And so that was just one out of several moments of disillusionment where a theory about how to make AI safe got blown up. And so part of the story here is an intelligent, good person trying to make AI safe, but theory after theory about how you would do that doesn't work.

36:05So let's use this opportunity to dig in a bit more into the foundations of the company. as you suggest, DeepMind was founded in 2010. Help me trace how Hassabis took this kernel of an idea that you mentioned in 1993 to build an AI company and turned it into a reality through the eventual founding of DeepMind along with his co-founder Shane Legge and Mustafa Suleiman in 2010.

36:32Sebastian Mallaby:Well, right. So he had this idea he wanted to build AI before he got to college. Then he read computer science for three years at Cambridge. In the meantime, he was still consulting on video game design with the guys he'd worked with before college. In fact, he persuaded the boss of the company. It was called Bullfrog Games. And the Bullfrog boss gave him a really fast car. It was a Porsche 911. And he would drive at like crazy speeds from his campus to the gaming company in the middle of the night when there was no traffic and put in a couple of days of work with the gaming company and then come back and do classes again.

37:11Sebastian Mallaby:So he was always juggling both computer science and kind of game design. When he got out of college, he worked a bit again with the same bullfrog studio. And then he started his own game design company called Elixir Studios, which did okay. He made a bit of money, but he kind of eventually had to close it because it wasn't doing well enough. And then he was kind of burnt out at that point from startups failing can be a pretty traumatic experience. And so both he and his co-founder, David Silva, who is a big figure in my story because he was the scientist behind AlphaGo and various other systems.

37:47Sebastian Mallaby:They both wanted to recuperate from entrepreneurship. So to relax, they did PhDs. And David Silva went off and did a PhD in reinforcement learning in Canada. And Emmett took a more eccentric route. He did neuroscience. And the theory was, to build machine intelligence, I need to understand how human intelligence works. If I understand the kind of main existence proof that you can even have intelligence is the human mind. So if I do neuroscience PhD, then I will understand the endpoint of where AI is trying to get to. So he did this neuroscience PhD, was crazy good at it. He published a paper when he hadn't even finished his PhD, which became one of the most cited neuroscience papers of that period.

38:31Sebastian Mallaby:And then after that is when he founded DeepMind in 2010. So it's actually important to note, now that we're talking about neuroscience, that this was still in the period where symbolic approaches to AI were the norm, correct? That's right. Yeah. Much programming had been kind of what we think of as logical deductive programming. If A is bigger than B and B is bigger than C, then A is bigger than C also, these kind of hard and fast logical rules. And the big jump in order to build AI was to go from deduction to induction. So with induction, there's no logic. There's just tons of examples. And you absorb all those examples and you kind of intuit the patterns from the examples.

39:20Sebastian Mallaby:And the thing about induction is that if I was to study some examples of humanity, I would, let's say I went and looked at 10 different New Yorkers, and I saw what they did in the morning, I would conclude that all humans drink coffee in the morning. But if I studied a million human beings, and they were dispersed geographically, I would understand, no, they don't all drink coffee in the morning. Any inductive conclusion is subject to revision when you have more examples. And so you need lots of examples, maybe even infinity of examples, in order to do good induction. And that's why I call the book The Infinity Machine, because this shift from deduction to induction only worked when you had a machine that could make sense of an infinity of data.

40:03Do you think it was also an implicit concession that we couldn't build artificial general intelligence, and certainly not intelligence that surpasses human intelligence, by simply manipulating formal representations, and that the move to inductive reasoning was an implicit, if not explicit, acknowledgement of the limitations of formal systems of knowledge. Yeah.

40:26Sebastian Mallaby:And just that as humans, and this is where understanding how the human mind works was very important to understanding how AI was going to work. As humans, we don't think in terms of logic all the time. Most of what we do, some face appears at the door and you recognize who the person is. There's no logic there. You just look and you recognize. And so when I talk, I'm not always grammatical. So even if you taught a machine the rules of human grammar, actually we as humans don't observe those rules very often. So just the whole idea that knowledge and communication could be decomposed into logical constructs is wrong.

41:06Sebastian Mallaby:A lot of what we do and how we think is not actually logical. It's inductive, not deductive. And you mentioned Gödel-Escherbach, the book that Demis read when he was a teenager. I mean, the most important thing about that book to Demis was really that Gödel, who is the mathematician in the title, Kurt Gödel, was most famous for what's called the incompleteness theorem, which shows that it's impossible to create a system of mathematical logic that completely explains everything that might be true in mathematics. So if it's not even possible for math. It's certainly not true for much messier emergent systems like how our biology works.

41:45Sebastian Mallaby:Or look at the irregularities you find in nature, bushes of all kinds of shapes and irregular floor. I mean, the ground is not even and it's just everything is super irregular and it doesn't follow clear rules. If I say, why is a butter knife in the same category as a carving knife? One is small and blunt, the other is long and sharp. You would say, well, they're both knives, meaning they both have the same function. But then if I say, why is a Dachshund in the same category as a Labrador? They don't have the same function. They're both dogs. But what does that mean? Well, they're both the same species because they can mate together and have fertile offspring.

42:22Sebastian Mallaby:But how do you know, how do you teach an AI that some categories of objects we classify according to their reproductive potential, others by their function, others by their color. I mean, there is no rule that describes that. And so just the point being that logic is super limited. And so AI was a way of developing software systems that could think inductively, not deductively. Do you think that the inherent contradictions within such systems stem from the limitations of language that show up, for example, in image recognition? Like how do you define that something is a cat, for example? You can't just say that it's a small furry animal with whiskers because it could be raccoon or it could be a baby lion.

43:09How do you formalize that into a set of rules that computers can understand from first principles? And is that an indication of the limitations of language?

43:17Sebastian Mallaby:You can't formalize it into a language that a computer will understand from first principles, which is why both facial recognition and language recognition have required AI systems that don't think in terms of first order logic. They are way more kind of subtle and complex than that. And the way that they can recognize a cat is they use these neural networks, which have thousands of decision centers that are picking up on very kind of subtle relationships between, okay, these two bits of black in the picture are a continuous line that delineates an eye and then there's pupil in the eye. I mean, it's just a much more subtle process, which is learned through trial and error.

44:00Sebastian Mallaby:So you show the system a labeled picture of a cat, and then you mix up the cat's pictures with non-cats. It could be tigers or fairy blankets or something that you might kind of mix up with a cat. And the system tries to guess what's a cat. And if it gets the answer wrong, then it will tweak its own internal code, its parameters, so that the next time it answers slightly better. And through trial and error, it lands on a set of parameters, basically on a code that unlocks the trick of recognizing a cat. And the human programmer doesn't know how that worked. It's the machine that has discovered through trial and error how it works.

44:44Sebastian Mallaby:And so this is an illustration of the limits of logic. So how does this relate to Hassabis' early skepticism around language models and their limitations and why he felt that was not the path to general artificial intelligence? Yeah, this is very interesting and important. I mean, there was this moment when the Transformer paper came out, which was his scientific breakthrough in how you find patterns in long sequences of data. And the one clear example of this is long text, sentences and paragraphs that go on and on. And that was a kind of unsolved problem in computer science. How do you have a good translation machine or a good language recognition machine?

45:28Sebastian Mallaby:In 2017, this paper comes out from Google Research, and it's a huge breakthrough. And immediately, on the very day that paper comes out, the chief scientist, OpenAI, Ilya Satskeva, tells me, he jumps out of his chair, runs down the corridor, finds his collaborator, Alec Radford, and says, drop what you're doing. We're going to do this. We're going to apply this transformer thing. We're going to apply it to language. Meanwhile, over at the rival lab, DeepMind, Demis is like, nothing to see here. I'm not interested in pursuing language. And he had a few people who were vaguely working on language and DeepMind.

46:03Sebastian Mallaby:And Demis says to me, you're being unfair. We did have a language team, but frankly, he didn't focus on this language team. The language team they did have was not really doing the right kind of research to make use of the transformer. And so it was sort of just like a missing piece of DeepMind's agenda at the time. They were more interested in these trial and error systems that were playing games. They'd done AlphaGo, they'd beaten the human Go champion. So they were interested in agentic systems that played games, not in language. And the reason was that partly that Demis felt that language was not enough to get to intelligence.

46:41Sebastian Mallaby:Language is a system of symbols and mapping symbols onto some other symbols, for example, pictures. So you have the word cat and you have a picture of a cat, you map one single one to the other. How is that connected to reality is what Demis was thinking. you need to be grounded in the real world to be properly intelligent. And that was his thought. And just language ain't going to get you there. And it took about three years between 2017 and 2020 for Demis to change his mind on that. And the tipping point came in the summer of 2020, when OpenAI released GPT-3, the third iteration of its language model.

47:20Sebastian Mallaby:And that was just way too good to ignore really. It did seem that a language trained system could conduct highly intelligent coaching conversation. And so that was the beginning of DeepMind waking up and racing to catch up. So I'd like to at least cover the Google acquisition before we move it to the second hour, Sebastian. That acquisition by Google happened in 2014 and it gave DeepMind essentially unlimited resources, but it embedded it inside of a global corporation with its own priorities, its own culture, and an incentive structure that could tend to view the kind of technological disruption that DeepMind was pursuing as inherently threatening to the company's core business.

47:59What informed Hasabis' decision and the decision of the team? And what do we know about how aligned the co-founders were, as well as some of the early investors in proceeding with the acquisition?

48:11Sebastian Mallaby:Well, one of the things I discovered is that Demis was such a dominant founder that the other two couldn't really challenge what he wanted to do. So it was all on Demis. I mean, he had at the founding 90 % of the shares and the rest of the shares were with the other two guys. And so he was dominant. And for him, the decision was really dealing with raising money from venture capitalists was taking up a huge amount of his time and taking him away from the scientific research he wanted to focus on. And at the same time, the amount of money he needed was starting to go up fast because it was just the beginning of deep neural networks getting deeper.

48:47Sebastian Mallaby:And so you needed more chips, more hardware, and you needed to compete to hire the best scientists because other labs were now trying to hire them as well. And so he needed money. And the only question was, would he sell himself to Google or would he sell himself to Meta or some other company? And he actually went to see Mark Zuckerberg, I guess it was Facebook then, not Meta, and talked to Zuckerberg. And he had this amusing test that he administered on him. He said, I'm working on AI and I think it's super important. And Mark Zuckerberg said, yes, very important. And so then Demis carried on having dinner.

49:23Sebastian Mallaby:And towards the end of the dinner, he mentions how great augmented reality is a technology. And Mark Zuckerberg says, yeah, it's fascinating. And we're really going to invest big in that one. And then Demis a bit later mentioned something else, whether it is artificial reality or some other kind of technology. And again, Zuckerberg is like, yeah, that's great. We're going to do that. And Demis thought, okay, I understand this guy. He says everything is great. He doesn't really mean it about AI. He doesn't get that AI is just way more important than augmented reality. And if he doesn't get that, I'm not selling to him.

49:55Sebastian Mallaby:So then he pivots back to Google and he does the deal with Google. He has to fend off Elon Musk, who is very keen to buy the company, hates Larry Page of Google at that moment and wants to compete with Larry Page by putting DeepMind inside SpaceX or Tesla. And Demis says, no, he doesn't want to do that. So he goes with Google. And look, it worked out incredibly well because Google was a good parent company. It did give Demis tons of resources to do compute and to hire great scientists. And so he made the right call. I love this story because it highlights how perceptive he is, how unusually perceptive he has his people.

50:35And yet in the book, you also show how he could also be incredibly naive. What explains that, this dual character in Hassabis?

50:43Sebastian Mallaby:I think he was naive about how Silicon Valley worked because he wasn't based there. He was based in London. He's quite patriotically British. He believes that the social democratic values of Britain mean a lot to him. If you go to Silicon Valley, it's quite hard to find any public space. if you go to the DeepMind office in Kings Cross in London, there are barefooted kids playing soccer in the summer who come from the local government housing, and they're playing on the government soccer pitch. And so it's just a different vibe, and Demis cares about that. So I think he was naive in terms of how to play the venture capital raising game.

51:18Sebastian Mallaby:He believed too much in Peter Thiel, who was the Series A financier, and he thought that Peter Thiel would give him more capital. And in fact, he was wrong about that. Peter was quite reluctant. He's such a contrarian that he invested at the beginning when nothing was working in AI. And then when it starts to work, Peter Thiel's like, I think it's probably overheating now, it's time to sell. He's always against the consensus, which I find hilarious. But anyway - That's Ian Streak. That's Rene Girard in him. He's certainly an interesting thinker in any event. I think clearly the venture capital game is something that Demis was naive at.

51:54And also I'd say the inevitability of competition amongst

51:58Sebastian Mallaby:multiple different AI labs is something he underestimated. I mean, the notion that would be a singleton lab, like one lab building AI on behalf of all humanity, obviously underestimates the truth that if you've got something as exciting as AI, multiple labs with multiple ambitious entrepreneurs in multiple countries are going to come after this goal. It's just too rich, too sweet not to chase. In retrospect, do you feel like Demis thinks it was the right decision to sell to Google, or do you think there's some regret there? Well, now and again, he would say to me, oh, if only I'd been in that position trying to raise money and it had been not 2013, I could have just raised my money independently.

52:41Sebastian Mallaby:I could have remained independent. But I think that's the minority share of his mind, right? The majority of the time he says, it worked out great. I got all the money I needed. I had a choice in my career. did I want to build an independent company and build it into a Google? Or did I want to use the resources of Google to build towards artificial general intelligence and do great science? And he has no doubt that he cares much more about the science than about company building. He's quite good at company building, but his primary objective is science. And so when he got the Nobel Prize, that was way bigger for him than it would be to be a 10 times billionaire or something.

53:22So Sebastian, I'm going to move us to the second hour. I think where I'd like to pick up there is ChatGT35, the launch in late November of 2022. And this also coincides with when you first went to him to pitch him on this project. So I think that's an important date to look at. That'll also offer us an opportunity to talk about Sam Altman because he has emerged to be one of the most seemingly despicable, dishonest, and untrustworthy characters in Silicon Valley. And I'd love to sort of explore some of the commentary in the book around him. And this also raises the larger question, which we can come back to, which is how important are the individuals at the center of this movement?

53:59Does it really matter at the end of the day what their characters are like? You also have this great conversation in the book with Jeffrey Hinton in his kitchen, where he gives you his assessment on whether

54:10Demetri Kofinas:we're going to make it or not. And this conversation about AI risk is something that, You mentioned Nick Bostrom early on in this episode. Nick had been on the podcast before and his book, Super Intelligence, was for me very formative. It was a book I read early on that exposed me to many of the philosophical ideas and thought experiments around AI risk. And it feels like this conversation has receded into the background and not for good reasons, but rather because of the competitive pressures that you talk about in the book, both commercial and geopolitical that are now driving this industry forward and that I want to talk about in more detail in the second hour.

54:48And I'd also like to explore some of the alternative viewpoints like Jan LeCun's perspective, his dismissiveness of many of the AI risk concerns, as well as

54:56Demetri Kofinas:Jeffrey Irving and his technical alignment approach. But we're going to do all of that in the second hour, Sebastian. For anyone new to the program, Hidden Forces is listener supported. We don't accept advertisers or commercial sponsors. The entire show is funded from top to bottom by listeners like you. If you want access to the second hour of today's conversation with Sebastian, head over to hiddenforces.io slash subscribe and sign up to one of our three content tiers. All subscribers gain access to our premium feed, which you can use to listen to the rest of today's conversation on your mobile device using your favorite podcast app, just like you're listening to this episode right now.

55:35Demetri Kofinas:Sebastian, stick around. We're going to move the second hour of our conversation onto the premium feed. If you want to listen in on the rest of today's conversation, head over to hiddenforces.io slash subscribe and join our premium feed. If you want to join in on the conversation and become a member of the Hidden Forces genius community, you can also do that through our subscriber page. Today's episode was produced by me and edited by Stylianos Nicolaou. For more episodes, you can check out our website at hiddenforces.io. You can follow me on Twitter at Kofinas, and you can email me at info at hiddenforces.io.

56:16Demetri Kofinas:As always, thanks for listening. We'll see you next time.