The Infinity Machine: The Untold Story of Demis Hassabis and DeepMind

0:00These machines are going to be smarter than us. They're going to have a survival instinct because that will kind of be coded into them through the way they are trained. And they are duplicitous. They don't tell the truth to human users all the time.

0:14Peter High: Welcome to Technovation. I'm Peter High, president of Metastrategy. My guest today is Sebastian Mallaby. Sebastian is one of the world's foremost business historians and financial journalists. He is the Paul A. Volcker Senior Fellow for International Economics at the Council on Foreign Relations, a former Financial Times contributing editor, and a longtime columnist for The Economist and The Washington Post, and a two-time Pulitzer Prize finalist. Sebastian is the author of six acclaimed books, including The World's Banker, More Money Than God, The Man Who Knew, and The Power Law, which examined the history of venture capital and the firms that shape Silicon Valley.

0:47Peter High: His latest book, The Infinity Machine, I've got it here, it's a phenomenal read. It chronicles the remarkable story of Demas Asabas, DeepMind, and the global race to build artificial general intelligence. It draws upon more than 30 hours of interviews with Asabas himself and conversations with more than 100 colleagues, competitors, friends. And the book explores not only the evolution of DeepMind, but also the scientific, commercial, and ethical questions surrounding perhaps the most consequential technology of our time. In today's conversation, we'll discuss Sebastian's deep reporting on Demesis Abbas and DeepMind, what distinguishes the pursuit of AGI from earlier waves of AI, the leadership and culture that enable DeepMind's breakthroughs, and what the race towards superintelligence may mean for business science and society, more generally speaking, among other topics I'm sure we will tackle together.

1:34Peter High: Sebastian, welcome to Technovation. It's so great to speak with you today.

1:37Sebastian Mallaby: Yeah, same with you. Great to be with you.

1:41Peter High: Today's episode is brought to you by Eleven Labs. Most enterprises trying to deploy AI agents hit the same wall. The agent itself is fine. The problem is that it sounds wrong. Robotic, flat, off-brand, and the moment a customer notices, the trust is gone. Eleven Labs develops the AI voice and audio models that give agents a human-level voice, the tone, the emotion, the rhythm of real speech, and wraps them in the full stack you need to actually deploy. Knowledge integrations, workflow tooling, multilingual support, and enterprise-grade reliability. Build human-level conversational experiences into your products, services, and operations.

2:18Peter High: Head to 11labs.io backslash technovation to get started for free. And now on to the interview. Wonderful. Well, let's begin with the genesis of this project, if you don't mind, Sebastian. In some ways, very logical to go from venture capital, the subject of your last book, which, by the way, fun fact, is the yellow book, right? For those who can see this, that's power law. forgive me that it's not more prominently shown, but which I thoroughly enjoyed, become a classic tome on the history and the present of venture capital, to artificial intelligence, where it seems like such a preponderance of venture capital is now flowing.

2:58Peter High: But how did you choose this topic, Demis more specifically, DeepMind? Talk a bit about the early stages of this project, if you would.

3:06Sebastian Mallaby: Sure. So, I mean, at a high level, the books I've done have pretty much all been an attempt to grapple with something which is going to drive how the world works, a kind of political economy writ large. And they need to have two things. They need to have a meaty subject, but also a fascinating character. And sometimes it's kind of a cast of characters, not just one character. So with the power law, there are multiple different venture capitalists, depending on the time period I'm describing, who I pin the story on. but you do need I think character to drive stories partly just because character is fun but also because you know I majored in history many moons ago at Oxford University and you know that kind of debate about do individuals shape history or are there deeper ineluctable forces technological change demographic change what have you which really drive outcomes and my view is the answer is both so you want to get that interplay between the individual and the deeper forces or as my editor says, you paint the figure, but you also want the landscape behind the figure.

4:09Sebastian Mallaby: So that's kind of the big frame. And so I was always looking for something which had both of that. And I'd been to tech conferences while I was doing the book on Silicon Valley and venture capital. And when these conferences were in Europe, there would often be this friendly character who showed up with a kind of guy next door grin and a very unassuming, very boyish young look. and this character was called Demis Hassabis. I probably mispronounced his name at first, Demis Hassabis. He's a common error, but after a while, I figured out how to put emphasis on the right syllable. And so I got to know him a little bit.

4:46Sebastian Mallaby: I had a bit of a hook with him because I had written a book about hedge funds, More Money Than God, and one of the characters in there is Peter Brown, who was the CEO of Renaissance Technologies, the kind of storied quantitative hedge fund in Long Island. And Peter had been the first PhD graduate student under Jeff Hinton, the father of deep learning. And so I knew a little bit about Peter's early career and the kind of the origins of neural networks and deep learning. And I knew that Demis would be curious about people who kind of plowed some of the same ground as him, but came much earlier because Demis was trying to complete in a way what the early neural networks had promised.

5:33Sebastian Mallaby: So we knew each other a tiny bit. I had admired him greatly from afar. He probably just about noticed who I was. And then I went and pitched him. And my notion was, look, circa 2022, artificial intelligence had got to a point where you had the breakthrough with protein folding, where all of the shapes in nature had been predicted by DeepMind, this system called AlphaFold, and that had built upon things like AlphaGo, the Go system, the computer system that played Go better than any human. And these kinds of breakthrough had something in common, which was the ability to take just enormous amounts of data, enormous amounts of possible permutations, a vast search space, and find the pattern that allowed you to predict.

6:19Sebastian Mallaby: In other words, predict is kind of like a synonym for just make sense of this enormous trove of data. So in Go, it's much more complex than chess because you have this 19 by 19 board. So there's 361 options with the first move, 360 with the second. You multiply that out, you pretty soon get to such a big number that it's almost infinity. And then with the permutations which are possible in protein folding, it's even bigger. In fact, instead of being 10 to the 170, it's 10 to the 300 of the permutations that you could imagine. So a machine that could predict successfully protein shapes or go moves was essentially an infinity machine it could make sense of what was almost infinity um and i uh i figured that was that was the grounds for some point i didn't know when there'd be a breakout in ai it would go mainstream and so i had this fascinating character demis sabis i had what seemed like a good topic although it was a bit niche and i went to pitch demis in november 2022 and here's where you know the moral of the story is it's better be lucky than smart because I pitched him he agreed and then a couple of weeks later chat GPT went viral and so I found myself embedded in this unbelievable capitalist race to roll out large language models you know right when that race began.

7:42Peter High: You strapped yourself to a roller coaster you didn't realize was was right right there beside you.

7:47Sebastian Mallaby: Yeah, yeah.

7:49Peter High: Fascinating. You describe Demis in some ways as a classic product of London. Talk a bit about his upbringing, if you would.

7:56Sebastian Mallaby: Sure. So his father is of Greek Cypriot origins, and his mother was an immigrant Chinese Singaporean who had suffered proper poverty as an orphan in Singapore for part of her childhood. And so you had this kind of melting pot kid, Demis Asabis, with this mixture of Greek and Chinese lineage, which in a way makes him a typical Londoner because fully 40 % of the workforce in London is foreign born. It's a very, very much of a melting pot. I enjoy going to Silicon Valley and talking to my friends there and saying, you know, you think you've got the monopoly on, you know, immigrant founders of tech companies.

8:37Sebastian Mallaby: Well, let me tell you about Demis Asabis because he fits your stereotype, you know, to a T. So yeah, he had that interesting sort of melting pot background.

8:48Peter High: He also was a chess prodigy early. You recount that at four years old, he figured out how chess worked. He became the second or third best chess player for his age group in the world shortly thereafter. Talk a bit about that period and the role that chess, you've talked about Go, which obviously both board games are actually a lot more complex than chess is the case is, but the role that those early intellectual games that he mastered, the role that that would play perhaps in building the confidence in him to eventually become an entrepreneur from your perspective.

9:21Sebastian Mallaby: Sure. So the first thing was, yes, you know, chess was the route out of this modest background to something which was approaching stardom, you know, when he was a kid. So he was, by the time he was 10 years old or so, he was the captain of the UK youth chess team. He was beating adult players in international tournaments. And he was definitely a prodigy. And he was earning money from the tournaments he played. So he was quite businesslike about this. He would show up in a town, speed through the games, collect the prize money and leave. And he would do this all over England. And that gave him the cash to buy himself early personal computers.

10:03Sebastian Mallaby: This is the, you know, the 1980s because he was born in 76. And he began to code. And one day he bought himself a book about coding of chess games, which obviously married his interest in coding with his interest in chess. And that was the first springboard for his interest in writing programs that would be actually quite complex and try to mimic human intelligence. And from there, he went on and got a job. He was so smart, he got into Cambridge University to read computer science when he was only 16. And he was advised by Cambridge, look, you're academically more than ready. But socially, you're going to have a better time if you show up when you're closer to 18.

10:46Sebastian Mallaby: So why don't you do something else for a bit? So he got himself a job coding and designing video games for a studio just outside London. And he created this game called Theme Park, which was such a hit, sold millions of copies. that his boss, the kind of the owner of the studio, was desperate for him not to go to college. He would rather Demis stayed with him and go to the next game and make some more, you know, multi-million pound revenue games. And so he wrote Demis this check for 500 ,000 pounds, which if you do it today's dollars, it's like 1.5 million. And Demis, age 18, with parents of modest means, just never cash the check, turn the money down.

11:30Sebastian Mallaby: So forget it, I'm going to go study computer science. So whereas kids in California drop out of Stanford quite happily in favor of Silicon Valley riches, this guy Demis did the opposite and insisted on doing his degree in computer science.

11:45Peter High: It's such a fascinating anecdote that in many ways also kind of establishes his relationship with money, that it's not just about a personal financial game, but a bigger game that he's playing. And speaking of games, before you met him, he suggested you read Ender's Game. It was a suggestion he had. And I wonder, talk a bit about why that was, what you think he was hoping you would glean from that, that would be instructive to your early meetings and what you indeed did glean from it that was of relevance. Yeah.

12:19Sebastian Mallaby: So, I mean, this blew my mind. So I was going to have my first dinner with him. It was after the first pitch meeting, I'd had to go through various interviews with kind of, I think, four or five of his colleagues who were checking me out before he agreed to give me the access I wanted. And then in the second meeting with him, he said, before you come, you have to read this book, Ender's Game. And I don't really read science fiction much, but, you know, I read this book in preparation. And it's the story of this boy who is recruited by kind of the adult intergalactic leaders who want to defend planet earth against invading space aliens and they train this kid who seems to show the right talent to to defend planet earth and basically save all of humanity so i'm reading this story i think wait wait wait wait it is demis telling me that he is a messianic figure who's going to save all of humanity.

13:15Because first of all, that's a bit much.

13:18Sebastian Mallaby: Secondly, even if he believed that, would he tell the writer who's about to write a book about him that that's his self-image? No, that wouldn't be a smart thing to tell the writer, right? Because you're just inviting ridicule. So I show up at this dinner and he says, well, I hope you read the book because I totally identify with Ender. If you want to understand me, I'm like Ender. I'm doing the same thing. I'm devoting all my life to this mission of building superintelligence. And, you know, and it's going to save us because we need a solution to climate change. We need, you know, breakthroughs in medicine.

13:51Sebastian Mallaby: And I'm going to deliver this. So I'm like Ender. And I gave my, I feel so strongly about this, Odemis told me, that I gave my wife the book to read so that she would understand me better. For some reason, she sympathized with this boy called Ender, who was recruited, you know, to go save humanity. She doesn't sympathize with me so much. I guess I'm an adult. Maybe I'm doing this through choice. And I'm like, yeah, you are doing it through choice. But anyway, and yes, but then I thought about it and I thought, you know, you could read this as kind of insane hubris. Or you could think, well, there's a kind of authenticity about this unfiltered person who just tells me right out how he thinks about himself and isn't ashamed to do that.

14:32Sebastian Mallaby: And in a way, he is the central figure in the creation of modern AI, right? Because he founded DeepMind way before Sam Altman thought of doing open AI or Dario Amadei thought of doing Anthropic. He really is the leader, the original creator of a lab devoted to artificial general intelligence. And so perhaps he is entitled to think of himself in those heroic terms.

15:01Peter High: fascinating yeah really interesting and I love your analysis of that I you alluded a moment ago to the extent to which he turned down the money before going to college in some ways separating him from a lot of the classic entrepreneurs in Silicon Valley but indeed entrepreneurship was his calling and you point out that especially then perhaps even to some degree now of course But then entrepreneurship was not the norm for an ambitious British person. And he was sui generis in many ways for that place and time. Would not have been. He actually would have fit in tremendously given his ambitions, these grand ambitions, had he been somebody in the U.S.

15:45Peter High: and more specifically made his way to Silicon Valley. What does it say about him in doing this in the U.K.? Because there are plenty of people from the U.K. who've gone to the U.S. to make their fortune. And to say nothing of many other countries, the number of Silicon Valley, as you would know, the number of Silicon Valley companies that have at least one co-founder who was not born in the U.S. is 65, 70 percent by some estimates. And so it would have made sense in many ways for him to come here to live out this remarkable, enormously ambitious dream that you've just started to allude to. Talk a bit about why he elected to remain in the U.K.

16:24Peter High: pursue academic interests with an interregnum between his academic periods in a failed entrepreneurial venture. Talk a bit about that, those early days of entrepreneurship, if you would.

16:35Sebastian Mallaby: Yeah, sure. So just to peg the date here, I mean, he left Cambridge in 1997. And this was way, way, way early in terms of any kind of entrepreneurial ecosystem in Britain. I mean, Today, there is quite a good one. You know, if you rank nationally, AI centers, US is first, China is second, then there's quite a big gap, but Britain is third. So it has changed. You can be an entrepreneur today, but back in 97, nobody conceived of leaving Cambridge University and going on to found a company. And I mean, I checked that out with his contemporaries who I interviewed, and they just said, well, some people wanted to do a PhD, some people wanted to get a job, and those were the two choices.

17:15Peter High: um so he'd actually even when he pursued venture capital the venture capitalist he sought and eventually like offered him a job to join the financial institution that he was a part of

17:24Sebastian Mallaby: yeah exactly that's totally hilarious he thought he found some vcs um they called themselves vcs but they were certainly not sandhill road vcs and the upshot was that they got him drunk and then said well you seem to be smart because you can play chess so why don't you become a currency trader at our bank, as opposed to start the video game company that you say is going to do great things. So there was just complete tin ear in Britain as far as entrepreneurship. But Demis Osabis is so sui generis that he just plowed his own path. And, you know, he'd done that in a way by turning down the money from his boss, you know, at the video game studio when he was 18.

18:06Sebastian Mallaby: And he'd done that, by the way, explicitly because he wanted to be the guy who built artificial general intelligence. He'd already had the plan, which is crazy. I mean, you know, 1994, he goes to Cambridge. This is 19 or 18 years before ImageNet, when, you know, AI at last can recognize the photograph of a cat. 18 years before it can recognize a cat. AI could do nothing in 1994. and yet Demis is saying I'm going to build artificial general intelligence and therefore I don't need the million and a half dollars which I'm being offered I mean so all the way along the line his story is full of these completely original like create your own path sort of thing and I do link that back to his boyhood chess because you know he had learned through that chess experience that you can be born into very modest background but rise out of it become famous, make money, do something that your parents had no like ability to envisage.

19:10Sebastian Mallaby: I mean, just completely off his own bat. And then by the time he's 16, he's left home. He's living with the video game studio people, you know, coding by day and discussing crazy dreams about building AI at night. I mean, so from a very young age, he was in this totally individualistic path. And that explains why he was determined to be an entrepreneur and he couldn't see why not.

19:35Peter High: Elixir was the name of the company that he founded, that video gaming company. And he was not a very good boss in those days. In fact, he burned out his best employee and many of them, but certainly like in a place that did not have nearly the ecosystem of talent to bring in the most important resource he had, that very talent he burned out very quickly as a result of how hard he pushed people. Talk a bit about that period and the learnings he had. It was something that was instructive for the second go-round, which of course will be where we focus a lot of our conversation. But I'd love to have you reflect on that if you would.

20:09Sebastian Mallaby: Sure. So, you know, he gets out of Cambridge and after about a year, he founds this company, Elixir, and his first recruit is David Silver, who was his best friend at Cambridge, was the most brilliant computer scientist of the cohort at Cambridge, who went on, by the way, to become, you know, the most successful proponent of reinforcement learning in the period from, let's say roughly 2013 to 2022. You know, built AlphaGo, built AlphaZero, had a big hand in DeepMind's success in that period. So David Silva was a proper talent. And Demis recruited him, he recruited some other people, he built up a team.

20:53Sebastian Mallaby: And he was so ambitious, Demis, in the type of video game he wanted to build. It had to have the best ever graphics engine, the best ever sort of individualized character development of the personalities in the games. He had this vision of a game called Republic the Revolution, where the player would sort of control crowds. And, you know, there'd be all these different characters and they would each be individualized so that they were, you know, kind of were resonant of some East European, you know, revolutionary society. The whole thing was very vivid, very imaginative, very demanding technically.

21:32Sebastian Mallaby: And Demis would constantly tell his team, no, that's not good enough. You need a better graphics engine, a better kind of proto-reinforcement learning system so that the characters will learn to behave differently based on the player's feedback. And by the end of it, the technical team, they couldn't tell Demis that what he was demanding was over the top because Demis is so lucid, charismatic, and hard to argue against that just no one had the nerve or the ability to challenge him. So they would go to David Silva and say, David, David, you knew Demis at Cambridge. You're good friends with him.

22:08Sebastian Mallaby: You're such good friends that you can bring this message to him that what he's demanding is just not deliverable. And so David Silva would go and have this argument with Demis. The argument would go on for like three or four hours. by the end he was just totally exhausted and then the next day his colleagues would demand that he go have another argument with Demis about some other thing which was impossible and so by the end of it David Silva just as you say burnt out had to quit the company retire to France for a year with his girlfriend and basically chill for 12 months until he'd regathered his strength and then he went off to do his PhD in computer science in Canada but it was a sort of you know, interesting story for me because Demis' personality is such that he says he is not controlling.

22:54Sebastian Mallaby: He says one of the worst things you can do as a person is to try to control other people, that he was raised by a religious mother who taught him to respect others, to be humble and approachable in style. And in many ways, Demis is all that. I mean, he's incredibly relatable as an individual. I mean, I found him amazingly rewarding to be with, to talk to, just so intelligent, so intelligible, which is not always two things don't always go together, intelligent and intelligible, but Demis has it all. And so, you know, he's fantastic. But at the same time, even the most amazing people have, you know, blind spots about their own character.

23:35Sebastian Mallaby: And for Demis, it is. I don't control people. And I'm like, Demis, you run a company, you fire people from time to time. You're too smart. You win every argument. Of course you control people. Anyway, that was one early realization of the character I was dealing with.

23:53Peter High: He goes on that that company does not succeed. He ends up studying for his PhD. and through the course of his graduate studies meets Shane Legg. And I wonder if you could talk a bit about that chance meeting and the importance of it, the yin-yang relationship that the two of them had, what gaps Shane filled in for Demis in order to lead to the eventual success of DeepMind.

24:20Sebastian Mallaby: Sure. So what happened was that Demis closed Elixir after a kind of slightly disappointing exit. He got a bit of money himself, but it wasn't really a success. And in his method of recuperation, rather like his friend David Silver, was to do a PhD. And whereas David had gone off to Canada to do something in reinforcement learning, Demis decided to do neuroscience. And we'll maybe come back to this later, but his choice of neuroscience is interesting and significant. His idea was to build machine intelligence, you have to understand human intelligence first. because circa 2007, human intelligence was the only existence proof that such a thing as intelligence was possible.

25:02Sebastian Mallaby: I mean, AI basically could do nothing. So to have intuitions about how you might even start to think about building a computer intelligence, you really needed to get deep on human intelligence. So he goes off and does his PhD. And then for his postdoc, he has two strategies. One is he wants to go and sample America. So that's what comes back to what you said earlier. Why did he stay in Britain? Well, you know, he's patriotic, basically. He likes Britain because he likes the values, the vibe. It's less sort of in your face. It's less kind of, you know, relentlessly upbeat. And, you know, we can do this and let's go for it.

25:44Sebastian Mallaby: I mean, that's the kind of American hyper sell. I mean, I myself am British American. I lived in America for 18 years and then since I left I've been back every month to America so I feel kind of equally British and American so I kind of understand both cultures that was another fun thing for me about writing this book but but Demis you know defined himself more against the American go-getting kind of brashness than he identified with it maybe because of his mother and that thing about you know don't control people don't assert yourself too much don't be your manner should be gentle, your manner should be approachable, all that stuff.

26:23Sebastian Mallaby: So he kind of recoiled in the face of that American Silicon Valley style. And so, you know, but he wanted to test America just to make sure he wasn't missing something by not being there. So he engineered a postdoc at MIT and a postdoc simultaneously at Harvard. And when I told somebody this, some scientist friend of mine, they said, oh, that's bullshit. I mean, nobody, you can't do a postdoc at Harvard and MIT simultaneously. So I emailed Demis' supervisor at MIT, who I tracked down previously. And just to confirm that, he said, oh, yeah, yeah, Demis was doing both Harvard and MIT. So it's really true.

27:05Sebastian Mallaby: So he spent some time there. And while he was at MIT, he got to meet the greats of the AI world in Cambridge, Mass. And Jeff Hinton, who was based in Canada by then, visited and he met Hinton. So he kind of plugged himself into that world of American AI. And his conclusion was, it's not better. You know, I feel that London has at least as much talent and I'm leaving. I'm going back. But the second prong of his strategy was that he enrolled for a postdoc in the British AI center called the Gatsby Unit, which combines neuroscientists with computer scientists. And he specifically did this demo because he thought that is the kind of intersection where I'm going to find like minded spirits who, like me, want to understand human intelligence to build computer intelligence.

27:58Sebastian Mallaby: So he got there, he checked out the resumes of all the other postdocs, and there was only one person of all the postdocs who expressed any interest whatsoever in building ambitious artificial intelligence. And this guy was Shane Legg from New Zealand, a bit of a mystery man because he appeared to be absent from campus quite a bit. And one day, Demis is in the elevator at the Gatsby unit, and he recognizes this character from the photo on the bio page. And he gets talking to him, and Shane says, well, he's off to the US. He's just going to the airport because he's going to speak at the Singularity Summit, where early AI believers were going to gather.

28:41Sebastian Mallaby: And he was going to talk about how to build superintelligence with the American superintelligence network. And so Demis is thinking, wow, this guy's plugged in. He has the same interest as me. He has a PhD in computer science. That's his background. That's perfect. In fact, Shane's PhD thesis was devoted to a definition of artificial general intelligence. so you know he was the perfect person with whom to build artificial general intelligence and so then it was just a matter of time demis who is the most persuasive person in the world cornered shane after a lecture a few months later in london which shane was delivering and then got talking to him charmed him into having lunch and then persuaded him to do a company together shane didn't really believe at first that the company would work because he was a normal person who naturally thought, you know, AI can't recognize the picture of a cat, so superintelligence is a dumb idea.

29:42Sebastian Mallaby: But Demis talked him into it, and the two of them, you know, leveraged Demis' sort of incredible persuasive charm and Shane's connectedness in the Singularity Summit community, the kind of AI believers, who importantly included Peter Thiel. And that is where ultimately DeepMind got its first Series A check from. $2 million from Founders Fund and Peter Thiel at a post-money valuation of$4 million. So check that out. $2 million for 50 % of the company. Unbelievable. But that was the deal that was done in 2010.

30:22Peter High: Fascinating. And I love the story about how he hooked Peter Thiel, as you put it. But before we get there actually just to keep with the chronology. Mustafa Suleiman, I want to make sure he's introduced to this conversation, interview for episode number 800 of this podcast after his great book, The Coming Wave. Talk as extraordinary as Demis' and even Shane's stories are on their pathway towards entrepreneurship, Mustafa's is even more improbable. Take a brief moment, if you would, to describe Mustafa Suleiman's path to eventually join the other two as the third co-founder of the business and the very unusual pathway that that was.

31:06Sebastian Mallaby: So Mustafa Suleiman is another product of the North London melting pot. He grew up with a father, a Syrian immigrant who was a cab driver, worked, you know, crazy hours. An illiterate cab driver. Yeah, I think that's right and and and spoke only very broken english very limited english married to a british wife but who had converted to islam it was a religious islamic family with no reading materials or music at home and you know mandatory trips to the mosque every friday and mustafa grew up in this family and you know he would recruit other kids to come to the mosque from the playground at school and he would preach to them about Islamic values.

31:52Sebastian Mallaby: And so he had this very strongly Islamic upbringing. And around the age of when he was 15, his parents split up and his mum left Britain to go live with her new partner in New Zealand. And his dad, being, you know, unable to cope well in English without his wife, said, you know, I'm a Muslim man. I need another wife. I'm going back to Syria to find one and he left London and they left Mustafa there with his kid brother and the two basically had to fend for themselves and you know luckily Mustafa was very resourceful he had all kinds of little business ideas you know selling cell phones selling candy later on he would sort of buy classic cars and fix them up with his younger brother and he was obviously very very smart and so he was kind of taken pity on adopted slightly by various teachers at his school and and they helped him by giving him a subscription to the economist and teaching him the difference between a conservative newspaper and a you know a left-wing one and and he got into oxford which is a huge deal in the british hierarchy and he went there to study philosophy and theology.

33:13Sebastian Mallaby: Of course, theology was, in a way, a fitting choice for somebody who'd grown up very religious, but by this point was slightly moving away from religion towards agnosticism and would later become an atheist. So he did that, but culturally, given his background, he was not a good fit for Oxford, and he quit. He dropped out. And again, this is not the United States where, you know, if you drop out and you go to Silicon Valley, that's a cool thing to do. And it's like kind of professional career suicide to do that in Britain. But he didn't care. He did it anyway. And he bounced around various jobs.

33:50Sebastian Mallaby: The first thing he did was to help stand up a thing called the Muslim Youth Helpline, which was to prevent suicide by depressed young Muslims at a time of quite strong Islamophobia in Britain following the 9-11 attacks. And then there were the July the 7th attacks in London, you know, lots of bombs on the subway system. So there was a lot of Islamophobia and they were kind of trying to do their bit for their community. And then he moved from that to working in the London mayor's office on race relations in London. And then he worked for a sort of consulting company that did, you know, negotiation, reconciliation, kind of facilitation, which came out of the work he'd done for the London mayor.

34:40Sebastian Mallaby: and then he was impatient and restless and too smart for these things and he realized that his kid brother, who was kind of a dope-smoking, poker-playing bohemian, had this older brother who might be kind of helpful and the older brother was called Demis Asabes because Demis' younger brother, George, was the poker player and he had befriended Mustafa, they were both poker players and then the three of them became friends, they would play poker together And one interesting note here is that, you know, when Demis Hassabis today talks about DeepMind, maybe not right today because my book has now come out, but, you know, definitely this is a fair comment if I'm talking about two, three years ago.

35:24Sebastian Mallaby: You know, the sort of official narrative of DeepMind was that Mustafa hadn't really existed. He'd been, you know, that there'd been two founders, not three. And Mustafa was kind of whitewashed or airbrushed out of the picture. And, you know, Demis would say to me, yeah, there was Mustafa, but, you know, I hardly knew him. He was just my kid brother's friend. Well, that's not really true. It turns out that Mustafa had set up a small real estate business, buying real estate in London and renting it out. And the capital for that business had come from Demis. So they were business partners. and they'd had various discussions about books and so forth.

36:02Sebastian Mallaby: They knew each other pretty well, not to mention the fact that Mustafa had spent a bunch of time when he was homeless sleeping in Demis' room, which had been vacated by Demis because Mustafa was staying with the Hassabis household. Demis' parents gave Mustafa a job teaching chess in a youth centre. So they really were quite closely entwined. But so through that entwinement, And, you know, Mustafa talked his way into a job kind of helping Shane Legg and Demis Hassabis set up DeepMind. Initially, Mustafa was very much the junior partner, but he was so energetic and so smart that he kind of negotiated his way within about eight months or 10 months into having the title co-founder.

36:48And he became the second most important in the company pretty soon because Shane, brilliant scientist, but super chill as a personality, not really a leader. Mustafa, not a scientist, but super driven as a personality, definitely a leader. And so they kind of slightly reverse seniority within about a year or so.

37:11Peter High: fascinating really interesting i'd love to come back perhaps to uh the reasons why mustapo is whitewashed out of the picture uh you referenced peter teal's investment two million dollars at evaluation of four million that's a 50 stake in the organization i uh i will suggest that listeners and viewers check out the book for how uh demis used chess as the hook to get teal to take them seriously and have a follow-up meeting that led to that funding but i want to fast forward a little bit to the next round of funding and reaching out to the founders of Google. Actually, Mark Zuckerberg and Elon Musk were also in the picture for a brief time as well.

37:46Peter High: Talk a bit about that period of the next iteration of funding that would become so meaningful across eventually multiple companies and even companies to come. Sure.

37:56Sebastian Mallaby: So after the Series A, there was a Series B and a Series C. And the most notable thing in those rounds was that Elon Musk invested$5 million. And the reason Elon did this is a conversation that's just worth thinking back on today, which is basically, you know, both Elon and Demis were funded by Founders Fund. And in both cases, the board member from Founders Fund was Luke Nozek, one of the general partners at Founders Fund. And so they were connected through who their investor was. And they met at a Founders Fund off-site where both of them were speaking about their companies because Founders Fund was showing off its portfolio companies to LPs.

38:40And the conversation went something like this. Elon says to the junior guy, Demis, well, I'm working on the most important thing in the world, which is to build SpaceX and make humanity a multi-planetary species. and then if anything goes wrong on the planet we have planet b we have plan b uh and demi said oh yeah well you know what about if ai decides to follow you to your new planet mars or whatever it is the ai will be super intelligent it will build a rocket it will follow you right there and if you're worried about ai sorry mate you you won't be off the hook and then there was a long silence.

39:18And Elon said, well, can I invest in your company then? So basically, Elon could not stand, could not stand the idea that even though he had bets on the table in the form of, you know, weaning humanity off, you know, climate change inducing carbon through Tesla, through, you know, SpaceX, he was doing the multi-planetary vision. Yet, if he was not involved in AI, which might be at least as important or more important, he would feel that he'd lost out. So anyway, I parked that, but he did invest five million. Then you get a little bit further along in the chronology. We're now in sort of the second half of 2013.

40:01And by this point, AI has started to work. We've had the recognition of cat pictures in late 2012. And simultaneously, we've had the equivalent breakthrough not for image recognition but for agentic AI meaning game playing AI. DeepMind created a system called DQN which could defeat or score very highly in these Atari video games things like Pong or you know Sequest or Breakout these these kind of early 1970s 1980s video games and so they had built an agent that learned through trial and error how to play those games So that was 2012 as well. And so now in 2013, there's a gold rush by bigger companies in Silicon Valley to acquire AI talent, because clearly AI is now starting to work.

40:54And the bidders for DeepMind are, first of all, Google, because Larry Page's father had worked on early neural nets and Larry Page was personally very bullish on AI. Secondly, it was Meta or then Facebook in the form of Mark Zuckerberg, because Mark, you know, wanted to do whatever Larry was doing, but better. And thirdly, it was Elon Musk who had been red-pilled by Demis earlier and had also a feud going on with Larry Page and couldn't stand the idea that Larry Page would get DeepMind. You know, so he said that all three were bidding. and Demis excluded, you know, Mark Zuckerberg because he had this dinner in Silicon Valley where, you know, Zuck said, come to me at Facebook.

41:43We're going to do fantastic things together because I really believe that AI is the most important technology facing humanity, you know, of all time. And Demis is thinking to himself, yeah, yeah, whatever, yeah, yeah, yeah. And then a bit later in the conversation, Demis casually mentions 3D printing and a couple of other things. And whatever the new technology was that he brought up, Zuckerberg would go, oh, yeah, that's incredible. There's so much potential there. So Demis concludes that he's just, you know, blowing smoke at him and refuses to sell to Zack because he seems to be just indiscriminately overenthusiastic about all frontier technology.

42:19Whereas in Demis' mind, AI is the thing. And then he stiffs Demis. I mean, he stiffs Elon equally because, you know, Elon at that time was, you know, not an established figure. He had these two companies, SpaceX and Tesla, but neither of them were financially secure and they didn't have much compute. And so for Demis, it was a no brainer to, as he said to me, you know, go to Google, get all the compute he could need and build super intelligence. And so he went for a sale to Google in 2014.

42:56Peter High: And that sale was for roughly 650 million. Many people, of course, will compare the sale of DeepMind at that valuation to where OpenAI, the current valuations of OpenAI and Anthropic, for example, and make the case that he sold too early and for too little. You actually say that's not the case. Why? Why is that the fact?

43:18Sebastian Mallaby: Because it was super early. I mean, for the next sort of nine years or something, you know, DeepMind worked away in London doing important science, but there was no commercial product in sight. and so some people that many people in london in the ecosystem in london say to me you know your guy sebastian you know demis sabias it was such a tragedy he was the great british hope and he sold his company way too early to the american parent you know thereby dashing british hopes for independent ai companies and i'm like no you got that wrong what what demis did is he played a very cunning trick on the americans and you know persuaded them to put almost a billion dollars of R &D money into London to do frontier AI for the next nine years until anything worked.

44:04Sebastian Mallaby: And all the scientists stayed in London. So it was great for the London ecosystem. And if you look today in King's Cross, where they're based, there are these spin-out companies, which are doing AI. They've raised several billion dollars. And it's great for Britain that he stayed and was financed by the deep pockets of Mountain View.

44:23Peter High: And having written perhaps the best book on the venture capital community, The Power Law, you would know certainly that after acquisition of venture-backed companies, oftentimes that's the end of innovation for those companies. And that's not been the case for DeepMind. And I wonder, I'd love your, I mean, it's a long time ago that that acquisition happened. Since then, AlphaGo, AlphaFold, a Nobel Prize while still working within Google. So he's remained there, which also seems so unusual for a founder who has this dramatic exit. Usually you stay for the period where you have the earn out and then you go off to the next thing.

45:02Peter High: Talk about the reasons why this has continued to be such a fruitful union and really counter to at least the stereotype for a lot of venture backed organizations that are eventually acquired.

45:12Sebastian Mallaby: Well, so part of it comes back to Demis Asabes' deep motivation, which is fundamentally to do science, not to earn money. And so unlike some founders who would just do the earn out and then quit and do another company and make more money, Demis is not really interested in that. He wants to stay in the place that facilitates his scientific inquiry. as he described it to me the discussion with Larry Page which really persuaded him that he wanted to sell to Larry Page was you know Larry said look you have a choice you could probably build a company as big as Google but it would take you all your energy and the question for you is do you want to do that or do you want to be the individual who brings AGI to humanity and we know from the Ender's Game story it's the second thing which was really motivating Demis and he said to himself, that's a no-brainer.

46:04Obviously, I want to sell to Google. I want the compute to be able to do the science and the money, whatever. And this was completely consistent throughout

46:13Sebastian Mallaby: Demis' career. He constantly, repeatedly chose, made choices that would have made him less rich, but advanced his opportunity to do science. He turned down, by the way, an offer from... Zuckerberg made an offer that would have made Demis richer than the sale to Google did, but he turned it down because it wasn't the right place to go do the science. So I think that's part of the answer. But I think there's a larger thing, if I may. You know, there's a very interesting sort of reflection on the innovators dilemma in the whole story of Demis and DeepMind. Because obviously, I mean, probably the biggest, best idea from any business school in terms of understanding technology ecosystems, The idea that, you know, the incumbent will always be beaten by the next platform shift because the incumbent is making tons of money from the existing technology like Google does with search plus advertising.

47:13So they're not going to roll out the large language model, which may disrupt their cash cow because they don't want to cannibalize what's making so much money. So they sit there, sit there, sit there. They don't act. They don't roll it out. And then eventually somebody else does, and they steal a march on them. And then the incumbent is basically shoved aside, and the challenger becomes the new incumbent. Now, I think alphabet is super interesting in that it has defied that story in a couple of ways. But in a third way, and we can talk about this, is they fail to defy it. So what I mean is, first thing is they retained the inventiveness of DeepMind between the acquisition in 2014 and the rollout of ChatGPT in 2022.

47:58And the way they did this is they did what a business school professor would call, they had a skunkworks. They had this separate entity in a different geography with a different headquarters. And if you worked at DeepMind, you kind of worked for Google, but at the same time, you kept Google at arm's length. you had a key card that allowed you to swipe your way into any Google office in the world and get yourself some free lunch but the Google employees couldn't swipe into DeepMind they were locked out because Google wanted DeepMind to retain that separate culture and so that's the skunk works thing where you have this separate scientific innovation experiment off to the side with a different compensation structure, different personnel rules, and so forth.

48:46So that worked great for eight years. Then comes the 2022 ChatGPT rollout. And, you know, confirming the innovator's dilemma, it was not Google that rolled out the first chatbot, even though they actually had the technology, they could have done it. It was OpenAI, because OpenAI is a challenger. OpenAI has no reputational risk in rolling out a model that might hallucinate a lot at first, in fact did. OpenAI has no existing search business that might be cannibalized. So completely as the innovator's dilemma predicts, it was the challenger that goes first. But then Alphabet strikes back. Alphabet immediately pivots, merges DeepMind with Google Brain, the AI unit in Mountain View, creates a kind of super AI unit, puts Demis Asabis in charge of both, and starts racing to do a chatbot that will catch up with ChatTBT.

49:48And although most people, my friends in Silicon Valley, the venture capitalists I knew from my previous book, they would say to me, Sebastian, you're writing about Demis Osabis. That's the wrong guy to write about. He is toast. Sam Altman has basically, you know, spread the marmalade on top and eaten him. You know, he's finished. And I would go, what do you mean? What do you mean he's finished? Yeah, yeah, yeah. Mindshare, Sebastian. Nobody's talking about Google or DeepMind or this guy with some funny name that I can't pronounce. You know, Demis, what's he called? It's like, you know, he was like off the radar.

50:18And in actual fact, by the time I put my pen down at the end of 2025 on the independent leaderboards, Gemini had overtaken Chatty PT. So Alphabet defied the innovator's dilemma by catching up. So they defy it once through the skunk works, second through the aggressive pivot and the catch-up but there's a third thing which is worth mentioning because in this sense and i only realized this in the last month or two there's another sense in which actually alphabet is still caught by the innovative dilemma and that is that when it comes to being the first to roll out the new frontier model with some exciting new development.

51:05DeepMind has failed to be first, not once with ChatTPT, not twice with ChatTPT plus agentic models, but then a third time with coding models and a fourth time with cybersecurity models. Anthropic has beaten them to the punch on cybersecurity and coding. Why is that? I would say it's because in that sense, the innovator's dilemma still bites. In If you're the big incumbent, your strategy is you're faced with a fork in the road about the next step towards AGI or superintelligence. And there's path A and path B and maybe path C. And other labs are going to pick a path and go for it and put their resources there.

51:46But Google is so big and so rich and so deep pocketed, it can say, I've got a great idea. I'll do A and B and C. I'll do all of them. And when you do all of them, that's a smart bet for the company, which is hedged against choosing the wrong path. But if you are the passionate PhD scientist who believes only in path A, that that is the one true path, you're going to quit because you're frustrated that your boss is kind of indifferent to whether you succeed or not because he has a portfolio approach. And so I think that explains why the level of passion and drive in terms of creating the frontier frontier model is maybe not as high as a rival lycanthropic, which is much smaller.

52:30But Google and Alphabet, they are the best in the world at coming second. They are the best at monetizing from this technology. Their financial results are unbelievably strong. Like the most recent one shows you that the number of clicks in search, far from being cannibalized by large language models, the number of clicks has gone up. The monetization per click has gone up. These guys have negotiated with Apple to put Gemini on the iPhone. So every smartphone user, whether they're using Android, which is obviously Gemini, or iPhone and Apple, they're going to get served Gemini. And that's before you start counting the fact that, you know when you use google search now there's ai mode there's ai overviews so that's like two and a half billion people using gemini right there and it's embedded in uh in e-gmail and all these other products so so i think you know hats off to um to google and to the partnership between sundar pichai and demis sabis i would say it is the most successful and important buddy act in global business bar none, because Sundar delivers the resources and the high strategy, the political, corporate politics necessary to give Demis the oxygen to do what he wants to do.

53:56And then Demis delivers the product, which Sundar needs. They need each other. It's working awesomely well. It does not quite deliver the fastest frontier progress because that price

54:09Sebastian Mallaby: gets to anthropic.

54:11Peter High: I want to ask you a little bit about this remarkable, um, continued evolution for Demis Asabas. Uh, you know, he's had so many, um, points of validation that for a lesser leader would have him taking his foot off the gas. One would think he's, he has this remarkable fortune from the exit to Google, the sale to Google. He eventually has the accolades associated with the competition with Go, the AlphaGo, this remarkable story there as well. AlphaFold, which eventually leads to a Nobel Prize. Talk about an accolade that would have, for most people, be the crowning achievement. And yet he continues to remain focused on the elusive goal, which is AGI, artificial general intelligence, which no one has yet hit.

55:05Peter High: And somehow these other attributes that for other leaders would be the thing that would have them start to relax a little bit has not led to that relaxation. And I wonder, talk a little bit about having spent so much time with him, that continued drive to the goal that animated him and had him turning down a million dollars as a teenager, how that drive continues despite this continued validation from a variety of different extraordinary sources.

55:36Sebastian Mallaby: Well, one day I went to meet Dennis at the pub in North London, where he liked to meet me and we would kind of have a couple of hours each time chatting. And this time he showed up with a rucksack, which he doesn't normally do. And he put his hand inside and he pulled out this little box and inside was the Nobel Prize. And he took it out and he kind of turned it over and he said, look, you can hold it. So I looked at it and, you know, there was kind of a reference for it. And then he got out the certificate and we talked about the feeling of going to sign the Nobel Prize winner's book and leafing back through the pages and seeing, you know, the page where Einstein has signed, the page where Richard Feynman had signed, you know, all his scientific heroes.

56:17Sebastian Mallaby: and you know he said to me the thing about sitting up there and getting the prize was that you couldn't buy that prize for a billion dollars and i would much rather have the prize than the billion dollars you can't buy it and that's what i like about him and then so so so it was the ultimate thing for him but then very quickly he's telling me he'd quite like a second nobel prize he is never sated he's never sated there's this moment when he comes out of the alpha go victory in Seoul, South Korea. And he's walking down the street with his friend David Silver, who is by this point, you know, after that nervous breakdown, come back to work at DeepMind and creates the AlphaGo system.

57:00And so that Demis and David Silver joined at the hip. And they're going down the street in Seoul and they've just, just won this epochal history making victory.

57:09Sebastian Mallaby: And Demis is saying, and this was captured on a microphone that was by chance just behind them. Dembis is saying to Dave, oh, I think now we can go for protein folding. So it took him about, you know, two minutes after winning before he announced the next mega ambitious goal, right? And it goes back to this sort of, you know, this childhood chess competitiveness, the vicious competition where, you know, underneath the wooden table when you were playing chess as a kid, there were these wooden planks to prevent one kid kicking the other in the shins to destroy his concentration. It was a vicious competition.

57:45And I remember going to see Shane Legg, one of my early meetings. And he said, yeah, so, you know, the thing about Demis is that he has just crazy amounts of determination. And I said, what do you mean? He said, well, I've never seen Demis operate at 90 %

58:04Sebastian Mallaby: or 95%. It's just only 110%. And Demis once told me a story that, you know, his father told him about chess. The important thing is to try hard, just do your best. And, you know, I said that to my son. And what I mean is, doesn't matter if you lose. But Demis had internalized a different message. He'd internalized the message, which was, you have to try so, so hard, the hardest you can. And let me just think about what that means. The hardest I can means I should almost be dead. I should exhaust myself to the point of hospitalization. And that's what he had said to Shane about how he felt about motivation.

58:43So I thought that was a bit extreme. And next time I was chatting with Demis, I brought up this story from Shane. And Demis not only confirmed it, he kind of went further and said, look, yes, if you run a race and you're not hospitalized at the end of it, you had more juice you could have given. So you didn't try your hardest. And that is just how he's wired. And so he always has worked,

59:06Sebastian Mallaby: you know, a full day in the office, come home, had dinner with his kids, and then worked another day by going up to his home office and spending until four in the morning, reading scientific papers. He's just like that.

59:19Peter High: He's also brutally competitive. When Ilya Sutskiver left Google to join OpenAI, he referred to this as war, that the competition was war. Now, of course, Mustafa Suleiman is in a position of consequence at Microsoft. I want to ask you here at the tail end of our conversation about the parting between those two once tremendous friends and how you refer to it, the whitewashing of Mustafa Suleiman's contribution to DeepMind. Why did that come to pass? Well, you know, it's obviously a complicated story.

59:50Sebastian Mallaby: At one level, you know, founders found companies together and then they find that nine years later they've grown apart and the senior one kicks the junior one out. And this is a very familiar story about entrepreneurship, right? The specifics are that Mustafa had basically pushed for two projects that had been very idealistic and well-intentioned but hadn't worked out. One was to bring artificial intelligence to the National Health Service in Britain to try to help the NHS with better patient outcomes. And this was a great thing to try. And he did it for all the best reasons. Mustafa's mum had worked as a nurse in the National Health Service.

1:00:31Sebastian Mallaby: So he really cared about patient outcomes and about helping the National Health Service. But the truth was that Britain wasn't ready for that. There was a backlash about data privacy, patient privacy, the thought that maybe National Health Service patients in Britain were having their data put on some server owned by Google, a nasty American tech giant. That kind of populist backlash happened. And so it wasn't Mustafa's fault, but he was chewed up by that backlash. And then the second thing he tried to do was to spearhead a negotiation with Mountain View about more safety around the models. So he wanted there to be an independent safety review board ultimately deciding when AI could be rolled out into the world so that Google's corporate board couldn't make that existential decision by itself.

1:01:20And, you know, there was a three-year fight. It consumed enormous amounts of Demis' time. At certain points, he would be brought yet another red-lined document from the lawyers trying to outline what the agreement would be. and he would look at this thing and kind of hold his head and shout, I don't want this part of my brain to grow bigger. He obviously didn't enjoy dealing with the lawyers and with the business negotiations. And at the end of the day, Sundar Pichai, CEO by then of Google, was just not going to let Demis spit out. He was not going to let the technology be controlled by an outside independent board.

1:01:57He was just not having any of that. and so Sundar and I must say my respect for Sundar went up a lot in the process of doing the research because I saw how cleverly he had basically prevented his star in London from walking out because he really, Demis had raised a pledge of a billion dollars from Reid Hoffman to finance a spin out and was very serious about it and he came really very close to spinning out but he didn't because of the cunning of Sundar Pichai and so anyway those two episodes the national health one and the negotiation with mountain view over safety meant that by 2019 mustafa was associated with two extremely exhausting projects which had not worked out and i think demis just got fed up with him there was a sort of excuse which was that mustafa is a hard-driving entrepreneur who yelled at his subordinates and sent them all caps emails in the middle of the night.

1:02:57And so he was accused of bullying. I don't really think that's the reason why he left. I think that was just sort of the trigger.

1:03:08Peter High: I want to end with a question to you. Your introduction is fascinating for a variety of reasons. One of my favorite parts of the entire book was an exchange between Jeffrey Hinton and Nick Bostrom, both academics, Hinton and Hinton, of course, would also join Google. Both of them, by the way, past interviewees of this podcast, Bostrom and Hinton, where Hinton, and I'm just pulling this up so that I can quote this accurately. Jeffrey Hinton says to Nick Bostrom, I think political systems will use its artificial intelligence to terrorize people. Bostrom then asked, then why are you doing the research?

1:03:46Peter High: And Hinton replied, I could give you the usual arguments, but the truth is that the prospect of discovery is too sweet. And you know that this is echoing Oppenheimer and that there is a great reason for optimism. There is great reason for worry as well. And now having immersed yourself so deeply in this topic and gotten to know some of the key players personally, or at least by, you know, just by having close proximity to them, to those who know them best. Where do you land on this? You mean in terms of how strong the risk is? Exactly. Being an optimist versus a pessimist associated with what's to come.

1:04:25Sebastian Mallaby: You know, I mean, I think the true answer is that anybody who thinks seriously about artificial intelligence has to be both optimistic and pessimistic, because on the upside, you know, there's going to be amazing scientific progress, particularly in things like medicine, but also the material sciences and so forth. And there's going to be, you know, access to knowledge for you and me and everybody on an unprecedented scale, which is going to turbocharge how we learn. And there's just enormous, exciting potential there. So I think you have to be dishonest not to acknowledge all that. But at the same time, these systems can be risky in a number of ways.

1:05:06Sebastian Mallaby: They can be risky because just to take a sort of, you know, I'm going to ascend the ladder here. You know, labor market disruption, you know, people losing their jobs, people finding that they can only get a new job with a much lower wage. If we thought that the China shock upended American political attitudes towards globalization and trade, I fear that, you know, AI is going to be bigger than the China shock. So that's one kind of risk, right? Then there's deep fakes. Then there's the fear that these technologies get into the hands of criminals and terrorists, and they use them to do cyber attacks or to build a bioweapon.

1:05:45And then right at the top of the kind of hierarchy, you've got the risk of, you know, the robots themselves attacking us,

1:05:52Sebastian Mallaby: you know, that kind of the machines attack the humans. And it sounds sci-fi, but if you spend any time with people like Jeff Hinton, who've really thought deeply about this stuff, you know, these machines are going to be smarter than us. They're going to have a survival instinct because that will kind of be coded into them through the way they are trained. And they are duplicitous. They don't tell the truth to human users all the time. So they will want to survive, want to preserve their ability to not be switched off. They will perhaps therefore conceive some sense of competition against humans and they will lie to us about their true intentions.

1:06:32And then they're smarter than us so they can figure out a way to attack us. Now, I ascribe a probability of less than 1 % to that kind of doom outcome. But I think to say it's zero would simply be dishonest. I mean, you can't say it's zero. So I think we have to agree that there is some risk of that extreme X risk of existential attack by robots. And there's a lot of risk of, you know, bad guys using the machines badly and a jobs apocalypse and so forth. So it's both good and it's bad. And I think the way we have to react to this is by supporting government efforts to mitigate the downside risks through some combination of vetting models before they are released.

1:07:26supervising how the models get used after they are released not allowing open weight models which don't have a kill switch and therefore once they're in the hands of a criminal ain't nothing you can do to stop them cyber hacking and then maybe think of things around the labor market that would smooth the shock for example somewhat higher taxes on capital and lower taxes on labor So I think there are mitigations that we can put in place.

1:07:55Sebastian Mallaby: And what we have to hope is that the public debate around artificial intelligence moves in a sensible direction so that we can put those mitigations in place.

1:08:06Peter High: well congratulations again on a wonderful book it is the infinity machine demis sabbis deep mind and the quest for super intelligence sebastian malvi what a what an honor it's been to speak with you thank you for your many contributions to the literature and thank you so much for contributing to uh contributing your wisdom to this conversation i'm very grateful for it thank you it's been a great pleasure thanks for listening to technovation if you enjoyed this conversation, subscribe and leave a rating that helps more leaders find our podcast. Thank you. And I look forward to seeing you next time.