Guillaume Verdon: Exploring the Intersection of Quantum Deep Learning and AI

0:05Hey everyone and welcome to Generative Now. I am Michael Magnano. I'm a partner at Lightspeed And this week, I'm speaking with Guillaume Verdun. You may know Gil as the founder of Xtropic, a company creating physics-based AI processors. You may also know him as the person behind Based Beth Jaisos on Twitter or X, and the founder of EAC, the movement that is advocating for rapid acceleration of artificial intelligence. We talked about a ton of things in this episode. I think you'll enjoy this conversation with Gil Verdun. Hey, Gil. Hey, Michael. pleasure to be here how you doing doing good doing good I've been very much looking forward to this there's so much to get into uh obviously want to talk about Extropic want to talk about EAC would you be willing to take us back um you know maybe even before Alphabet some of your research I think would be really really interesting to start with yeah I mean you know ever since I was a ever since I was a kid I wanted to understand the physics of the world because if you really understand the stack of the universe, then you can hack it better.

1:08You could produce more interesting technologies. So, you know, that journey led me through theoretical physics, right? I was trying to understand, you know, as one does growing up in theoretical physics, you're like, oh, I'm going to figure out the unifying theory of everything. I'm going to write a couple of equations and they're going to be able to predict everything in the universe. And then everybody goes down that path, hits a wall, and then realizes that actually there's a lot of complexity in the universe. And there are things that you can represent complexity with complexity. And so I had a sort of change of heart throughout my career in theoretical physics that instead of having sort of human interpretable laws of physics, we're actually going to augment our intelligence with machines, right?

1:52And similar to how there's complexity and emergence in nature, we have complexity and emergence in our algorithms with deep learning and AI, right? And to me, that was one of the most profound things that we can mimic nature's complexity with these artificial, complex, self-organizing systems. Most people are familiar with deep learning and neural networks, but there's a bunch of them that you could build. And so for me as a theoretical physicist, trying to have a way to understand the world at all scales, that led me to a path to go into and pioneer a field called quantum deep learning, which is basically how to extend notions of intelligence to understand the quantum mechanics of the world, which, you know, a lot of the world around us is quantum mechanical at the very small scales.

2:37And day to day, it's not as relevant because it's, you know, we're at the large scales. But if you're - Explain what you mean by that. A lot of the world is quantum mechanical. So, you know, you have different laws of physics that are the best theory, the most practical theory at different scales, right? Like you don't need to do particle physics simulations like the LHC to predict, you know, the temperature of your coffee in the morning, right? It's like a different scale of physics. And so you have different theories at different scales. It's called, you know, renormalization or effective theories.

3:09And at the very small scales, things are actually quantum mechanical. Things are not even probabilistic. Things are in superpositions of different configurations. So you could think of them as sort of having parallel universes and they can interfere with one another and then you either collapse or condition on being a certain branch when a measurement happens. And so it's very funky, but a lot of GenoVeI is based on probability theory and statistics and information theory. And there's actually generalizations of all these things to quantum information, right? And so I came from a school of thought that was trying to revolutionize theoretical physics through the lens of quantum information theory, right?

3:52So my mentor, Patrick Hayden, who's now a professor at Stanford, he was kind of the Claude Shannon of quantum information theory. And so that's what got me into this sort of school of thought. In a way, you could think of information theory being about entropy and compression and so on. And AI algorithms kind of operationalize information theory. They search over the space of transformations to figure out how to get a compression code or get an error correcting code, that is what information theory predicts is possible, right? And so extending the sort of philosophy of AI and deep learning algorithms to quantum mechanics and having those algorithms run on quantum mechanical computers that were kind of emerging at the time in like 2014, 2015, right?

4:39That is what I was doing. So I was one of the first people to propose running a quantum neural network. I ran the first algorithms on Rigetti, for example, which was you know, one of the first Silicon Valley plays in quantum computing. Very early on, I realized that quantum computers were going to take some time to scale, right? It's kind of a similar to nuclear fusion. It's kind of a civilizational, you know, important note of the civilizational tech tree, but it's going to take some time to get there. And so, you know, I decided to pay my dues and sort of connect the world of theoretical physics with quantum computing through quantum AI.

5:15I still believe in that being sort of the sort of end game to have the deepest answers about the universe. It's going to be a quantum mechanical form of AI that's combined with our modern systems. You're going to answer those questions. But, you know, as I was seeing sort of gen of AI slowly take over the world or eat the world of software, I saw, you know, opportunities to use the sort of background we had in physics at the intersection of physics and AI to do different kinds of devices that are not quantum mechanical. And that's what led me to, you know, doing Stropic after a career in quantum tech, more broadly, at Alphabet.

5:53Got it. So just connecting the dots a little bit, you mentioned that early on you wanted to understand the world. And you said, you know, in physics, you, I think the words you use, like you hit a wall jumping ahead, like is, is the idea that basically quantum deep learning is, is the way that we will sort of break through that wall and be able to better understand things that we just can't with, with the tools that we have at our disposal. Yeah. So, so, so it's a big question of how, like how far can representations that work on classical or probabilistic computers, like we're building, how far can they reach into, uh, understanding and compressing the world, right?

6:33Like let's say you had to figure out one embedding, one compression code for the whole universe, right? What would that look like? And studying sort of the limits of compression in nature, actually, I used to study black holes and how they compress information because they're the densest objects in the universe. And what I was teasing out of it is that actually black holes, their representation or their compression code uses a mixture of a classical representation and a quantum representation. So that's how I, you know, pushed the algorithms towards that area. TensorFlow quantum was how to hybridize regular deep learning with quantum deep learning.

7:11And, you know, I continued that, that sort of research at X. But yes, I would say that, you know, let's say you're trying to understand materials. The alphabet X. The alphabet X, to be clear. Yeah, yeah, there's new X now. Which also plays a role in your story. Yeah, which is a big part of my life now somehow. So to me, I think like if fundamentally you think of intelligence as compression. If you want to learn a compression code for all the data in the universe, it's going to be quantum mechanical in part. It's actually, you know, nature has a lot of complexity and you're going to tackle complexity with complexity.

7:41You're going to tackle compute and nature with compute that is artificial. And actually really digging into that question of what is the complexity of nature and what is the difficulty of learning representations of complex systems? It's what gives me so much sort of first principles backing to saying things when I am in AI debates, like, foom will never happen fundamentally, right? Like, there's a sort of, like, inherent difficulty in grokking the physical world, right? There's an, as Stephen Wolfram would say, there's an irreducibility, right? You have to use similar amounts of compute to what, to the compute used by nature to have an emergent behavior, right?

8:22It's really hard to predict at large scales what's going to happen for the microscopics, right? Just like, and this is true for many complex systems, whether it's your human body or the economy, right? Or, you know, name it. There's quite a few complex systems out there. There's a conservation of difficulty. You won't have one God algorithm that compresses the whole world and, you know, can turn it into gray goo overnight, right? I think that's pretty provably impossible from what we know. It's just a weird kind of science, right? It's a weird kind of science because it's beyond like reason, right?

8:58It's like, it's like, you can't even put into words why a certain material is going to be superconducting, right? It's a phase of matter. It's literally like a vibe shift if you're like, well, this feels different, right? And that's why we started using AI to detect phase transitions and matter because we don't necessarily have words for these kind of emergent behaviors at larger scales. And that's why, you know, I'm super AI pilt, right? Like I think the future of physics, the future of being able to predict, you know, perceive, predict, and control our world is through AI. And I have been uniquely focused on physics-based AI.

9:35So physics inspired by the world to understand the world in part. Of course, you can use it to understand all sorts of other complex systems. But then the dual to that of embedding AI, the dual of embedding physics and AI is to embed AI into the physics of the world, into physics-based devices. And so that's what I spent my whole career doing. That sort of duality between the physics of the world, the AI algorithms, and the physics of the devices that we use to run these algorithms. You have a couple different threads to your career right now and from what I can tell your life, but what do you say the overarching, what is your overarching mission statement?

10:16Yeah, I mean, it's really accretive to scaling intelligence throughout the universe, right? And with EAC, you know, we're figuring out how to scale up Kardashev scale, which is the energy production and expenditure scale of civilization, right, on the log scale. So that's creating more watts per civilization, let's say. We have one civilization right now. And with Xtropic, it's more intelligence per watt. And so if you scale both of that, you just have more intelligence in the universe. And yeah, to me, it's been, you know, seeking answers to what does it all mean? Why are we here? And where are we going?

10:56But I think there's starting to be sort of nuggets of answers here in this sort of principles of self-organization and viewing all of Earth as one big Petri dish that's like a self-organizing system. It's a very useful abstraction. It's very sort of, you know, people sometimes are taken aback by that. It's too physics sort of brain. It's not anthropocentric enough, but, you know, at the end of the day, we're all embedded in the laws of physics. And, and, but at the same time, it's a very beautiful concept that the laws of physics have driven us to self-organized and help us create all the beautiful order and all the beautiful things we know day to day.

11:39And it's steering us towards even greater things, you know, on average, right? And we're part of this process and we're conscious that we're part of this process, which is a weird, it's a weird thing. So it's kind of a sort of, we're kind of leading a school of thought that will, you know, originally I was trying to change theoretical physics with it, right? Like emergence and self-organization and AI for complex systems. I think we're bringing that sort of thinking to culture, politics, to some extent, policy with EAC. And then on the computing side, it's like, hey, actually, you're no longer going to program the computer yourself.

12:20It's going to sort of do gradient descent by itself, get it self-organized, and you're just going to inject data and like tune how plastic it is. And that's about it. And then the rest of you, you're giving up control to some extent, right? And so that's a different mindset and a sort of new school of thought that we're trying to you know change science change technology or tech tree and and change our culture you mentioned early on that this is all going to take a long time and it seems like even just um you know in your lifespan this is there's a there's a through line you've you've kind of been seems like you've been kind of doing some form of this mission your whole life or for most of your life and you know if something is going to take a really long time, it feels like there needs to be a very long-term movement around it.

13:09And I wonder, you know, how much of, let's say, EAC was intentional, you know, which is now sort of, you know, I don't know if you want to call it a movement, I don't know what you refer to it. How much of that was intentional along those lines versus is maybe somewhat accidental. I mean, EAC is both sort of observation of these principles, right, that govern nature, but also govern our society. And it's a sort of awakening that actually, because of these principles, we can predict that, you know, the whole system is biased towards growth. It is fundamental to its good. It's fundamental to its adaptivity.

13:50It's fundamental to its robustness. And if we value life, intelligence and prosperity, the path to that is through growth. It is through constant adaptation. It is to constant sort of morphing, embracing variance in every aspect of our lives, whether it's culture, technologies, ways to organize ourselves, ways to live. In a sense, you could think of every aspect of civilization having a sort of evolutionary pressure upon it because ways to organize ourselves that are advantages towards growth get selected for because they're part of this bigger organism, right? Like, for example, corporate cultures is an example, right?

14:45If you have several companies that have different corporate cultures, one company will have a culture that confers it an advantage and then they dominate in the market and then other people will want to mimic that sort of corporate culture and then it starts spreading. Right. And so we've seen a lot of that happened with Google. Exactly. Right. And now, you know, who are the iconic cultures? Maybe it's going to be, I don't know, X or OpenAI or we'll see. Yeah. But, you know, it's this observation. And in a sense, it's like that adaptivity, right? Maintaining that malleability, right? That flexibility, that constant disruption is how we remain anti-fragile to changing landscapes, right?

15:28If we have technology that's evolving very fast, one way to think about is having a very stiff system and you have technology changing very fast, you're going to have a collapse, right? Or you're going to have your complex system failing massively, right? Which is, then it's like slow things down, slow down progress, right? Because our slow systems can't adapt. The other way of thinking is actually if all our systems are malleable and we progressively integrate this disruption on a continuous time fashion, then actually the system is plastic and We can adapt and we can avoid a collapse, right?

16:00And so for us, it was like, I think there's a whole spectrum of accelerationism movements. Some of them are like cheering for things to accelerate and for the system to be not able to adapt. We're the opposite. We're cheering for the system to adapt faster, to keep up with the natural pace of things always changing. And this change is the fundamental thing. It is the fundamental driver behind life. It is the fundamental driver behind progress of our civilization. We should embrace it. We should celebrate it. And we should lean into it. But then really, it's not a prescription of how exactly to live your life, right?

16:36It's more of a meta prescription. It's like, let's maintain freedoms. Let's maintain, always be experimenting, always be learning, always be iterating. Never, you know, say this is the one way things are done at a global scale or whatnot. We just, we want to maintain variance because you never know. You could always be hedging your bets across everything because things change over time and you always got to be adapting. How much of that was intentional? This movement? Did you say, hey, we want or did you and your founders of the X say, hey, we want to start this movement? Or is this more of just a way of thinking you all were doing?

17:11And then between that and, you know, the Twitter account, which we'll talk about, it took on a life of its own. Originally, we were we're just having conversations about what what does it all mean? Right. And I think it was dark times, you know, in 2021, you know, everybody was locked down, you know, the vibe was off, if you will. And we were trying to see, like, we were seeing, hey, there's this spread of sort of negativity and people are so negative right now. And that tends to be very high fitness in terms of memes, like on the Twitter sphere and whatnot. And we were like, how do we fight this?

17:44Like, there's so much to be optimistic about. and optimism is actually important because if you're optimistic, you start thinking about better futures and then your brain figures out how to close the gap between the world and your generative model of it, which is a whole theory of AI called active inference, which I'm a big fan of. That's going to be a whole different conversation, but there's a sort of real science behind this concept of sort of what we call hyperstition, of sort of thinking positively, like helps you amplify the likelihood of those positive futures. And this is a heuristic we know very well in Silicon Valley, right?

18:21Like you need an optimistic founder that's ready to run through walls and doesn't take no for an answer and just keeps going no matter what. And to us, it was like, how do we scale that sort of mindset so that it has positive impact on the world? And we kind of have it as a cultural mimetic product that can spread. There's no real product. It's just an idea, right? And in a sense, like, again, we were thinking of like, what is the fastest timescale adaptation, right? You know, you have the adaptation of technology, you have an adaptation of our bodies and genetics on a really long timescale. But the thing that moves really fast is memes, right?

18:56Which are like packets of cultural software. Those can spread really fast on the internet, and they can do a lot of good very quickly. And so we felt like a sort of imperative to spread positivity, optimism, and sort of a sort of signal like, where is this all going? Right? Because people will stoke fear, people will tell you, this and that is going to happen, give me more control, give me the keys to the kingdom, I'll keep you safe. You know, we think that is a sort of dangerous vulnerability in many people, and want to make people immunized against that by spreading, hey, it's going to be all right, the system's always adapting, this is how it's worked in the past.

19:35That's how it works in general. As long as we stay flexible, many people are working on this. We're going to adapt to whatever is to come. And so we're not saying this is exactly how the future is going to play out. It's like statistically, this is how things work. This is actually a very powerful principle that has creating very advanced things, including our human bodies, right? And a lot of our tech tree, right? The markets are self-organizing, our bodies are self-organizing. You can have a sort of faith in it, faith in the laws of physics, if you will, right? That the system will adapt as long as we don't shoot ourselves in the foot.

20:10This also applies to policy, right? We want our policymaking to be much more agile. And so we've been pushing towards sort of freedom and less regulation because there's obviously a lot of interest towards more regulation. Of course, the optimum is somewhere in the middle, right? But But because politics is often bimodal, right, you have to make a solid case for one extreme to balance the other, right? I meant to ask you that, actually. Like, do you feel like it should be a balance? Or is EAC all for total complete freedom, zero regulation? You know, I know there's this bill that this California senator recently proposed, which just seems like it would be really, really damaging to startups.

20:55I'm sure you know the one I'm talking about. Really, really, really good for big tech, right? Yeah, and that's what we said was going to happen for two years, right? We were like, guys, they're going to use this AI safety talk to shove bills down our throats for your own good, but that are really for their own good, right? Yeah, exactly. And so it's like, if there was a piece of regulation that was really clever, you know, we'd be in support of it. But for now, I haven't seen such proposals. I would say that some regulation can make sense, but I just don't think any heavy-handed regulations, any very stiff priors over just how things should be done, your model should be in this parameter region, and so on.

21:39It's very prescriptive. So you see you're restricting our search over the space of this technology. And things are changing so quickly. Things are changing so quickly, right? And so if you don't know where things should be, you should have a very loose grip on things. And so you should have very sort of high level policies. What about the Doomer fear? They have a lot of fears. They have all sorts of scenarios. Maybe like the most overarching around, you know, natural selection and, you know, AI is sort of out evolving human beings once there's a, you know, a stable state of AGI. There's a couple, you know, aspects that people are worried about.

22:20There's one that is, you know, this fable of a foom that like AI will be self-improving and there will be an intelligence explosion overnight and it will recursively self-improve. I fundamentally don't believe in that. I mean, I worked on AI for designing matter for many years now. It's much harder than people think. And there's a sort of complexity to nature that is inherent. It's inherent breaks on any sort of, you know, hacking that you could try to figure out. Right. And in a sense, the proof of this is that biology is a self-adaptive system running an algorithm very similar to our optimization algorithms.

22:59And it's been running this process for 2 billion years, trying to have a system that grows and acquires more energy and consumes it all. Every biological system is trying to do that. It's incentivized to do that from the laws of thermodynamics. right? And yet through this sort of adversarial equilibrium we've reached and the fact that energy isn't just like infinite and easy to access at all times, there's actually a finite amount of life. We haven't consumed our whole planet yet. And maintaining this sort of adversarial equilibrium is what sort of keeps us in check. And so I don't believe in FOOM on that end for fundamental reasons of complexity based on physics.

23:41And then when it comes to, let's say, maybe a more reasonable concern is like, AI is going to take all our jobs, right? Or that's like a traditional, you know, and... Yeah, that's the near term fear. I think that people aren't realizing we're becoming transhuman already. We have our phones basically glued to our, you know, pockets every day, right? And it's an extension of our cognition, right? We have shared priors. We have shared information retrieval ways. Nowadays, you just have to remember a way to do a Google or perplexity search to retrieve the information you need. You don't need to memorize things.

24:17And that's how our brains work. We're already sort of augmented. And I think as we have sort of AI more and more embedded in our lives, we have AI assistants that are going to see everything we see. They're going to shadow us. They're going to have a prior of what we've seen, what is the context of our lives, what we like to do given context. and then they'll have a predictive model of that and you'll interact with it. Just like you interact with an assistant or a teammate that you've worked with for many years, you share a common prior, almost like one system. You have sort of group flow and you're kind of an extension of one another's intelligence.

24:47That's already happening. And it's a sort of soft merge where we're extending our intelligence and providing massive intellectual leverage to ourselves. So that's for sort of white collar work. For blue collar work, we're gonna have robotics, right? And body design. And that's gonna give us operational leverage. And frankly, I think we could use a lot of operational leverage because there's a ton of projects in the physical world that could massively speed up. Right. We can't even build railways anymore. We don't have the labor and know how. So we need to accelerate there. And that's going to provide massive prosperity and make our lives much better.

25:21Right. Like when you think about it, scarcity has been the source of pain of modern times. right scarcity of say uh doctors and and intellectual work and and biotech and so on is is why medicine and treatments are very expensive right like it takes a lot of money to do uh drug drug discovery research it takes a lot of money to train a doctor that's going to get cheaper takes a lot of money to build housing that's going to get cheaper with robotics takes a lot of money to build infrastructure that's going to get cheaper so a lot of things that are sources of pain in modern times are actually from sort of deceleration, inflation, overregulation.

26:03And so the other thesis is that actually letting the markets do their thing is actually the most moral thing to do because it's going to lead to more prosperity and better lives. And ultimately, I would say in terms of selection pressure, people like to talk about this. EAC and all this sort of thermodynamic talk we like to do is really a theory of how everything is actually being selected for, including products and technologies, right? If you think about it, there's currently a selective pressure on the space of LLMs, right? Because you have different LLMs, they have different hyperparameters, different ways of doing things, slightly different ways, and they're all offering competing APIs, right?