Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models

0:00Nathan: Hello, welcome back to the Cognitive Revolution, and happy 4th of July to everyone in the United States. Today, my guest is Ramin Hasani, CEO of Liquid AI, a company founded by MIT researchers that's developing device-native foundation models. I'll say up front that just before recording, I encouraged Ramin to go deep into the weeds on the technical details of Liquid AI's work. And as you'll hear, he did a truly excellent job, demonstrating a mix of technical sophistication, differentiated vision, and a contagious passion that, in my humble opinion, makes this episode an instant classic. We start with an overview of the team's research into tiny, biologically inspired, differential equation-based neural networks that Ramin and team developed at MIT and which inspired them to start the company.

0:51Nathan: Some of the capabilities they demonstrated, such as parking a car with a control module that consisted of just 12 liquid neurons, still sound a bit like science fiction today. And while those systems haven't scaled up to today's capabilities frontier, the company has maintained the liquid philosophy, which today means taking a neutral, empirical approach to designing and optimizing neural networks to perform under all sorts of exotic constraints, including most commonly the need to run on edge devices with limited memory and processing power. Considering the fact that the global smartphone and laptop market is roughly$800 billion per year, a number which the global AI data center build-out is only now surpassing, that demand for inference threatens to price much of the world out of the frontier model market, and that so many enterprises and individuals value privacy and the ability to control their information, this is an absolutely massive market opportunity unto itself.

1:50Nathan: And Liquid has serious proof points, including holding the number five spot on the hugging face United States Downloads leaderboard, plus notable partnerships with companies such as Shopify and Mercedes-Benz. Anyone who doubts can do a quick download and demo of Liquid's Apollo app, which shows, in my experience, that even a one-billion-parameter model, which combines a small number of attention layers with a very simple gated learned convolution, while admittedly far from the frontier, can run fast enough on an iPhone to be a real option for basic use cases, such as privately searching through and classifying one's own local documents.

2:27Nathan: Perhaps most interesting is the network architecture search process that Liquid uses to develop networks for particular use cases and runtime environments. Having found that proxy metrics too often lead the process astray, they now evaluate models on real downstream tasks, on the actual target hardware that their customers intend to use. Ramin shares a lot more detail on their findings. But in short, while attention-based architectures continue to generalize better than any known alternative and therefore continue to dominate the frontier, the more specific your use case and the more limited the compute resources you have available, the more likely their search process is to land on an exotic architecture.

3:09Nathan: This, for now, is where architectures like Mamba and other subquadratic innovations really shine. Toward the end, Ramin teases a platform that Liquid will soon be introducing to allow customers to fine-tune small models for their own use cases on a self-serve basis. For multiple reasons, including the potential to ease demand for frontier models and improve access to AI globally, I, for one, will be very excited to see that come online. And so, without further ado, I hope you enjoy this high-energy look at how Liquid AI is squeezing as much intelligence as possible out of any given computational resource.

3:48Nathan: With co-founder and CEO, Ramin Hassani. The Cognitive Revolution is brought to you by Mercury, the fintech that more than 300 ,000 ambitious companies and individuals trust to run their finances. I've wired AI into nearly every corner of my life. My email, my messages, my calendar. I even gave Mercury virtual cards to my agents, with low limits and category and merchant restrictions, for their autonomous use. But still, my AI's access to my financial data has remained limited. With a normal bank, I might export a bunch of statements and have my assistant process them for me. But for real-time, up-to-date information, and certainly for taking any action, Trying to get your agent to use the bank via the browser is just too hard, too slow, and too error-prone to be worth it.

4:38Nathan: And that's why Mercury's new conversational interface, Command, is such a big deal. It's built directly into Mercury, which means you get natural language access to your finances without exposing anything outside of your bank account. No exports, no spreadsheets, no pasting your transactions into third-party tools. I really think a lot of people are going to prefer it this way. And it can already help you take actions too, with everything bound by the permissions and approval policies that you've already set up in your account. I am genuinely impressed to see this level of AI integration in banking in 2026.

5:16Nathan: And so I invite you to join me in the future. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column NA, members FDIC. Thank you to Mercury for supporting the Cognitive Revolution. And now, on with the show. Ramin Hassani, co-founder and CEO at Liquid AI, welcome to the Cognitive Revolution. Thanks for having me. I'm excited for this conversation. I've been following Liquid AI from afar for a number of years. and fascinated by some of the architectures that you guys have developed going back to your time at MIT together.

6:03Nathan: Also fascinated by the trajectory that I perceived the company to have taken over the last few years as it's become more customer-focused and commercial entity. So maybe for starters, how would you tell the kind of broad story of Liquid AI leading up to what you're doing today and what the company's mission is today? Yeah, absolutely. So Liquid AI, three and a half years ago, we spawned out of MIT, C-Cell, building on a technology that we have actually have been working on it like 10 years before, like basically like a decade before when we started the whole company. we've been, our objective function at MIT has always been maximizing the amount of intelligence that we can into smallest format of algorithms.

6:49So efficiency has been the cornerstone of our research. And we have been working on specifically thinking about robotics and systems that were coming into the real world. The idea of liquid neural networks, that was kind of a discovery that's my PhD thesis and together with my co-founders, the four co-founders that I have, we've been researching ideas about how can we build machine learning solutions that can go on robots and doesn't have like millions of billions of parameters so that we can actually host them directly on, let's say, CPUs, NPUs, or let's say like smaller GPUs that are mounted on top of like physical systems while delivering the reliability of much larger kind of instances of artificial intelligent systems.

7:34Essentially, what we try to do, we try to, you know, like build alternative algorithms in order to get creative in the algorithmic space to see how can we build machine learning systems that can generalize beyond the data that they have seen. Because when you go in the real world, the distribution shifts becomes like a real thing. You know, like you imagine you deploy a robot in an open world kind of, like let's say an autonomous car, a flying drone, a fixed-wing vehicle, you know. So all of these systems are going to go and very, very rapidly, you're going to get out of distribution. So you have to actually build systems that are really comfortable around being able to get out of distribution.

8:09You know, humans are extremely good at it and humans do that extremely, I mean, natural learning systems, animals also like the same way, right? So they actually follow a really nice trajectory out of distribution as well, you know? So that means like the learned concepts can generalize to data that you have not seen before, right? So our idea was that like, can we build systems that can have that kind of properties, you know, better out of distribution generalization. This has been not just our goal. I think an entire artificial intelligence field has been like working towards this thing. And especially like in the robotics and closed loop environment, you know, like closed loop environment where you have an agent acting in an environment, you know, you want to have that kind of properties.

8:52So naturally the place that we started looking into was brains. You know, we started looking into animal brains, you know, and we started looking into how one thing that I really liked, I wanted to look at it from a first principle kind of approach, like how neurons exchange information with each other. You know, we started looking into brain of worms, you know, small animals and why worms, this specific worm C elegans, you started working on the brain of the worm. The reason behind it is that it was the only animal that in 2015, when I started this type of research as part of my PhD with my co-founder, Mathias Lechner, this was the only animal that we knew the entire nervous system as a whole.

9:30And this animal exhibits massive amount of sensory reactive kind of behavior, like amazing levels of control with 300 cells in its nervous system. And that's what's fascinating for us because this is much smaller than any neural network that performs control at that time on, let's say, autonomous systems, you know, and this worm can do better, you know, dexterous movements and stuff like better than the best robotic systems that we actually had in the world, you know. So we thought that, okay, so let's start understanding how neurons exchange information in the brain of the worm. From there on, let's start building nervous system more complex and complex kind of neural circuits so that we can get to the stage where we can, let's say, like build the next animal, build the next, you know, like, basically follow the path of evolution of nervous systems, you know, and see how we can rise and evolve as part of this thing.

10:27So there are equations that describe the neuronal dynamics of, let's say, like two neurons. In the brain of C. elegans, because the worm is very small, the neurons do not spike. They are graded kind of neurons. Like they behave like in an electrotonic way, you know. So they're very similar to artificial neural networks that we have, because they're also very differentiable, right? Because you don't have spikes in the activity of neurons, they're very differentiable. That's why, like, it was even nicer for us, more attractive for us, because we could apply learning theory to these type of neural networks once constructed, like, let's say, two neurons, four neurons, eight neurons, like, let's say, 100 neurons next to each other, and then you start, like, training them.

11:08You can apply backpropagation, like, as a differential kind of programming on top of a system that is built by these type of inspirations that we got from nature. The type of differential equations that were there were also very well behaved, you know, you can make them as complicated as you can, you know, and then when they get more complicated, they mimic their biology much more. And, you know, of course, like from a computational footprint, they become more complex to scale, you know, but it's still like you can sacrifice, where you can actually tune how much complexity you want to encode inside a neural architecture.

11:42You know, in artificial neural networks, like, we try to abstract away, like, those complex differential equations that describes the neural dynamics between two cells, and we just show them with a sigmoidal function, gated sigmoidal functions, and now we have, like, matrix multiplications that are coming, like, we're capturing the impact of, like, the inputs that are coming to a system with a transformer-based architecture and attention mechanism and all of those things. But these are all simplified kind of computations for us to be able to scale these machine learning solutions. For us, from a neuroscience perspective, it was very interesting for us to explore to see if we actually start making, like bringing back those differential equation-based computation and a little bit more elaborate form of computation into the behavior of every single neuron and mimic the behavior of like how two neurons exchange information with each other, maybe we can unlock something like greater than what we have seen from an artificial neural networks perspective.

12:38So, I mean, early on, the results were fascinating. Like we saw that with 12 neurons, with actually 12 neurons, you could parallel park autonomously like a car, you know, like a small car. With 19 neurons, you could drive a car. With 30 neurons, you can fly autonomously, like navigating kind of a drone, you know. So, and you can get sensory information and process this information with this a little bit more complex and elaborate version of this neural dynamics, you know, which we called liquid neural networks and liquid for adaptability. I called it liquid time constant neural networks, like that was the LTC kind of paper that we got out.

13:16And we coined the name liquid for the fact that the dynamics of this distance are staying flexible even after training, you know. So the models would be able to react to new type of inputs that they receive and even learn during backpropagation how to be adaptive towards the inputs that you're receiving, you know. So it actually encoded a little bit more degrees of freedom on flexibility of the learning dynamics of a neural network compared to artificial neural networks, compared to other systems that you've seen. That form of dynamics actually allowed us to scale this technology into not just, you know, not just in robotics, but also like applying it to predictive AI in financial services, in medical domain, and in many, many different places that we applied in, like, let's say, audio modeling, you know, multimodal video understanding, you know, like we applied this technology over and over, and we saw promise that this technology is actually very, very amazing.

14:11Like, you know, every neuron is much more complex than a normal kind of neural network. But at the same time, like, you don't need that many points of computation in order to actually get to the results that you want. We also saw that the out-of-distribution performance of these models are extremely good, and they were suited for robotics applications. So, I mean, we had interactions with the United States, of course. I mean, at that time, Boeing was actually hosting my postdoc at MIT. And there we showed that you can actually fly even jets, like with these type of neural networks, with like a handful of these neurons, which was a very interesting kind of topic to see how far you can push, like, let's say, out of distribution generalization with a handful of, like, in a very, very small set of system, you know, a system that is, like, powered by these liquid neural networks as opposed to artificial neural networks.

15:02Then, so as I told you, that each node in a liquid neural network is very complex. You know, like, it's a differential equation. You have to solve that, and you have to solve a set of different – the more neurons you have, the more complex is a forward pass and a backdoor class through a network. Therefore, we needed to think about efficiencies. These systems are highly nonlinear. We try not to sacrifice on nonlinearity because adding nonlinearity to learning systems allows you to build more expressive systems. This is a proof that actually I have in my PhD thesis where I showed nonlinearity actually directly contributes to, especially on the smaller size of models.

15:38But then the idea here for us was that how can we scale these systems? You have nonlinear systems, but it is extremely difficult to actually scale non-linear systems. So you have to solve. The reason behind scalability is the computational complexity of the models are hitting like even cubic complexity. You know, it's not even quadratic. You're talking about cubic complexity, which is like too much. And then you have like, you usually have, like when you have a set of differential equations, you can roll them out with a numerical solver. You run the numerical solver and you can step-by-step compute kind of the outputs that are desired.

16:11And then, you know, like the more steps you want to use in order to actually compute, then you would get more and more accurate kind of results at the end. Right. But then the problem becomes the scalability again. You know, like you cannot really get to infinite precision with these numerical solvers. One idea that came to us was that what if we just solve the whole system in closed form? You know, like literally like just you have a differential equation system with these liquid neural networks, each of them representing neural dynamics of, let's say, two cells that are exchanging information with each other at a certain level of abstraction.

16:44And we saw that, OK, let's take this system and try to solve it in closed form. Turns out the closed form solution for this type of equation hasn't existed since 1907. You know, so 1907, there was a scientist called Louis Lapique that actually modeled the membrane potential, like how to model mathematically membrane potential kind of in cells. And that format of equation became like a fundamental of channel modeling, like how information propagates through ion channels inside a cell and then how neurotransmitters are getting propagated to the other one. So 1907, this was Louis Lapique's kind of membrane potential equation that is like an open differential equation.

17:30And then we've seen like some scientists called Huxley and Huxley. They started working on really biological kind of grounding this type of differential equation and adding a little bit more complexity into the model. They developed a neuroscience model of one neuron, how actually a neuron reacts to, let's say, a membrane potential. So 1953, they started. 1963, they won a Nobel Prize for the fact that they actually developed a better and a more accurate representation for a neural dynamics, right? And then from there on, so that in every textbook that you read, these type of formats of equations, they do not have a known closed form solution, you know?

18:11And this format, like liquid neural networks, were also part of that type of equations. We, for the first time, we actually solved that. And this was like 2022. Around 2022, we solved the liquid neural network kind of interaction of neurons with each other in closed form for the first time. This became a Nature Machine Intelligence paper published in November of 2022. This is called the closed form continuous time systems. You know, like this is liquid neural networks in closed form. And the closed form, I mean, it has massive implications. Why? Because now I don't need to use any numerical solvers to actually run a liquid neural networks.

18:46I can now have not only hundreds of neurons, but now I can have billions of neurons next to each other. And I can actually scale this competition. Still keeping the non-linearity as part of this thing. Then in January of, actually beginning of February of 2023, an article came out of a quantum magazine. you know, about, let's say, me and my co-founder, Matthias, we, it was about a profile about, like, all the implications of having some closed-form solution, finally, on neural dynamics, you know, like, and how important this can be for both machine learning and also, like, for brain science as a whole.

19:26And then, so my inbox was full of VCs, you know, and Silicon Valley started, like, talk, like, everybody wanted to throw it, like, a term sheet of us, like, to really get started and really scale this technology, because this was fundamentally different than a transformer-based architecture and attention-based architecture. This was grounded in biology, like the type of math that we have. Around the same times, we have alternative models are coming out, like states-based models, like you've seen faster iterations of states-based models, like convolutional neural networks that came out to be scalable, all of them in a linear form because the scalability of alternative models, you have to linearize them and then you can scale them, you know, and we have something here that is also another category, you know, like it adds some operators that we learn from biology and from physics as we grew the research.

20:14And we got to the point where we can now become very competitive and, you know, like building models, like for solving more and more general purpose tasks. And when I say more and more general purpose tasks, I'm talking about modeling signals beyond predictive performance, modeling signals for language, for audio, and for vision in a way that humans understand, right? That's the complex format that humans understand. So the mission of Liquid AI became building efficient general purpose AI systems at every scale. And we coined the word efficient in our mission, and we care about it so much because I think we are, from a foundation model company's perspective, we are probably the most efficient foundation model company on the planet.

20:58I'll tell you why about this in a minute. But the reason why, you know, like we set the mission into efficient machine learning, because we saw that how much of the considerations that we had to do in order to really build a closed-form solution, then taking this closed-form solution and scale it to, let's say, a system that can model language, a system that can model audio models, a system that can model video and vision in general, you know, how much considerations we had to go into these things so that we can computation, in a competition, attractable way, scale our neural architectures to the point where we are today.

21:37Today, the base of our technology, I'll talk about the core technology of the company later on, but I will tell you today the technology became liquid foundation models. Liquid foundation models or elephants are pretty popular right now. We are ranked number five actually in the US from the number of downloads. From the popularity perspective, we have over one million downloads per week on the hugging face. Like that's a lot of downloads of these small models that we're building. They're very popular. Like the top ones from the US side are Google, Meta, Microsoft, and NVIDIA. And the fifth one is Liquidei.

22:09So from the US side. So, I mean, we got ourselves there by consuming about 1000 GPUs, having 1000 GPUs in-house. So that's like from a foundation model perspective, like getting to that level of popularity and releasing more than 50 models in differentiation of models that people are using in enterprises, that's kind of the place where we brought liquid, building on top of inspirations that we got from nature and the past that I actually like portraying for you.

22:35Nathan: Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic, makers of Claude and Claude Code. Over the last few months, Claude has helped me build and refine a personal deep context database that now contains all of my emails, Slack messages, tweets, DMs across platforms, video calls, and podcast transcripts going back a full five years. On top of that, we've now layered summary articles describing my relationship with hundreds of contacts, organizations, and ideas. And now that this exists, there's almost nothing that Claude can't help with.

23:14Nathan: For my angel investing, Claude can now draft investment memos in exactly the form that my venture fund requires, based on the calls I've had and the emails I've exchanged with the founders. And when someone needs a favor, Claude can often do it as well as I can. Recently, a friend reached out to ask if I know anyone who might be a fit for a role that he is currently hiring for. Initially, nobody came to mind, but then I thought to ask Claude, and sure enough, it identified two great leads. Claude is the AI for minds that don't stop at good enough. It's the collaborator that actually understands your entire workflow and thinks with you.

23:51Nathan: So, for problems worth solving, Get started with Claude at claude.ai slash TCR. That's claude.ai slash TCR. And check out Claude Pro, which includes all of the features mentioned in today's episode. That's claude.ai slash TCR. Brilliant. So many different directions I want to go and follow up there. Maybe for starters, one thing that just on a very fundamental level jumps out at me is When you describe the liquid neural networks, you describe them in terms of the number of neurons, as we're used to hearing about the number of parameters. And something tells me there's like a kind of paradigmatic difference there that underlies the difference in description.

24:38Nathan: So, yeah, help me develop my intuition a little bit more for, I guess it probably connects to non-linearity. And this is so fundamental, right? Because there's so much incredible progress that has been made. But on things like really robust out-of-domain generalization, we've still got a lot of work to do. On things like adversarial robustness, we still have a lot of work to do in the mainline paradigm. So, yeah, tell me, unpack a little bit. Why are we counting neurons versus parameters, and what does that tell us? That's a great question. So I think early on, we actually started talking about the number of neurons as the unit of computation, like for in our research, because that was like very interesting.

25:20Like you can associate, it's a, you know, like every neuron is a process that can be modeled by a mathematical equation, you know. Now you can allocate basically like in a sigmoidal kind of from Gates or the more functions you have, the more parameters getting added to that single cell, you know. And the cell that now in a liquid neural network in terms of parameters, you can say multiply by seven, you know, and that would be the number of parameters often. You know, so that in that differential equation, but that cell itself, because of the equations that it has, it computes more than just, you know, like a forward pass competition.

25:54It has like internal feedbacks. That's three degree feedback kind of mechanisms inside. All those parameters are actually like one-to-one analogous to any artificial neural network system that you would build in terms of parameters. So a rule of thumb for when I say number of neurons, you multiply it by seven, you would actually get the number of parameters of the system. But this is for early on version of liquid neural networks. You know, as we started the standardizing parallelization schemes for the type of mathematical functions that we want to explore and let's say inspirations that you're getting, we just converge to the number of parameters.

26:31Like, of course, like today, when we when we are talking about a liquid foundation model that has, let's say, one billion parameters, we are literally meaning one billion parameters in the sense of like the any other kind of GPT that is getting built out there. Does that make sense?

26:46Nathan: Yeah, but let's zoom in maybe a little bit more on that neuron. And I guess I'm wondering a little bit too, what is the fundamental limit or what are the first bottlenecks that you hit when you try to scale the original liquid approach? With the sigmoid type functions that we have today, those are, I think, certainly carefully selected to be easily run on GPUs. I'm guessing that these sort of multiple degrees of freedom inside a single neuron maybe present challenges in terms of executing that process on available hardware. Like, we can get these amazing results from tens of neurons. So the obvious bitter-pilled question would be, what if we take that exact paradigm and go to millions, billions?

27:37Nathan: But you must be hitting some bottlenecks along the way. What are they? So the main challenge is turning sequential computation into parallel computation. So when you have degree, like when you start like going from a single neuron dynamics to multiple neural dynamics, weight parameters, like parameters of your system, instead of being scalars or vectors, they become matrices and tensors. So now you're talking about matrix multiplication. that you want to turn the scalar computations into tensor computations, right? You want to be able to, this is how you parallelize kind of, let's say, sequential computers.

28:19The problem is that not all the time, like if you have nonlinear relationships in an equation, you cannot trivially create a one-to-one map between a vectorized computation and a tensorized computation. So those nonlinear kind of nonlinear attributes of liquid neural networks and any other nonlinear recurrent neural network, I mean, recurrence itself is adds like some degrees of complexity into the math of the whole equation. I mean, let alone if the recurrence has a nonlinearity on it, it becomes like a lot more complex to disentangle vectors or write them with typical linear algebra kind of methods that we know into tensors.

29:01and you have to be able to turn these competitions into tensors to be able to compute in parallel you know like you compute them once at a time you see so that's that's kind of the the mathematical it's not related only to liquid models any mathematical operation that you want to tensorize if it is non-linear you're going to have the troubles that we're talking about that's why the state space models that actually came out these are all about around linear dynamics you know, we are talking about like linear dynamical systems, right? And why those linear dynamical systems are important is just pure fact that we do not have proper ways to scale nonlinear systems.

29:41You can apply nonlinearity to a linear dynamical systems. For example, in a, let's say like a gated, or let's say, you're like, you can add a sigmoidal function after you perform the dynamical system, like in a linear way, you know, like you compute the, you compute all the matrices and everything like in parallel, but then you can do a point-wise kind of application of a nonlinear operation on top of the entire tensor. That's what you can do. But what if the relationship between the parameters themselves are governed by some non-linearity? So you cannot really use typical linear algebra. You got to always approximate that nonlinear system into a linear system, then you would be able to actually parallelize the systems, right?

30:25Does that make So that's the fundamental bottleneck.

30:28Nathan: So how much does the closed form solution address that? And how far has the, with available computing resources, how far has the original liquid network paradigm been able to scale so far to present? Fantastic question. So one of the properties of liquid neural networks was the fact that they have multiple feedback mechanisms. You know, they're not just one format of feedback. They have like three layers of feedback between two cells, in the synaptic dynamic itself, because we wanted to mimic how it is done in the brains, right? So it has multiple degrees of feedback. It has then basically nested nonlinearities on top of each other.

31:17That nested nonlinearity itself would also add degrees of complexity. Even the closed-form solution version of a liquid neural network, from a dynamics point of view, you can now, instead of 100 neurons, you can have 100 ,000s of neurons, maybe 1 million to 10 million neurons. But the nonlinearity, you still have to compute the model's sequentially because of these nested nonlinear relationships that actually happens in the circuit. So even the closed form kind of solution would be limited in scalability in the sense that you cannot parallelize them very effectively. But there are also like people are people are also like working on these topics a lot, like parallelizing, but not just parallelizing, but the speeding up sequential computations.

32:03So we're talking about sequential scan, you know, like scan is an operator that you can run on top of like, let's say, like doing matrix multiplication instead of doing it in cubic or let's say quadratic. you can do it in subquadratic kind of time, right? So you can actually reduce the complexity of computations even on nonlinear operators without paralyzing them, without linearizing them, you know? So there are ongoing research going on, like we are doing like some of that kind of research ourselves to really find ways to really sequentially run those nonlinear systems because clearly there is an advantage to have like those form of nonlinearities when you do function approximation, you know?

32:41We are talking about also scaling laws becomes also very important here. I believe scaling laws define architecture. How does it do that? When you're talking about transformers being this revolutionary thing, we are talking about maximum scale. The reason why transformer architecture and attention mechanism is such a brilliant architecture is the fact that it is unstructured. There is no structure. When I'm talking about nested nonlinearities and all this crap that we have like in liquid neural networks, you don't have that in transformers, right? You have basically just matrix multiplication as the core functional kind of things that represent.

33:22And it is on a structure. You can literally multiply any matrix into any matrix of any size. So the whole idea here is this. The larger neural networks that you make into infinite size, the larger neural networks you make, the more you want them to become less and less structured. And we've seen the success of transformers at the trillions of parameters. Now we are talking about tens of trillions of parameters. I mean, the next generation of models that are going to come, we're talking about trillions of parameters. And you can do that with a transformer architecture. As soon as you start adding a little bit of bias in that architecture at scale, things become completely messed up.

34:02So we are talking about right now liquid neural networks, these alternative architectures that we are talking about. They have a scale to heat. You know, like there is a regime of parameters that you can just do better. You know, like let's say up to, let's say 100 billion parameters, up to a trillion parameter. You know, like that's kind of the range we are operating right now. In this range, the smaller is the model architecture, the more you want to specialize them for a certain application to solve. And they are actually, like mathematically, they are biased to actually solve a certain type of task better than other types of architectures.

34:39So I would say biases on algorithms, you know, the more kind of biases you put on it. And by biases, what I mean is this adding a lot more non-linearity, adding like multiple gating levels on top of a neural network, you know, adding like recurrence you know like multiple different types of recurrence adding let's say convolutions like as a i mean convolution itself if you just keep the convolution or kind of neural networks they're also pretty unstructured because you can apply convolutions on any size of neural networks you know so that that that question becomes like how much bias and what kind of problems you want to solve so that the bias in the neural neural architectures becomes kind of a function of i would say scale of the neural networks as well as like the the use cases you want to solve them all.

35:23You see, so the spectrum would be, the more you go to the larger size, the more unstructured kind of mathematical operators. Use pure matrix multiplication. Use pure convolutions. Use pure, let's say, sequential scan. Pure operators. You see, it's like you get much more like, and then forget about gates. Forget about like adding like, you know, like fancy like control, like forget gates and all those things. And at scale, this is something that we have seen, like, because we have scaled neural networks, and we have, like, a really good understanding of that architectural kind of variation at different scales.

36:00Nathan: Just for calibration, when you talk about the liquid neural networks that have gone up to hundreds of thousands or pushing a million neurons, what kind of hardware does that run on? Is that, like, a CPU supercomputer? You can do that on a CPU, like, on even simple CPUs or simple GPUs, because they don't have that much of, probably one of these liquid neural networks would fit on, I don't know, one to 25 megabytes of file size, basically. You can literally put them there, and then a CPU would be enough, a Raspberry Pi would be enough to perform computations. We have shown that on a lot of predictive, specialized applications of AI, that these systems can be very, very powerful and do stuff, and they can be as powerful as the systems.

36:45These closed-form variants could be as powerful as the open differential equation version of the neural networks in speech synthesis. You know, like that's one of the applications. In, as I said, predictive sequence, like imagine you have complex formats of sequences coming in, multivariate kind of sequences that are coming in and you want to perform some sort of a prediction on top of them. These neural networks are actually pretty good. And then if you're talking about like more, let's say, out of distribution generalization, to the extent that you're like within a bounded period, you cannot have like an open-ended continual learning.

37:18These are not, liquid normal methods are not continual learning systems. They are more adaptive formats of computations because of the many gatings, many feedbacks, and many, let's say, input-dependent parameters that they have in their mathematical kind of organization.

37:34Nathan: What is the definition of continual learning that you're using there that they don't satisfy? So continual learning would be a system that is continuously receiving new data and retunes itself. So it changes also the parameters of the system. Liquid neural networks are, the dynamics of the systems are input dependent. So that means new data comes in, the dynamics of the system would behave to those inputs, but the parameters of the systems, like any other neural networks is fixed. It's just because the format of every neuron or every node in a liquid neural network was like a differential equation, you would have a different dynamics.

38:11So dynamics, you know, it's another dimension that you add to number of parameters. It allows you to compress more information, compress kind of more knowledge. That's why with a smaller instance of the models, there's no free launch, there is no magic here, like you're doing like also with liquid knowledge. It's just that the access of dynamics has been something that we added to the neural networks for what? For like being more adaptable. So I'll give you a very tangible example here. Imagine you're driving and all of a sudden it starts raining, you know? So depending on the format of the rain, you know, your autonomous driving system that is actually taking control of the car could react to those kind of driving scenarios.

38:50And if you have like a traditional neural network that hasn't seen that kind of environment, it might actually get like biased, like really, because that rain that actually hits the camera, it's noise, you know, a certain type of noise, you know, or a certain type of adaptation of the input that is coming. But the reality hasn't changed. The confounding variables of the whole environment hasn't changed, right? So that's why liquid neural networks, because they react to the input completely differently, you know, like they absorb the input, they apply the low-pass filtering on top of the input, you know, they are much more adaptable to those kind of scenarios.

39:27That doesn't mean that they change the parameters of the system to be more adaptable because there are two axes here. One, changing the parameters of the system. The other one is changing dynamics of the system. I would consider continual learning to be attributed to where you continuously also change the parameters of the system. In liquid on networks, we don't do that. Gotcha.

39:48Nathan: Okay. Helpful. Okay. So let's fast forward to the present. You guys are now in the market working with customers. One notable conversation I heard with one of your customers was with the CTO of Shopify on the Latent Space podcast, who had some very good things to say about you and your technology. I was struck by the fact that you've gone neutral in a sense. Now, what sort of jumped out at me in terms of your approach is that you've developed an architecture search process where the promise to customers is not that, hey, we developed this one paradigm. And old calculus teacher used to say, when all you have is a hammer, everything looks like a nail.

40:33Nathan: So you're explicitly promising to people that even though we came from this lineage of this particular kind of network, we're not just going to blindly apply it to your problem. Instead, we've created this sort of higher abstraction or more meta process for searching through architecture space to find the thing that's going to work best for you. And the two notable details of that are, one, proxy metrics you found don't work that well. Just measuring perplexity or whatever, you found you needed to actually go farther and test models on the actual downstream tasks that they're going to be asked to perform.

41:13Nathan: Um, and second hardware in the loop, actual target hardware in the loop to test the architecture subject to the very real and physical constraints of the robot, the sensor, the phone, whatever it's going to be running on. Maybe I understand that the LFM model came out of that, but maybe before we even get to the LFM, could you sketch out the range of different types of problems that we're putting into this architecture search? And then maybe also a subject that implies, of course, what kind of hardware are we targeting? And then what kind of different architectures are winning for different kinds of problems under different kinds of constraints?

42:00Absolutely. So there's a system that we developed in-house, we call it automated foundation model design, AFMD. That's a mental learning system that puts the hardware in the loop and then tries out many different operators with an evolution strategy. The criteria is an evolution strategy, optimizing for a couple of things, optimizing for memory consumption on that device, optimizing for latency, optimizing for speed, while no sacrifice on quality. When we talk about quality, perplexity is not the measure. It's actually the application, the downstream applications that we care about. You know, it's not just also public benchmarks.

42:38We're talking about 100 different benchmarks. So the problem space becomes, from a meta-learning perspective, becomes like a very, very complex kind of problem. Now, I'll tell you that, why did we take this approach, you know, like to design an architecture? Why? Because we wanted to remove all the human biases early on as we are actually like building architectures. One of the things that we realized like culturally at companies, Like, this is what I can tell you. Like, even at largest foundation model labs in the U.S. right now, in Tropic and OpenAI, there are a bunch of people, like people that are coming from the science.

43:09Like, I call them the Avengers of the architectures or Avengers of the, you know, like post-training or let's say pre-training. These groups of people, there's usually like a very small set of people that are calling the shots on like, oh, you know what? You're going to tweak this portion of this architecture so that it performs better. Why? Because in my personal experiences, it has started working better. If you're really, truly thinking, and this is something that is broken in all the foundation model labs. You cannot say that like somebody has a fix to this, but now the recursive self-improvement kind of process is actually fixing for that.

43:43Because now people are just finally realizing you've got to give it to the algorithms. You know, you have to be better lessons. You have to be better lessons, people. So you've got to be giving it to a systematic way to actually find out what is the true architecture for the problems that you want to solve. You can build like a general purpose computer. The insights that I shared with you in the format of the scaling laws of neural networks is coming out of our massive exploration of the space of architectures. You know, the fact that in the smaller kind of category of models, some biases on the architecture help.

44:19In the larger instances of the models, you don't need to bias the systems. You can actually go pure convolutions, pure transformers, and you can go pure, let's say, recurrences that are like very simplified. You don't need to add any form of kind of, you know, specialized kind of treatment like gating there, you know, gated delta nets, you know. And then there's the mambas and there's the jambas and there's like there's a lot of different variations of these architectures that are coming out. But what you want to do, though, you want to be completely unbiased. Even us ourselves, you know, day one, like when we actually started Liquid, we had one of the inventors of SSMs actually as our founding scientist, Jimmy Smith.

44:58You know, he invented S5, you know, like the simple. He basically brought the parallel scan idea to the SSM world, you know. Then we had Stefano Massaroli and Michael Polley. They were actually designing like the, you know, like the hyena hierarchy. You know, there was like a convolutional kind of path that is going on. All of these are linear systems, you know, like within the realm of linear systems. And then we came out of like these nonlinear, like hyper control theory kind of enabled foundation models, which is liquid neural networks. A lot of big heads in the room. And you got to you got to call the shots around.

45:32Again, the same problem in all the labs. You know, like you got to you got to figure out. We thought that, OK, let's fundamentally solve this problem. Let's really like systematically, first principle, let's put all the operators of interest, whatever we think that could be an operator that can give rise to a general purpose computer, like a system that can, let's say a liquid neural network on its own is a general purpose computer. You can actually scale it. If you can, theoretically, you can get it. So you put all those kind of equations, all those kind of things in a unified kind of theory, you know, like with linear input varying kind of operators, you know, like operators, All the, let's say, space of operators can actually be encapsulated inside this linear input varying or, let's say, in general, liquid input varying kind of operators.

46:16Because input dependence was something that we have been talking about this for the last 10 years. That's extremely important. And we see that right now in transformers is also extremely important to have. That input dependence is actually coming naturally with the attention architecture as well. But in a very, very unstructured way, it's not as structured as, let's say, what is done in RNNs, in liquid neural networks, in SSMs, in all of those things. You don't see these gated variables to be very structured in transformers. Now, we actually have various variants of the convolution operators, various variants of a recurrent operator.

46:56And then the attention themselves. There's group query attention. There was the original transform. There's many different things. Then on some of these variations of dynamical systems, and also various types of convolution, I said convolution before. But then what we brought, we brought also the liquid computational blocks. For example, the double-gated conference of gating a certain type of biases onto different dynamical systems or different operators. We added them, and probably the space becomes something around 50 to 100 different operators. and then you want to build hybrid models so that you can reduce space.

47:35What's the goal here? The goal here is that maximizing efficiency of computation without loss of accuracy. That's the goal that we set. The objective function of the search space is also this. Let's run. Let's have all of our compute actually thrown at this problem and let's try to see how the system is going to design this. And we started doing scaling laws on this. Like we went as small as a neural network as, let's say, 10 million parameters to running the scaling laws up to like 72 billion parameter models, you know, like in these hybrid kind of structures. So we have done a massive early on, early days of liquid, like 2023 and 2024 has been always like proving out on a certain type of processor what's the most efficient type of architecture that can come out.

48:20And it turned out, when we put all of our biases away, all the gating stuff that we are putting around operators, they got simplified. So in Mamba-style architectures, you have a bunch of gating mechanisms. In, let's say, gated delta nets, you have a bunch of operators in there. In linear attention, there are gated variants. Everybody is tweaking a certain parameter in the network by hand because in their own experiments, actually, they observe something. Turns out all of this has to go away if you want to get to the most efficient format of architecture. And it became the double-gated convolution that actually came out of this massive search space, like AFMD, this original kind of system that we designed.

49:09And this was one of the candidate architectures that came out that turns out to be very good on general-purpose computers that are called CPUs. On CPUs, because CPUs don't have, like, they have a special kind of structure, right? On all the CPUs coming out of AMD, Qualcomm, let's say Intel, you know, everyone that is building a processor, all the ARM processors, we try to get to the place where tested different operators, what's that generic kind of structure, and let's say computational graph, that gives you to the most simplified, no, you know, no added hand-tune. There's no hand-tuned features to anywhere.

49:48It's just literally coming out of the test, the semantic test that we've done. And this became the de facto architecture LFM2 structure that we actually announced. But while exploring these things, we've observed many different candidates also popping up. There's too many things. And then we try to really figure out, for example, for, let's say, an NPU and neural processing units that is like inside an AI PC powered by AMD or powered by Qualcomm or powered by Intel, which of these variants would be like a better neural architecture that gives those hardware providers and silicon providers a boost on the amount of efficiencies that they are unlocking or let's say a speed of competition, latency that they get, memory footprint that they get while having a computational graph that doesn't sacrifice any format of quality.

50:36So this was like the whole thing, removing all the human bias with a systematic approach to architectures, even our own biases, you know. And the only thing that actually remained in elephant two on CPU competition is this double-gated, as I told you, we have nested competition in liquid neural networks originally. This nested format of competition was something that actually became very interesting to be part of these very simplified kind of neural networks that we built. And the fact that we have unstructured 1D convolutions as the layers of choice, 70 to 80 % of our networks are structured by these gated convolutions that we have, you know, double gated convolutions that we have.

51:14They're extremely simplified and they replace attention. They reduce the computational complexity by a lot. They reduce the memory footprint by a lot. They maximize kind of the speed of computation by a lot and at scale as well. And at the same time on the quality, I mean, you have seen like some of these models are actually like extremely kind of competitive to the transformer-based alternatives. I think you asked a bunch of other follow-up questions as well, but I would pause here like for any follow-ups here.

51:42Nathan: Yeah, let's get into use cases in a minute. I'm definitely interested in that. And I also want to talk about the future of hardware and what your work implies for the future of hardware. But I think it would probably be helpful for a lot of people, including myself, although this is something I've studied in some depth. I still would like to grok it better than I do. This concept of gating, I'll give you my rough and ready understanding, and then you can improve it and deepen it. I especially noticed this with Mamba when I went down that rabbit hole and became very excited about it. It seems that the kind of trick that is played over and over again with these gating mechanisms is we want the transformation that is done on the data to be input dependent.

52:31Nathan: So it's not enough to learn a transformation. We want to learn a transformation, but then also have some relatively, typically the gate is relatively low dimensional. Sometimes it could just be like a scalar that's applied to that transformation. It could be obviously more complicated than that, but it's some relatively simple mechanism that says for this learned transformation, information here's how we're going to modify it given the input currently under consideration and that seems to be great it seems to unlock a tremendous amount help me understand more anything you think i'm missing there or help me deepen my intuition for why that is such a powerful and recurring theme in all these different architectures that's a liquid that's a liquid structure you know that's when i say like this input dependent kind of thing that is actually coming into Mamba.

53:23So right before Mamba actually got out, you know, like actually not right before, but a year and a half before Mamba come out, we released a paper called Liquid S4. You know, this paper was, if you just read the abstract of that paper, you know, like we're actually for the first time, we're introducing this idea of input dependent SSMs, basically input dependent SSMs. The idea there was that let's bring the fundamental, let's say, building block that we found has a lot to do with representation learning, with capacity of learning, this input dependence kind of element, to really bring that format of gating to the neural architectures, to SSMs.

54:06And then bringing that to convolutions as well with the other kind of systems that we are designing today, liquid foundation models as well. that kind of format of gating, it actually adds a lot of, like you should just think about it, like naturally it makes a lot of sense. If the neural networks, it's not about the forward pass of the neural network that it is adaptive, you know, like because when you're input dependent, you have, you know, like input dependence, in the learning, from a learning theory perspective, all the magic happens in the backward pass, you know? If you think about when you're computing the gradients backwards, that input dependent kind of operator itself, it's going to represent itself.

54:45So when you're learning from that data that you're seeing, you're also learning some sort of dynamics, input-dependent dynamics there. So that second axis that I was talking about, it's not just the number of parameters of the neural networks, it's also the dynamics that the neural network actually learns. You have some representation of that dynamics being learned in the backward pass. Now, the complexity of that gate is going to make a huge change. And also, it turns out that gate is extremely important also in language modeling, especially when you have sequence models like RNNs, like, let's say, classical controls, like, let's say, continuous-time RNNs, let's say, discretized RNNs like LSTMs, and then you have liquid neural networks, like, again, very nonlinear versions of these.

55:29Linear versions of the Reconnaught Normal is an SSM, and when you add this type of gating, you can improve the language capabilities as well. It seems like for, let's say, discrete sequences, this gating also helps a lot. You know, it's not just for the continuous time sequences, but also discrete sequence. Because I would consider language to be like a discrete sequence, right, of information. So you would have like that sequential dynamics that is kind of added by this gating. So it is a very right frame of mind to think about that kind of feature. And for us, we think that this input dependence is one of the fundamental discoveries that we've done that we learned also in biology.

56:10It happens. It comes out of physics. If you just put the math together, you're going to realize, oh, what is special about this dynamical system is the fact that it has this nonlinear input dependence. That's kind of a structure. And by biases, I mean the complexity of that gating. Even existence of that gating is a bias that you're adding to your system. Now, is it needed or not at scale? That's a big question mark. You're going to see when neural networks are at 100 trillion parameters, is matrix multiplication is enough to really perform general purpose? Could it get to AGI's of the world or not?

56:46That's a big question mark. But also, we need to also put this discussion about architecture and obsession about architecture in perspective. Another perspective we have to put this thing is the learning theories themselves. Right now, we are talking about different learning schemes that are coming. For example, you can train a model with next token prediction. You can train them in a word modeling kind of context. You can train them in a sequential, in a long-term horizon and long-term history. You know, there's so many ways you can construct also the objective function of the learning algorithm itself, you know, that that would also contribute to a lot to the learning process.

57:22Architecture, I'll tell you that we found the purpose of the architecture and the most important application of architecture or kind of research has been inefficiency, right? Really getting to efficient formats of competition without loss of quality. That's an extremely important thing because we're talking about the resource allocation problem right now. As the models are becoming bigger and the demand of AI is exponentially going higher, you want to have more efficient versions of the systems to actually run the systems at scale. Otherwise, how can we actually provide access to AI to all? And we are seeing early instances of this thing that the larger labs are wiping out the compute off of the planet.

57:59Why? Because they have to train and they have to then host these models for all of us. So efficiency becomes a fundamental property of these model architectures that we are doing the research on. I would say if you want to get into architectures that enable next generation of neural networks, sorry, next generation of intelligent systems in a way that they can become like human brain that actually performs, you know, like performs competition with 20 watts of power. And it's already an AGI that does that between that's like the massive exploration that goes beyond architecture. It goes into memory research.

58:34It goes into learning algorithms kind of feature. It goes into data feature. You know, it goes into prior research, you know, like looking at it from a statistical learning perspective. That becomes kind of it. And then learning theory itself, like the limitations of learning theory itself, it also gets imposed on today's learning systems. Because the definition of learning theory itself is actually broke like at scale. Like what we have right now, like everything is IID. You know, like we are building averaging machines. You know, you have seen the quality of writing and the sycophantasy that actually comes out of AI systems because of the fact that, you know, we haven't gotten like this.

59:10Maybe like to some extent, like multi-agents has been becoming the solution, you know, and test time kind of scaling became kind of a solution for the caveats that you're seeing from just learning like systems with autoregressive kind of modeling as a pre-training kind of method. But I think there will be innovations needed, not just on architecture, but the whole thing as a whole. You have data, you have models, and you have learning algorithms. Altogether, they can design basically the future, that ultimate kind of holy grail, basically, in this space.

59:43Nathan: Yeah, that's very interesting. So basically your perspective is that as we get into the recursive self-improvement era, the real advances will come more from new learning paradigms, new objectives. And then those advances will be made efficient through architectural optimization. but the architecture comes after the paradigm question of what exactly are we learning and how are we giving a signal? It is one component of it. That's what I would say. Looking at it from a data problem, data representation from, let's say, an axis would be architecture and the other axis would be just the learning algorithms themselves.

1:00:26And then we get into that recursive self-improvement, which is kind of the continual learning kind of characteristics of the new system. fancy name for the continual learning kind of research that has been happening for more than probably four decades now. That's how I would actually characterize the whole space. We kind of just look at architecture in isolation as the fundamental thing that actually changes everything.

1:00:56Nathan: You remind me a little bit of Ali Beirouz's The Illusion of Architectures. If we have maybe we can touch on nested learning. Okay, well, let's stay focused on your work for the moment, at least. So in LFM2, this is the result of this architecture search. And the kind of surprisingly simple thing that comes back is some reduced but still critical number of attention layers. And then the other layers are, and this is, we've seen this kind of with SSM attention hybrids as well. But I think the kind of surprising revelation from the result of this search process is the non-attention layers can actually be extremely simple as long as they have some gating.

1:01:46Nathan: So you've got the gate and then you've got just a real simple convolution that I think I only considers a very short span of tokens, right? Is it like just the short ones back? And so if we were going to update the headline, attention is all you need from however many years ago now, we would maybe say... So attention is something that you really do still need, at least at a certain scale. But you also need gating on your other layers. But you don't actually need anything like super crazy, fancy, sophisticated, the state space model, all that kind of stuff. it turns out that keeping the gate is actually the part that really drives the most value.

1:02:29Nathan: And then you can have a really simple mechanism behind the gate. And then you can have 70 % of that and 30 % of attention and subject to obviously resource constraints that ended up being the winning formula. Am I getting anything wrong there? No, I mean, you're touching on the right things. And this is basically, it's like the regime you're operating and the goal of your system. Are you trying to build super intelligent? Are you trying to build the most powerful version of the AI system? You need the most unbiased version of an algorithm. Now, attention is an extremely rich, unbiased format of algorithms.

1:03:08Even if the computational complexity of attention is n to the power 2, maybe we really need n to the power 2 to really get to that kind of level. And maybe even we need more complex architectures. We've always tried to actually reduce the complexity of architectures for the sheer purpose of the fact that we are resource constrained. As humanity as a whole, we are resource constrained right now. So I would say the discoveries that we have right now, it just shows that there is a gradient on architecture that you can follow as you scale models. the gradient that you're following is the fact that for smaller kind of models and specialized models you can put as many biases like these gating mechanisms that you're bringing in you know and as you can you can play around with a with as many operators of interest like in the in your computational graph you know and it is going to work and it is going to give you some sort of a boost if you're really maximizing for like linear um let's say linear time complexity like you want to implement linear attention systems, you know, like just the fastest kind of if the speed is like so important and you're actually wanting to even sacrifice a little bit quality, you can bring in like linearity and the whole system could be linear, you know, you don't even need some of those hybrids.

1:04:21But hybrid is just boost that accuracy because as we see there, as I said, the O and two, like basically the complexity that that the computational and complexity at a certain level, at a certain scale, it is needed for us to really get to those performances that you want to do. The larger your network becomes, the more unstructured you can make it. That's kind of the learning from that whole kind of algorithmic approach that we started designing in neural architectures. Yeah, very interesting.

1:04:50Nathan: Okay, let's look at the other end of the spectrum then. As you work with customers, what are some interesting examples of when, given resource constraints, given the narrowness of the domain of interest, other kinds of bias are actually winning in the architecture search process? Great question. So, for example, if you go to, let's say, biology, you know, and you want to model sequential data, you know, in biology, you're talking about DNA data in biology. DNA data, from a vocabulary kind of perspective, they're very limited, right? They're not like language that is like maximum kind of amount of vocabulary and stuff, you know.

1:05:29In DNA language, they're very simplified, but the lengths of those sequences you have to actually process are, let's say for a human being or maybe for a bacteria, I heard, it's somewhere between one to a hundred billion sequences, like basically elements in a sequence, you know. For that long context kind of things that you want to perform, when you do not have that much of a large vocabulary, you don't need attention. so you can you can actually run on on on this kind of data you can you can run pure convolutions pure ssms your pure liquid neural networks like you know system like recurrent neural networks and parallelized version of these recurrences and you know linear attention for extremely long context you cannot do it any other way the reason behind it is because context has become so large that that quadratic cost of attention just kicks in you know so let's say on biological data you would want to have like some some sort of a structure there then uh there are places like video modeling and stuff you know like in those kind of places let's say you might want to have like different architectures and even different learning algorithms you have seen like the success of diffusion for example there right like you've seen like diffusion is also like a i would say like this format of a prior that you put on a certain architecture but it's still like you there's debate between like diffusion and learning algorithms or diffusion is actually part of the architecture, you can actually make that connection and you can make that distinction as well.

1:06:51So you have seen like on video and seen understanding probably an elements of diffusion would be needed. You know, like some people still believe that with autoregressive kind of modeling, you could actually get there anyways, but we will see if that stays true. Then when you're talking about audio signal as well, like if you audio alone, for example, if you're talking about audio alone and language is not part of the whole thing, it's just pure like voice-to-voice, let's say noise-to-clear kind of signal, you know, signal-to-signal, these are places where recurrent neural networks are still very, very powerful.

1:07:27They're extremely powerful, especially in the low data regime. You know, I would say models that have a lot more biases in them, you know, like in this smaller kind of regime, when you do not have that much data, this is kind of the place where you can bring a lot of value, you know. So, and recurrent neural networks and biases in architecture can help you in the low data regime to really, like, fill out that gap with the feedback mechanisms that they have, you know. The more complex architectures, the more you would be able to handle, like, the closer the architecture is to the dynamics of the data sets that you're trying to solve, the better of a learning system you're building.

1:08:09In fact, you are, you know, like in the physics modeling, you know, like when you're talking about there was like a class of models like physics informed neural networks, you know, like these are kind of, again, another architecture that we are talking about here. They would be really good for physical kind of simulations, you know. And, yeah, so this is the other side of the spectrum that I'm talking about, like the property of the data, the lengths of the context lengths and, you know, like all the other considerations that I said would change the architecture by a lot, you know. And then, again, you want to scale this to the largest kind of regime, make it, again, unbiased.

1:08:45Again, a transformer would be able to add scale to beat this. But at a smaller scale, transformers would not be able to beat any of the other formats of dynamical systems that we discard.

1:08:55Nathan: And how about the differential equation-inspired original liquid neural networks? It's clear that the worm is running on a very small amount of watts. In practice, do you encounter things that are so resource constrained that you have to go to these extremely specific or extremely biased architectures to actually deploy today? Yeah, 100%. Think about places where you have to bring in latency to microseconds. The competition has to happen in microseconds. so you cannot really afford to have like larger computational complexity you want to have the most simple type of uh system that actually handles that and you know what there you can have you can have adaptive systems like you can have like many different formats of systems and there this is like one place the other thing is simulation in a whole as i mentioned like in physics you have like physical data that is coming like you want to build a digital twin of a let's a chemical reaction that happens at a factory.

1:10:05So in those kind of places, you would go towards an extremely biased and even maybe just differential equation-based models like Liquid World Networks today. They're getting applied to many, many different applications. I see that because the original repository from years ago is still open source. People are still like building predictive kind of machine learning models on sequential data or physical data or sensor data that it makes sense because the systems are continuous time and it makes sense to have like a continuous time dynamical system to apply to modeling those kind of behavior, you know.

1:10:40And then you can like today with these larger instances like clods of the world, you know, you could also like direct your agents to actually try out a bunch like in an auto research kind of format, you know, you can just direct them to, so, hey, you know what, go pick, like, the neural networks, like, the best neural networks, like, would be the best feed for this type of data set, and then here is kind of the space of possibilities you want to explore, you know, like, based on the biases that you have, you know, like, machine learning people are now mostly, like, orchestrating kind of these automated kind of agenting pipelines as we are building, so that's what's happening.

1:11:13In terms of Liquid ourselves, like, we are We are trying to contribute to the open source. We are open sourcing some of the instances that are coming out of our, let's say, search spaces, like, let's say, LFM2. LFM3 is going to be, like, again, you know, the objective is that the next generation should always beat the previous generation on the criteria that we care about, you know, without sacrificing quality. You know, like, that's extremely important for us. And then the range of possible, like we decide, for example, how large of a neural network we want to make based on these neural networks that we have.

1:11:47And then this becomes the open source version of our models. Then we work very closely also with silicon companies. You know, we work with AMDs of the world, Qualcomm's of the world. And what we do with them, we try to understand the silicon roadmap that they have, you know, the hardware roadmap that they have. And based on that roadmap, we try to kind of inform also like what they have to do even in the next generation of their ASICs, you know, because when we have like this understanding of the algorithmic aspects of intelligence and a variety of things that they could be supporting in order to reduce, dramatically reduce the cost or satisfy the constraints of the use cases that they want to enable, these are the considerations that you have to do.

1:12:28or even building like to the point that we can build also like, you know, like specific foundation model graphs, like graphs for them, you know, like that's, and that's kind of the projects that we do with semiconductor companies. In terms of like on the commercial kind of side of things, we also take models, we apply them in many different, I call it the category that Liquideye is very enabled. We call it device foundation models, you know, anything, all the processors that are outside of data centers, we try to apply our technology to those kinds of places. This is outside of data centers. Inside of data centers, we apply to constrained use cases.

1:13:04Use cases like low latency, like ultra low latency applications of AI. Extremely long sequences and wanting a very, very small memory footprint of your AI system. You want to have cost-efficient implementation of these foundation models that you have. We contribute to memory, speed, latency, without sacrificing quality, and cost. You know, these are kind of the elements that define use cases where liquid can come in. In terms of, like, access to our technology, you know, any new generation of our architecture that comes out of our efforts of, let's say, architecture search or new modalities that we check.

1:13:42For example, we Shopify. we're exploring many different kind of, you know, like range of applications, you know, across recommendations, search, product catalog, like understanding multimodal kind of systems very, very well. That's something that we do. Our models are in production like we Shopify right now, and they are improving the quality of the click-through rates and the criteria, internal criteria that Shopify actually cares about. Our models outside of data centers, they go inside cars. You know, some of the use cases that we enable is like basically in-car intelligence. For example, we power that.

1:14:15Recently, we have like a contract design. It's actually a historical one with Mercedes-Benz where our models, like it's going to be powering kind of the audio and also the visual kind of elements inside the car. So whenever you want to talk to your car, basically the new voice would come out of a liquid foundation model, you know. And we control that with a model that is like giving you the quality of the best models that you have seen so far. what like let's say like the audio models that you've seen like all around but at the same time it is 600 megabytes it can actually fit inside like the smallest processor that is inside that car you know so like and that that that's kind of that's that changes the game you know like because you're enabling local ai at scale like on places that matters you know on cars on let's say on mobile phones because you can do that there's a lot of devices like you're talking about billions of devices in the world that mobile devices can be extremely useful.

1:15:09And you've seen that market itself, by the way, just the mobile kind of business, it's$500 billion market. This is absolutely insane. And it is as big as kind of the data center kind of market. So you can imagine, like, there is a power here to be made, like, you know, the efficient markets, you know, and also conscreened intelligence market. It's something that Liquid AI is going after. Laptop markets, you know, like, let's say, wearables, robotics, manufacturing, IOT kind of systems. Anywhere that we have like a processor inside a system, Liquid can bring in intelligence on top of that. So the aspirational goal that I have is that like we really build an intelligence layer on top of the diverse formats of hardware that is out there and processors that are available in the world.

1:15:51You know, that would be something that I would want to have.

1:15:54Nathan: I'm glad you said that, Stat. I had that noted to make a point of it's a point. It's worth repeating. The global annual smartphone market is about$500 billion for all of the hundreds of billions that are going into the data center build out, and it will get bigger than the smartphone market. But it's only now getting to the scale and getting bigger than the annual smartphone market. So there's a lot of dark compute out there from an intelligence perspective that is not being anywhere close to maximum. That does not include the laptop market. So there's a trillion dollars, generally speaking, worth of compute going out into the world on an annual basis that are sitting in people's desks and pockets.

1:16:35Nathan: And there's a massive substrate there to take advantage of. And we need to do it. We need to take advantage of it because we don't have enough energy to host it. I feel like we are realizing it right now, how difficult it is to really power this format of intelligence that we have like at scale we really have to have be work smart we cannot just do the simplest format or maybe more more constrained use cases like let's say if you have like a one-shot uh let's say predictive task or maybe data extraction task and stuff you don't want to call the fanciest type of intelligence like the mythos level kind of model to perform data extraction for you you can have like a variety of different intelligence systems that are doing many, many different things, you know, and then for, let's say, the most sophisticated problems in the world, you can go to the most sophisticated AI systems that are existing in the cloud, right?

1:17:27So, it's inevitable that the world, the processor world outside of data centers should get enabled, like, as we speak right now, like, and I think there is actually some talks, like, coming out, there's a lag in Silicon Valley, like, investors, like, that are actually, like, just, oh, okay, now that makes sense, you know, like, there is that lag that is getting answered right now. And I think we are in exciting times also for efficient AI in general.

1:17:54Nathan: Okay, so on the one challenge, obviously, with this other trillion dollars worth of compute that's going out into the world is it's super heterogeneous, right? There's many, many different devices, different chips, etc., etc. So how much of the work that you're doing is about trying to get closer to optimal use of that compute? Obviously, we talked about scans earlier and the transformers and the data centers. That has been not, I'm sure there's still more optimizations to be made, but a ton has gone into it to make sure that you can get as close to max throughput as possible, right? When you go then to an architecture search and you're targeting some random cell phone or sensor in a factory somewhere, how much do you have to work on like kernels and scans as kind of foundation to even be able to do that search yeah well i mean that's that's one of the you see there's like different layers of abstraction you know like and we're we're testing out like also like the the ai see like how good they can get like on the kernel design kind of side of things you know for the things that they have seen that you know in the open source like the kernels that are available in the open source like this let's say the gpu structures that they have seen out there, but not the NPUs, like the most hidden type of NPUs that their IPs actually not disclosed, those require, let's say, another architecture search because we don't have those knowledge.

1:19:26But I would say kernel design is at a level that we can actually start automating. And also you can definitely have loops where you can juice out the best post hoc optimization Because some of the things that we learned, whatever we talked about so far has been optimization that we do before design of a foundation model. All sort of considerations before we start pre-training a foundation model. The approach that Liquid takes before getting into, let's say, pre-training a model, we run this massive architecture search before getting at mid-middle hardware in the loop. But then right after you design the architecture, then there's all these sort of quantization of webware training.

1:20:05changing the bits of the system, like going into the kernel level and trying to even juice out even more off of the system. All of those post-hoc stuff is also something that you can automatically orchestrate with a kernel engineer that does that and performs those optimizations. And we do that. We do that as a post-hoc optimization step. That being said, we have the capability in-house to define operators at a kernel level as well. For example, you could say matrix multiplication have on this format of kernels probably 100 ways that you can actually structure matrix multiplication, with caching, with not caching, distribution between CPU workload and then GPU workload.

1:20:54How do you actually do the entire forward pass as you perform? all of these optimizations could also in principle become part of that massive search space as well. And these are kind of the questions that some of the Silicon partners are asking us, you know, how can we, how can we even before design a foundation model, figure out the competition graph first, but what having like the guarantees that when the training is finished, because that's like a costly process, you know, multiple millions of dollars has to go into full training of a neural network, you know, even though the neural network is small, you want to have like all sort of like like inference optimization before a post-hoc, like quantization aware kind of optimization, right?

1:21:33But yeah, in principle, we can definitely launch something like that as part of our search.

1:21:38Nathan: So what do you think hardware makers should be doing differently? What are you telling them to prioritize to help us realize this more efficient distributed AI future? I feel like with every iteration of the technology, software technology, like the level of abstraction that the hardware providers should be building for, it's coming up. For example, we've been talking about CUDA modes for a long time now. You know, there is no CUDA mode anymore. Why? Because if you look at the AMDs, there are agents at the kernel level, at that level. Now they can automate that stack. The stack that foundation model companies should, sorry, hardware companies should start working on is the intelligence layer.

1:22:21like what NVIDIA is doing with their Nematron project. If you see how this project is actually quite successful, and probably a couple of billion dollars actually went into design of these Nematron projects. And if you really look at what does it do for NVIDIA, it's just basically building an intelligence layer on top of NVIDIA's compute so that you have an easiest barrier of entry as an enterprise to buy NVIDIA solutions. You know, so that's the sales of the solution is always like around. So from the I'm just talking about the business of like enterprise business of these hardware providers, you know, if they want to sell more hardware, you know, they're going to up their stack from that kernel level optimizations.

1:23:06Those are kind of things and they get into the intelligence layer. Now, does that mean like they have to become a foundation model company? To some extent, yes. And they have to be able to train that intelligence layer themselves. And NVIDIA is like a success example there. How these things actually like, you know, paying dividends for them, you know, like building the Mnemotron project. If you look at the other foundation model companies, like the other hardware companies, they have not done this yet. They have started like strictly like optimizing their models for the open source architectures that are out there.

1:23:38That's also extremely valuable. You have to do that. That's like a given. But again, if you want to create a differentiated value and you want to be successful, I would say you've got to be able to bring the stack to the intelligence layer. And that intelligence layer should naturally fit on top of your hardware. If your hardware has limitations, let's say, in terms of like to your competition, software can always help to give you that juice. Like, for example, if you're talking about maximum token speed on an AI PC between an Intel computer, between, let's say, an AMD computer and, let's say, a Qualcomm computer, they might have, like, their own tradeoffs.

1:24:13And the company that is going to have an edge is a company that actually owns that efficient intelligence layer on top of their hardware so that they can say, at the end of the day, you're going to run tokens on top of this system. It doesn't matter what's the bandwidth of my CPU, if it's a little bit lower than you or higher than you or the amount of memory that I'm actually having in there is lower or higher. With the intelligence layer and software optimization kind of techniques, I can actually get myself to the place where I'm actually the winner here. And for example, all of my laptops or all of my PC solutions that I'm actually building as a hardware provider, they are naturally coming with an intelligence layer on top of that.

1:24:51But NVIDIA is entering this game also heavily as a competitor in the CPU space now. As you're seeing, Google actually entered with the Android ecosystem that they have. They're replacing Chromebooks with Android laptops. This is kind of what they're putting out there. I think it's called Aluminium OS or something like that. I don't actually know what exactly it is. And then Meta is going to come with the sort of devices themselves as well. So all the processor builders, I have a feeling they need to get a little bit closer to that intelligence layer, you know, and try to leverage software and efficiency kind of considerations into account when they are going for their planning, you know, like planning ahead, you know, where things are going.

1:25:37And why that intelligence layer is important, because right now, agentic AI is something that everybody else wants. All that applications you want to do on silicon, you want to do that on a base of intelligence that is already provided to you, you know. You want to build harnesses. You want to build like all sort of the harnesses are basically applications you can build for solving a certain problem in, let's say, in a traditional sense of an application. Right. And that market can get enabled on top of chips if you have a very nice stack, you know, that enables intelligence on top of what you have.

1:26:10The other thing that I would say is that like. Like, silicon is always, if you're going outside of data center, I mean, there is a massive diversity of silicon, but then the resources are constrained no matter what. You know, you don't have that much power, you know, so you've got to be very careful, like, what sort of applications, like, are the most important uses of your hardware medium and platform. And then those use cases, if you already have an intelligence layer, you can identify when is the time to sunset, basically, a set of, let's say, devices. For example, I strongly believe, I think, glasses could be like a format of competition for hardware.

1:26:46Later on, like devices, I think glasses would be like very, very interesting kind of medium. And I think somebody is going to get it right, you know, eventually, you know, and it might actually be a replacement for, let's say, for laptops, right? And then, but is it, you know, like we need to really like figure out like what is it. But if you really want to define a market and define kind of where the field is going from a hardware perspective, I would say you got to up your software game by a lot and really get hands on and investing a little bit more energy and capital into the R &D side of things on the Foundry Mono side.

1:27:20Because that's the new base of software, like inevitably.

1:27:23Nathan: So does that imply a future where we have just a lot of vertical integration and a lot of coupling? the model will come with the hardware that i buy and it may not be so swappable in the future because the model is heavily optimized for the hardware and vice versa such that these things are not so modular in the future as they are today or why do you want to change the model you know like it brings it to that choice kind of question you know you're given basically a default now you want to switch this thing by all means but why do you want to switch it you know if this model that the intelligence layer that is in there, it's not fixed.

1:28:02It's kind of an adaptable kind of system. It is, let's say, self-improving system, like with a call to your data sets and stuff, you can actually have a platform that does, let's say, full fine-tuning of that system. It is enabled. And there is not just one model that you can load into this system. And there's not just one cloud model or one on-device model that you're going to use. You've got to be able to orchestrate between, let's say, many different instantiation of this model to be able to build application. We're talking about a model class that would actually sit on top of, let's say, hardware on a laptop, for example.

1:28:36And yeah, so I would say you need to have that. But I mean, if the default is just giving you that efficiency and that it's ready to go, that's the choice. That's something that I think NVIDIA is trying to propose in enterprises. is when they go and sell like the NIME project, NIME projects, when they go out there and sell these things, it makes a lot of sense because you already have like a project, you already have a multimodal model that is loaded on top of your, let's say the PC that you want, you know, and why should I switch? Because they already have done all sorts of optimizations for me, you know, and it is running like extremely fast.

1:29:13Why do I need to change it? And the model itself is tunable. I can use their Megatron kind of framework to actually tune the model. Now, if you don't want to do that and you want to just choose another model to host it, as I said, this has to be given. Your hardware should be already – you have to be optimizing for the entire open source ecosystem and models that are available. But at the same time, it would give you an advantage to yourself and to your customers. Okay.

1:29:40Nathan: Maybe in the last few minutes, how about a practical application of this? You have a blog post on local co-work. No cloud, no waiting, tool calling agents on consumer hardware with LFM2 24B A2B. So that's a, I think people know, that's a mixture of experts, 2 billion active out of 24 billion total parameters. Let's say I want to make that a part of my life. I am interested in how you would coach me on setting this up. Today I have for reference, and I, by the way, try to make the, sometimes I try to make the transcript of the podcast, something that I can feed to my agent. So you can think of this as partly coaching me, partly coaching my agent.

1:30:23Nathan: So I've got this deep context database that is the last five years of my digital output. This podcast will be recorded, obviously recorded, transcribed. It'll go into this database. So everything you said, everything I said will be in there and searchable. And it's got all my email and Slack messages and everything. Okay, cool. So now I've got Claude on my desktop that can call tools locally to get data back, but then it sends all the results to the cloud to decide which of those results are actually the right ones to be looking at. And so far, I've been okay with that. The benefits are certainly worth whatever risk I'm taking, I feel.

1:30:58Nathan: But I would maybe love to run that data through a local model first so that I don't have to send all my data to the cloud every time. Now, notably, when it gets filtered through, I'm probably still gonna end up running through a foundation model. So it's not going to entirely skip the cloud. But also I'd like to save some tokens too, because I'm going to be going to use Fable whenever I get it back for whatever it's most appropriately used for. So how do I get really good performance on my local computer with this model? Do I need to be doing fine tuning? Do I need a distillation strategy? How do I actually take the base model that you have and recover as much of, let's say, opus or even fable performance in terms of searching through understanding my data as I possibly can?

1:31:50Nathan: And how much is that? What should I expect in terms of how far I'll be able to push that process? How close to parity with frontier models can I get? So tell me everything I need to know and I'll go have my agent do it. Yeah. Well, I mean, that's a great question. Obviously, the local cohort is as it stands today. Obviously, it's just to open minds like, hey, you know what? This type of applications you can enable. The class of what this format of local agents are going to enable is basically a local computer. You have an orchestrator. If something is so complicated, it should be able to send it to the cloud, fetch the answers for you if it's not sensitive.

1:32:33We have, let's say, smaller models that are PII models. You should be able to use the PII models to really filter out all the personally identified information, send it to the cloud for you. You don't need to even see these models. These models should be on the background. The model that you have to tune is that orchestrator that be able to route between many different services or even smaller specialized models that are doing stuff, and also some of the cloud models that are out there. That router is the computer. That's the local computer. That's like a definition of, like, when you open your laptop, it should be just that.

1:33:06You know, like, you literally have, like, just that. And then you start, like, working with all the services you want to have. It's just the same way that you communicate with your, let's say, assistant. You know, like, it should be, like, in the format of an assistant that does all sort of those jobs, you know. Like, it takes a while for people to get not weirded out by the user interface, like, just being that and not seeing, like, all the, like, file formats. and you know what you have to do, but you've got to get used to it. You know, like now cloud code is like even as for the ID. Now, is the 24B off the shelf is going to be there, like on the quality that does all of those stuff for you?

1:33:43No, it's not. None of the local models today are there. On the local models today, you've got to fine tune them. You've got to get them to be specialized for the stuff that you want to do with the proper explanations for your cloud agent. You basically give it these things. But is Cloud being able to go out there and actually build this model for you? Let's say, do the fine-tuning and getting it to that place? No. Because today, even Fable-level models, they would not be able... First of all, you wouldn't have access to that because Entropic is actually having access to the auto-tune and stuff, automated models-generating models kind of platform.

1:34:24there are companies and also ourselves we are building platforms that enables you to do fine-tuning of this whole thing that would cost you between tens of dollars to let's say like low thousands of dollars to actually get you to the cloud quality for your and with all the checks production quality kind of model and it's not going to cost you like tens of thousands of dollars it's going to be like between tens of dollars to actually like those thousands of dollars you know And that's kind of the scale that we are thinking about because, again, efficiency here matters a lot because we don't want – and we want this fine-tuning to happen on the compute that you already have.

1:35:03Or if you don't have it, you would actually put it like in a secure and proper kind of data center that you would actually like host it, like from the providers basically. And you would be able to get to that quality of the models. There are platforms that hopefully, as we go forward in the next few months, we are going to announce some platforms that you can hook it just directly in your terminal. You don't want to do anything. You just say, hey, you know what? Go call this platform. Now, for fine-tuning this thing, go call this platform. This would give you a production-grade foundation model that you can deploy it now for yourself.

1:35:36Either fine-tune on that data or depending on the use case, depending on what you want to do, it doesn't even have to see that data. It can even synthetically generate data on its own and actually train the model to be like just the perfect, reliable tool caller and understands when is the shortcomings of it and it can actually go away and deliver it to some other places. Yeah, so I would say we're very soon close to get to that. I'll wait for your platform to be ready.

1:36:00Nathan: I don't have to DIY is the message. Yes. This has been brilliant. Maybe one last question and then I'll just kind of give you the floor to close however you'd like. How far do you think this goes in terms of kind of miniaturization of intelligence, if you will? One intuition would be like the biological world that we see is maybe on some sort of Pareto frontier already. And so the watts that go into our brain is maybe like getting us close to the maximum that we could get for that amount of power. But maybe you have a different intuition. What would you expect in terms of upper limits of intelligence that I could have on my phone or on my laptop as we really get to the kind of physical limits of the technology?

1:36:51You know, when you think about intelligence, like the way I look at it right now is that like transformer-based networks and also like our type of, let's say, the current architectures that are landscape of architectures that are available, they gave us, with scale, they gave us in-context learning capability. The thing that actually emerged from NextToken, you see, the word is important, emerged from NextToken prediction. Intelligence for me is an emergent property. if you want to get into like if you want to miniaturize kind of intelligence bring it like to the to the physical world i don't believe with the current set of algorithms like we would be able to get close to the let's say intelligence per watt that human brain is actually like providing right you're not going to get there why because i believe human brain over the we also have to like consider the amount of energy that went into design of humans you know as a whole biological evolution.

1:37:47It's actually a very, very long kind of process. And a lot of the pre-training process that people say, oh, a lot of, you know, like you're going to see the entire world. Humans do not need to see the entire internet to be able to reason about something, you know. No, but humans have gone through like years of evolution, you know. So I would actually attribute it a lot to the evolutionary kind of aspect of things. But again, what I would say, human intelligence came with multiple mechanisms of in-context learning, you know. You just don't have, when our current AI systems do in-context learning, they learned a vague representation of one algorithm, which is least square.

1:38:24You know, it's basically least a square, you know. So what they figured out is basically gradient descent in a mushy way, you know, like for some use cases, with some examples, you would be able to make the system understand and give you like kind of the next example. These are kind of beautiful properties. This is what I would call an emergent property of the systems. You set the algorithm to be next token prediction. You got a vague version of a gradient descent in context learning capabilities. For humans, you don't have only emergent gradients. You can learn by examples, but you can also do reinforcement learning.

1:39:01You can run simulations in your head. You can do all sorts of algorithms. You can do Bayesian statistics in your brain. You see what I'm saying? So you have like a diverse set of algorithms emerged from the way that humans actually got designed and got intelligence. For me, I believe if you start optimizing for those, if you try to force your way into the system to become a reinforcement learner, but learning from trajectories, that's not the right way to actually get to emergent property of intelligence. I believe the format of intelligence that is going to be miniaturized, you know, like being like in the smallest amount of time, maximum amount of intelligence, like the human brain, we need to come up with these identifiers, like these confounding kind of algorithms, which is next token prediction is one of those.

1:39:50What else do we have to do at the beginning of the design of these systems so that reinforcement learning, like curiosity-driven like intelligence, drives with limited amount of energy from that final system that actually comes out of it? So I would say that emergent property is something that we need to realize. There's a lot of research that has to go in that direction, I would say. And multiple kind of variations of way of learning, that would enable the next generation of artificial intelligence, I think.

1:40:19Nathan: That could be a great note to end on, but let me just give you one more opportunity. Anything else that I didn't touch on or anything else you just want to leave people with before we break? No, I think we covered a good lot of information. I think we went into some things that I don't usually talk about. Now, in my current role as the CEO of the company, I get to talk a lot more about the business opportunities and the market aspects of the intelligence and stuff. But it's still like, you know, at the core, like we are scientists and we are really like pushing the boundaries. Like we are actively thinking about it day to day.

1:40:57You cannot imagine like even right now if you're talking like I'm already in the back end, like I'm actually like training some stuff. And I really don't want to get away from that kind of mentality because I think everybody can actually be able to now build, you know, like and I think that the times could be better. the agents are pretty amazing, you know, at giving you like this opportunity of like really doing frontier form research, you know, if you're curious about something, it's just that we need to find people that are a lot more curious, you know, and improve the curiosity in people as well to really get instead of just, you know, like fear of like, you know, everything.

1:41:35So I'm a techno optimist that probably you've observed that like I usually like talk about technology really in a very, very positive light. And I want people like one thing that I always say is that we need to really get to the level where people are as curious as the scientists were, like from what gets us like into science and the purpose of science has always been satisfying our own curiosity, you know, and also understanding the world around us, right? So that's kind of the purpose of science. And I feel like with the current superpowers that are given to us, like with these AI systems, I think everybody could actually start like really contributing to understanding the world around us and also generating more value and also just satisfying our own curiosity.

1:42:18But also like it requires like a certain degree of rebasing our biases towards like, oh, what is work? What is not? And what is automation? What are what is an army of agents working for me? You know, what's that definition even look like? And then a change in culture and change in like way of thinking about this thing, not just for individuals, but also for enterprises we are dealing with as well as Liquid AI. And yeah, so I think that change is something that I'm looking forward to. And I think this is something that would happen. The pace of it is extremely fast, but I think we got to do it one way or another.

1:42:50Nathan: I love that. I consider myself extremely fortunate to live to a really remarkable degree, a curiosity driven life these days. and that's one candidate for a positive vision for the future that I think should inspire a lot of people, especially because, as you note, there's an opportunity to move into that phase already with the AI systems that we have. This has been excellent. I really enjoyed it. Thank you so much for the time. Ramin Asani, co-founder and CEO of Liquid AI, thank you for being part of the Cognitive Revolution. Thank you.

1:43:28Nathan: Coming down, coming down low Liquid, liquid Feel it coming down, coming down low Started with a worm, 300 little lights A tiny brain that learned to drive into the night 12 small neurons, pocket on a dime Who said you need a mountain just to redesign? Cut it down, cut it down, till it's lean and bright Big ideas, my machine, that's the new design Feel it move, feel it, but never frozen in time Flow into your hands Falling on the land Out the cloud and into the palm

1:44:27Nathan: Adaptable, alive Keep it moving, keep it light Bring the mind down low where we live A trillion sleeping giants in our pockets all this time Waking up the silver, waking up the shine Not the fanciest mountain for every little thing Just enough to make the whole horizon sing Cut it down, cut it down till it's mean and bright Big ideas, my mind, she wants to new design Feel it move, feel it bend, never frozen in time Liquid flowing through your hands Liquid falling from the land Pop the cloud into the bomb Liquid, liquid, liquid Adaptable, alive Liquid, keep it moving, keep it light Bring the mind down low where we live Cold, what's the secret?

1:45:39Nathan: Curiosity Cold, what's it going? Everywhere we'll be in the car In the palm in the open air Cock, cock, cock, cock, cock, mid to revolution Intelligence everywhere

1:45:57Nathan: Feel it coming down, coming down slow Out the cloud and into the palm We let it go, we let it go

1:46:24Nathan: If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guests and topic suggestions, and sponsorship inquiries, either via our website, CognitiveRevolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting.

1:46:58Nathan: If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at AI podcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.