Liquid AI's Ramin Hasani on liquid neural networks, AI advancement, the race to AGI & more! | E1928

32:07The reason why they're doing that is because when the models are actually getting larger, they become better and better. And maybe we can get into AGI and generally these kind of AI systems by enlarging kind of the architecture. And the focus is just that. There are two companies in the world that I think the absolute focus of the companies are building AGI is OpenAI as an anantropic right now. So there are kind of gutsy moves like what we are doing basically. We are basically changing the fundamental architecture. We are building new scaling laws basically on top of this thing. The scaling laws, let's see if we can make liquid neural networks also scale.

32:46That means if I have one trillion parameter liquid model, it might actually be as performant as a 50 or 20 trillion parameter transformer model. The other way of it is also true. If I have a hundred trillion parameter liquid model, it might be better than a 20x larger transformer-based model. So that means these are basically the kind of moves that we want to make. I mean, so far, we have been... If you're successful, when will Anthropic move over to your platform, do you think? Or are you a competitor to them, do you think? I think, I mean, right now, like we are going to another fundraising, like Series A of Liquid.

33:31And I think after this round, we are basically getting prepared to actually train very, very large models. So these models are going to be, I mean, after the release of those models, probably by the end of the year, I would say, then the community is going to see like that there are alternative kind of models that they can come in and disrupt the way transformers are actually disrupting. and they can scale the way transformers scale, basically. What hardware are you going to use? What platform are you using? Right now, we are using NVIDIA GPUs as well. Like it's very similar. It's just that the number, the amount of GPUs that we consume is about 10 to 20 times less than how...

34:11Got it. 5%, 10 % of them, what they're using. Startups and small businesses, listen up. You want a CRM that neatly organizes all your customer data so that you can avoid missed opportunities and you can deliver a personalized service. Rigid CRMs can adapt to your fast-growing needs, and that's where Ateo comes in. ATTIO delivers the goods. It's a custom CRM that's flexible and deeply intuitive. Ateo is built for the modern company, headed into the next era of businesses. It connects your data sources, adjusts easily to your specific setup, and suits any business approach, whether it's self-serve or sales-driven.

34:49Ateo automatically enriches all your contacts. Think about that. You might be missing a first name, a last name, an email, an address, all that stuff. It's going to sync your emails and calendars. It's going to enrich those contacts, and it's going to give you powerful reports. It's also going to let you quickly build zappier style automations. If this, then that type of automations. The next generation deserves more than a one size fits all CRM. Join 11 labs, replicate modal and more, and get ready to scale your startup to the next level. Head to adio.com slash twist, and you'll get 15 % off your first year.

35:21That's A-T-T-I-O dot com slash twist. And so talk to me about data, because it does seem like this is the next big shoe to drop. Licensing data, balkanization of data. Hey, maybe Reddit is available to Gemini, but not OpenAI. Twitter now is, you know, closing up access or X.com is closed up access for people. um and it and the new york times is in a lawsuit with open ai which obviously trained on their data without permission how do you see all of this resolving itself because obviously people are rightfully saying hey i own this data i have the archive of the new york times or i'm disney i own this archive of ip from star wars to marvel or i'm an author and i have these books how do you see all this shaping up in the coming years?

36:12Because is that going to be the limitation that the data you have access to? Or is it going to be synthetic data rules the day and you're going to be able to just make your own data to train on? How do you see all this unfolding? Yeah, definitely. I believe like at the end of the day, I think the data providers, they should be incentivized to provide their data and they should know they should know that their data is being used basically like you you need to have a payment scheme basically for people that you're using their data in your what should that be in your mind how would that work do you have any ideas we haven't we haven't gotten there yet like i think i think this would be like a challenge to to to think about but at the moment what we're trying to do is basically the way everybody does like we're basically purchasing data purchasing data right like you're basically paying for the data that you use in order to be able to you know like to legally you believe this is a good idea because it will keep people making data so journalists artists writers thinkers you believe hey this is a a fair deal here some sort of licensing arrangement where they get paid some reasonable fee to train your models or train Claude's models or OpenAI's models or Google's models, yeah?

37:30100%. The reason being, say, for example, a content creator on YouTube, right? So if people come and look at their content, basically, you know, like, and they get inspired to build something off of that, you see, so AI is also like basically doing the same thing, right? It's looking at the data that is basically available. And it's getting inspired by that data, if it's not directly the copy of that data, right? And that scheme of how we are doing it through, like, let's say, social media kind of channels, right? It has to happen like very similar ways that we can incentivize users of social medias, or users of AI, or providers of data for AI systems, to also like have this understanding of this is basically the same thing, the same kind of scheme can actually apply here they might be analogies here but again like you really have to be systematic systematically going after this problem which is uh one of the one of the main main issues like as we're thinking about the scaling our company it does feel like it's fair if somebody's put a lot of work into it that if an ai was built on top of the new york times corpus that yes they would have permission to do that because it is something that you could partner with the new york times as opposed to open AI and build this with them and monetize it with them.

38:45And it's their opportunity to create an AI based on the New York Times data, not open AIs or Gemini's. Everybody should have the ability to opt into these things. So I feel like that's a pretty smart approach that you're taking. How long before people will be able to use your platform and swap out Gemini or swap out Claude or swap out open AI for yours? For liquid AI? So, I mean, the first batch of products that are coming is basically already in use with some of the clients. It's a developer package, as I told you, for solving AI problems. Like this could be, let's say, like you have a predictive task where you have like video data from surgical kind of processes.

39:27And at the output, you want to predict basically what phase of surgery we are in, for example. That's a kind of case study where a developer can take our package and then basically use our system in that kind of real world application to solve that task. This is already ready and it's available to some of the enterprises through our system integrator contracts and through directly with some of them we are already working like in the financial sector, in the medical sector, in the healthcare and biotech. We have been like very active and automotive. Okay, this is already available. What's your definition of AGI?

40:04How do you determine that a system is generally artificially intelligent? Do you have, I mean, you must have heard a million of these different ones when you're at MIT, and there's a big debate around it, but what do you think? I think for AGI, I think that I just want to stick to something that we can actually still understand and talk about. For example, a system that is beyond human capable can perform beyond human capabilities, given the same resources. That means if I'm provided that the same kind of resources is provided to the to the human and to it to the AI system, the AI system is being able to perform that task better or orders of magnitude better than humans.

40:47Got it. So given the same resources, we both have access to the internet. We both have broadband. Can I beat this system at chess? No. Okay. But it would be at a new game that just came out today. Could it beat me? I guess is the question. Exactly. And in the AGI can exist in a virtual world as well. Like as you were mentioning, these are possibilities that are inside a virtual kind of existence, existing in an internet kind of system. But in real world, you need to have also embodiment. So that's why a lot of work is actually going towards, you know, like the humanoid kind of movements of robots.

41:26we are building humanoid kind of robots in OpenAI figure the new works that are going on at MIT there are many many people working on humanoid kind of research and also other types of AI systems that you can integrate in the society in a safe way so the point is there's virtual we know that those are creeping up like getting an answer to a legal question or making a marketing plan or writing something you know and obviously chess and verticalized games go it's crushing humans but it's got to be able to translate into the real world and if it's going to be doing picking strawberries we're going to need a robotic arm we're going to need computer vision but all those things seem to be aligning so a robot we had a company called root ai which i think actually had some of its origins at MIT as well with the robotic hands, being able to pick strawberries in the real world better than a human, faster, pick the right ones, not crush them, put them in a box.

42:29I think we're kind of there today. We're pretty close to it. For those kind of applications, yeah. But think about for an application of, I want to have a robotic soccer team or a basketball team. Can we have those kind of things, right? That's a level of fine motor skill. probably not yes yeah so when do you think we hit agi in your definition that it's able to beat a human uh at any task could be basketball could be cooking i think i think then the next two to five years is going to be very very exciting and i think we are going to see like uh leaps in in performance of these models as the size of the models are growing i would say uh we might actually see, you know, first versions of it, like very soon, I would say maybe after 100 trillions of parameters, this is where in terms of number of capacity in terms of number of parameters, we would be equivalent to a human kind of the amount that is available to what is that two more boosts of 10x.

43:32So we have like two more boosts of entropic training there, cloud, cloud four and five probably. so yeah somewhere around clawed five or chat gpt six something in that range of jumps two more jumps which might take another you said two to five years we get some what feels like smarter than any human on the planet i that was mine like smarter than any human on the planet able to be any human on the planet at any test now robotics might be hard because you do have some physical find motor skills that basketball and soccer seem out there but you know to work in a factory or to cook maybe it does work pretty quickly um yeah how do you think about job destruction societal changes you know this is always something that folks in your career and coming out of mit you know debate late at night when you're having drinks or whatever you're imbibing whatever the vibes are what do you when you're sort of off duty talking with people who are building this stuff what do you how do you think about retiring a whole swath of jobs that are arduous and painful but that also do provide meaning and purpose to some degree or employment generally for humans working in a factory picking strawberries writing marketing copy all this stuff seems to be at risk so how do you think about job destruction what's the back channel on this is it coming fast and furious?

44:58Or do you think we're going to be able to manage it as a species? I think we can manage it. Like any technology that comes in, I would say it's going to be disruptive. Like you can begin to think about like the evolution of technology in all the things that are in our hands. And it changed the type of the jobs that you would be actually having. But it's not going to like replace because right now, you can use these systems as an assistant. In some sense, I think I think that this AI revolution, this one in particular, is helping us to evolve into a better versions of ourselves. Like every kind of application that today you see in generative AI enables is like in the productivity space, right?

45:33So it's increased productivity. We can do things faster. We can build things faster because of AI. And I feel like this is going to be the trend, you know, and we're going to frame basically AI systems for basically helping us to become the better versions of ourselves and get things done faster. For me, the moment that I'm dreaming of happening is the fact that when AIs can actually solve new physics and new mathematics, new science, right? Like if AI can discover new math, I want to give an AI system basically the Einstein's equation, Maxwell's equation, and the theory of everything that cosmologists are working on.

46:13if I want to give them there and tell the AI system, hey, continue from here and go figure out what's next and that is going to happen. Now, if you solve physics, then you can solve basically the way we build structures, like the way we do science. If you solve mathematics, you can solve the economy of the world. If you solve humanitarian sciences, like the conflicts that we would have, we might actually have AI helping governments basically solve conflicts. You know, there might be so many use cases of AI enabling like new opportunities for work. But this is how I see AI helping us as an assistant, as an as an elevator of the way we live.

46:56yeah this is i think the most positive spin on it which is hey yeah you might get rid of some arduous jobs just like we got rid of being a phone operator like people used to have that job people used to work in the mailroom i remember when i was starting my career in the 90s working in the mail room or being a bike messenger was like a major career like you there were many jobs that you could do and you get paid really well bike messengers got paid a sick amount of money in new york to run documents back and forth for law firms from wall street to midtown and they don't exist anymore for all intents and purposes uh you don't have to run documents because you obviously the fax machine and email changed that forever but yeah you're right like you know what if we could actually solve existential problems or you know science problems around clean energy around farming around calories around health you know maybe we just live with massive abundance and And I think that's what people have to keep in mind.

47:51It was like this short-term look at it on the Jon Stewart show. I don't know if you saw that trending. What was your take on the Jon Stewart take that like, oh my God, we're just doing job destruction here. I got a little cameo in there because I was interviewing Brian from Airbnb. And he was talking about like, hey, we're not going to need a bunch of customer support people answering repetitive questions. Which, I don't know if that's a great career or not. I don't know if people, and there's some people who love being in customer support because they like interacting with people. But maybe it's not a great job.

48:19long term i think we just get better choices like as as a species like you you would basically have a choice to to interact with more with humans right and because let's say for a customer support job right what why a person would be interested in that job i would say the human aspect of it right yeah i like i like to talk to people i like to interact with humans you can do that in the in the process of ai just in a different way it might actually be less involved than than how you have to do it or you're forced to basically do it like for for that kind of human interaction i would say ai and and intelligence in general is giving us choice choice is like what is an important kind of element uh of of human civilization as well like the way the way the way we evolve actually became like this kind of the most powerful species in the world is by the fact that we have a lot of choice like choices are integrated in our in our site and the ability to have choice I think again as I always say like I'm going back to this of course AI would have like you know like downsides and upsides not all green and everything yeah but I think that the right version of AI is going to be extremely useful.

49:37What do I mean by the right version of AI? One of the things that is concerning is making today's AI systems larger and larger as black boxes. If you don't understand what you're doing with a system, that system is, no matter how much control, you're losing control. You're not going to have a lot of control in the system. the fact that everybody like entropic is actually putting like 20 of their workforce on on on explainability right so explain what this means for people who don't understand because this is a topic that i think is super important and underreported on understanding what the machine is doing it's hard for people to believe that people don't actually understand what the neural networks are doing so take a minute to explain this to folks yeah definitely So let's first define what do I mean by explanation.

50:29What do I mean when I say I can explain a system? I tell you the equation that I think most of your audience would actually be able to relate to. E is equal to mc squared. That's the Einstein's equation, right? May I ask you this? Do you think this equation is explainable? That means what? That means like if I have an object and I know the mass of this object, and we know that if this object is moving with the speed of light, then you can compute the energy that it would dissipate at that comfort. Yeah, you can explain this. Yes. You can explain it in full. It's explainable across time. Like it's basically like at any given point in time, if I just give you this equation, this is called a physics equation or physical model.

51:22Okay. This is the best type of modeling framework that scientists has ever designed. A physical model is a model that is completely 100 % explainable. And it explains a kind of reality that you can relate to. Right. On the other side of all. In reality. That's it. Exactly. On the other side of the spectrum, you have a statistical models. I said physical models, and now we have a statistical models. Statistical models are not 100 % explain the behavior of a system, but they observe data. And from data, they infer what is basically the construct of this topic that I'm modeling. Let's say a chat GPT.

52:10Chat GPT is a statistical model. Okay, it's guessing the next word it's guessing. Figuring out what the next thing in this thread should be. Just by observing data, right? Yes, because easy equal to MC squared. It doesn't need data anymore. It's explainable. You just need to plug in your data and it will always give you like the answer, you know, but if you were to say the quick brown Fox jumped over the lazy dog, this is something there's a get probabilistic kind of thing, you know, like you have to see whether do This is a statistical model. ChatGPT and systems like that are statistical models.

52:48Now, scale these statistical models into billions of parameters as well. This becomes today's AI systems. Today's AI systems are black boxes because of the fact that we cannot really understand why, if there is an input coming in and an output is getting generated, why this output is getting generated. There is no explanation to why this input output. No citation to a source. Exactly. What's the source material? Explain your work is or show your work is what people tend to do in PhDs, right? And in graduate school, you have to show your work. How did you come to this conclusion? You can't just solve the math equation.

53:31You got to show us how you solved it. So we get an idea of that. And in these neural networks, people have not been doing that. Exactly. And now we are basically hopelessly basically trying, there is a term called mechanistic interpretability. Mechanistic interpretability tries to point into a part of a system, a gigantic system, and tries to say, based on this interaction here, I suspect that this method is basically doing what? You know, this part of the system is, you know, responsible for biases in my system or something. Now, in the middle of these two spectrums that I plotted for you, okay, so I told you there's a statistical models and physical models.

54:14In the middle, there is a set of models, which we call causal models. Okay. Okay. Causation. Yeah. Exactly. That means like X implies Y. And if X, X implies Y, then what? You know, like basically like more structure into the way you're designing learning systems. what i understood from the liquid neural network kind of thing and actually i proved theories around this thing like in my phd thesis is that liquid neural networks are dynamic causal models there are one step ahead of the statistical models that means you can understand to some extent the behavior of what goes in and comes out and you can explain a little bit about the cause effect of tasks inside the system not 100 but to a really good extent compared to this statistical because they're simpler they're more basic exactly they're more basic and the math itself is kind of tractable the mass itself is like something that you can you can um you know you as as a as an as a technical person you you would you would be able to understand the machine now when i was telling you that we want to design the mission of liquid eyes basically is to design AI systems that we can understand and efficiently deploy in our society because we understand the math behind our systems.

55:40It's not like a transformer architecture that I just take it and scale it. Because it scales, it gives your eyes to very nice capabilities as a black box. But now we are designing systems that are kind of white boxes that at every step of the go, we have a lot more control into how these how these AI systems are doing decision making. Yeah, exactly. Exactly. And this is where like, I think there are some weird incentives to take the time to slow down. If you're open AI, Anthropic or Gemini, you're working on some big project to slow down and say, Hey, we don't want to make this model bigger until we understand it a little bit better.

56:21There's a perverse incentives here in capitalism. And in this race to see who can get to AGI first, or who can monetize this first, and get their next version out opening I 567, you know, Claude version 456 Gemini, whatever. Is there not an incentive to not slow down and not understand it? Like why put engineers if you're putting 20 % of them? Why not put 0 % of them on explanations? And, you know, explainability? Why do explainability when you could just, you know put more servers on and get more data and and beat everybody else that that's the perverse issue here right the alignment of incentives but i know i know what's their incentive like let's say what's the capital for agi the the the market cap of agi is 600 trillion yeah i mean it would be the market cap of human existence the world that's it's the world that's the market so that's it so that's where these companies are heading at you know so the market like if you have agi as you said like you can solve the energy problem you can solve once you solve the energy problem like what i mean you are basically the most valuable company on earth you know like think about that i mean if you can solve economy like if you can solve uh politics basically like the structure of governments you know this is the thing that we are hoping to get and there's a race everybody gets there at the same time like agi feels no you don't you feel some people will get to agi first first yes yeah yes of course like a lot of people have i mean of course like i would say open ai and entropic would be the first bets that i would say both of them i don't know which one first but i think they have a head start and they have uh in a lot of kind of information in house to to to get there i don't know about google i don't know where where their where their um priorities are but i think the two companies that are focused on really scaling ai systems into more and more kind of uh powerful beings, I would say at this point, I think it's going to be an traffic and opening.

58:22But they both have the they're both taking the approach that you can just use their system to build whatever you want on top of it. So of course, if it's open, like, it's not open, but it's available to people to pay for it. So then if there was the ability to, I don't know, figure out which stocks going to go up, you might have 1000 developers realize Claude and hoping to hire great at this i'm going to make the best trader in the world to go trade socks that's true but that's why that's why the release of those kind of huge models is still it by itself is actually like a huge challenge i would say today we haven't seen those kind of systems yet but the systems that are coming in the next two years as i was saying those systems even the release of those systems to public it has to be a rollout it has to be like a trial and error like we really have to see how what's the reaction we internally like these systems getting massively tested you know like it's not like they're just they today they get it and then tomorrow they enable it to to to everybody to get access to right like you need to do a lot of testing of the system to see the capabilities how how they come about do you think that's why there was that chaos at open ai is that maybe they felt like that next version was getting close and that's why there was sort of chaos because there was that whole sort of speculation like maybe they did feel like this thing was getting you know agis let's say do you think they're i don't know i i don't know i i seriously don't know like because i mean it's uh it's all behind closed doors i i i really just don't know the only thing i would say is like it might look like more of a more of a conflict like just just on mission i would say yeah as as opposed to like how um if the AGI has been achieved or not, you know?

1:00:11Yeah. And this is the open source models seem to be doing pretty strong as well. Do you think open source wins the day or do you think open source can keep up with the closed systems or no? The unfortunate answer is no, because the closed models are usually like a lot of resources are going in. A lot of concentrated resources is thrown out closed source modes. that's like that's just a simple allocation kind of task you know just think about like resource allocation like the massive concentration of resources and in the hand of like open ai and nvidia itself like you know google microsoft like all of these companies right so that alone is also slows down open source open source is going to always play the catch-up and then some the gap between the closed source capabilities and open source actually grows as well you know that's also another thing so i don't think the gap is shrinking so unless there's going to be an open source move on you know like a more facebook you know all the open source to their credit is moving you know it has moved to open source models apple's doing open source models so it's going to be really interesting to see if either of these can catch heat delayed open source think about how llama 2 llama 2 came out delayed open source is again the same story right the llama two came out as a commercial license first right and then they decided to open source it now let's see how llama three is getting uh released so it is it is important also like to think about timing on the open source like moves you know it is true that some companies are just putting out like for example mr also played like a amazing role in the open source kind of community right like they they put a model out but then immediately they they put the more powerful models like behind the paywall, right?

1:01:54So you have to, you always have to think about like, what, what is happening in the game. And I would say the unfortunate truth is the fact that closer smalls are really amazing. All right. So I think you're hiring, and things are going pretty well for the firm. If people want to join the firm, where can they learn more and come join the liquid team? Yeah, liquid.ai, basically, like there's a get involved section where you can. Today, we have like around 25 uh smartest people on earth i would say it's a really crazy concentration of people we have people with uh olympiad medalists in the team like we are people that are solving literally like really complex problems for us we have on the team like inventors of very important ai technologies and uh we have uh good philosophers also in house we have uh Joshua Bach also was part of our organization.

1:02:51And, you know, like it's always like it's a privilege for myself to work with such an amazing team of talent because this has been the power of Liquid AI. We have been like very good at bringing in like key players into the space to build like something from scratch, a kind of white box kind of intelligence, and then hopefully scale it into something that is meaningful. And again, we are obviously hiring as well. And, uh, we'll continue success with it. And thanks for sharing this crazy vision and, uh, you know, be thoughtful about releasing this stuff. Let's not end the world. Let's make life awesome for everybody.

1:03:29And we'll see you all next time on this week. And start by great job so much.