This AI Grows a Brain During Training (Pathway’s AI w/ Zuzanna Stamirowska)

12:06Corey Noles:when you switch the perspective.

12:08Grant Harvey:Yeah, things kind of start to fall into place. So, I mean, yes, we believe that there are just, you know, some things that we need to roll back to. I mean, Transformer opens, it's an amazing, absolutely amazing innovation which opened the entire market and actually it's done two things. One is a technological innovation, right? Yeah. Scientific and technological innovation. Second, with the go-to market that happened, it managed to tickle the imaginations of everybody. And this is huge for scientific innovation. Just think about this. Oh, yeah. But we are still early in this kind of AI market shift.

12:48Grant Harvey:So, so far, I mean, 0.7 % of GDP was spent on this AI technological shift. If you compare it to other such shifts that, you know, like in the past century, I mean, just the telecom in the 90s took over 2 % of GDP to be accomplished. Wow. And I'd say that probably AI is more fundamental, right? Yeah, me too. So we're super early and well, Transformers most likely not. I mean, as Bradford would say, it's not the ultimate technology to get us all the way through it. And yeah, we need something else. There's a lot to be done.

13:29Corey Noles:All right. So we have to ask about the name Baby Dragon Hatchling. How did you come up with it? And I have a theory, but I want to hear the actual explanation and I'll tell you my theory.

13:39Grant Harvey:This is like Dragon Hatchling. So in the paper, like specifically, it's Dragon Hatchling and the abbreviation is BDH. And of course, the question is where does the B come from? Dragon Hatchling per se is coming from Terry Pratchett's The Color of Magic.

13:55Corey Noles:I love it. Okay, cool. I have not read that one, actually. I need to.

14:01Grant Harvey:I strongly recommend it. And I mean, this book is worth at least 10 business books, I think, that you could find out there for so many different reasons. Dragons being probably even a smaller one. But I mean, specifically in The Color of Magic, there are dragons that appear more the more you think about them. And it was actually very funny for, I mean, we found it funny for reasoning models, you know, because we literally had to reason about reasoning to build a reasoning architecture. Actually. So the dragon started to appear. Yeah, yeah, exactly. And well, it's like what you see publicly, right, is an architecture in the paper.

14:46Grant Harvey:So this is a hatchling and this is it.

14:48Zuzanna Stamirowska:I love it.

14:49Grant Harvey:And then, yeah, we do get some questions about why BDH. And the truth is, and everybody tries to put something that would naturally fit the B. The very simple truth is that, I mean, I just thought that AI dudes really like three-letter acronyms. I agree. They're also kind of...

15:08Corey Noles:You're not wrong.

15:09Grant Harvey:Easy to pronounce and it worked. But I had one person, a physicist, who came to our office and he said, listen, I read your entire paper. I read everything. And because I really... I think I still need to read the appendix because I still don't know where the B is coming from. And you had to explain it to yourself. I'm like, well played, well played. Oh, I love it.

15:32Corey Noles:Is it, so is the B because it's a small version and you're going to grow it?

15:38Grant Harvey:So to be perfectly honest, it's like the most truthful explanation is really just the three-letter acronyms. Yeah, oh, gotcha. The B per se comes from the fact that like the model working on, like the working name is Baby Dragon. Yeah. So architecture is Dragon Hatchling and then you have Baby Dragon because it's already, you know, somehow grown. And it was just inherited, like, it inherited the bee because our internal name was Baby Dragon. And then, you know, we do have some dragons flying around the lab, so... I love it. We even have a random name generation for dragons. Oh, that's amazing.

16:12Corey Noles:Like, in Dracoric? Like, how nerdy are we getting here? Like...

16:17Grant Harvey:Oh, no, no, no. I, like, we literally have an LLM dude because we have versions whenever you have you have versions you're getting you two no test model like one thing against the other and stuff so we have we literally have a random dragon names generator that's cool I love it yeah my my theory uh was

16:33Corey Noles:that if this is truly continual learning it's kind of a dragon in a sense because it could be very powerful and dangerous if we're not careful but I imagine it's more like a dragon in Game of thrones where they're controllable um so i guess the question is you know one we'd love to know how how it works and and two you know if it is continual learning how do you control something

16:57Grant Harvey:that's potentially powerful you actually hit something here as well because part of why we venture towards dragons at all is that well this is a mythical creature right yeah uh that nobody believed it could exist this is where okay we're in the business of building dragons uh so i think It was a realization we had very early on. And indeed, we're talking about continual learning, in time, long horizon reasoning, and adaptation over time, right? To new data, new learnings, et cetera. So the way it works, it's actually a bit like a brain that works on silicon. So a bit like a brain, I mean, there's a concept, this is very nerdy, the concept called Hebbian Learning.

17:44Grant Harvey:It's like a very simple principles of how the brain works. So I guess you all can imagine that the brain has neurons, which are like little cells, right? And then connections between neurons, which are synapses. And this is kind of it. So we have kind of dots or like neurons and then links between the neurons. And this is like a big network that we have in our heads. And this is a very simple model because, you know, somebody who's like a neuroscientist or a biologist would say, well, yeah, but you have all those chemical reactions, all this and that. I mean, here we're really just looking at the very stripped down, very basic, very simple, you know, structure.

18:18Grant Harvey:Like, we know that birds fly and they have wings, right? Yeah. We don't need to know how they've really, like, moved, how many bones they have in them and stuff. No, it's just literally something to fly in to have, like, the surface of wings. And there we go. Right. So we're looking at those kind of little neurons, almost like particles, that entities, and they have links between each other. and they're actually passing signals between them. Oh, wow. So this is why you have connections, you have this structure. This structure, we know, has to be dramatically efficient. Right. Why? Why? Because, well, our heads are somewhat limited in space.

19:00Grant Harvey:We walk on two feet and we kind of fall over, so our brains kind of get larger.

19:05Corey Noles:Yeah. Right.

19:07Grant Harvey:So it has to be very efficient. we know it is very efficient in terms of power but it does offer this kind of capabilities of lifelong learning keeping kind of very like infinite context pretty much so we know that there exists a physical system that is capable of doing those kind of dragon like things it's not fully impossible so this we know question is how to make it work and especially how to make it work on the hardware that we have right now. Right. And you always have to work with the hardware that you have, with the materials that are possible whenever we see big technological shifts. I mean, it's usually some sort of inflection points where many things come together.

19:54Grant Harvey:I mean, so much compute with this algorithm, all of a sudden this gives us a boom. So what we did is we looked a little bit at Transformer and thought like, okay what is it really missing to like from from the brain right like to to to get closer to the brain and then and then yeah that was actually adrian you know our um our chief scientific officer who who went who went on this journey like literally like with very strong conviction that it has to be local interactions looking at the brain i mean we have to have those like small particles and our model like our architecture bdh the way it works is that you really have small neurons neurons are connected whenever you have a new bit of information like as folks call it tokens but you have some new information coming in only the neurons that are interested and connected light up okay so one neuron gets information passes it on to its neighbors those with whom he's connected not everybody not everybody lights up just the neighbors if they care enough about this thing, they light up as well.

21:04Grant Harvey:So this is the principle of neurons, you know, who are connected to them. They fire together.

21:11Zuzanna Stamirowska:Does that vary based on how connected it is? For example, the idea being that some information would be more tightly tied to point X?

21:22Grant Harvey:Yeah. Okay. Yeah. And actually, but this structure emerges naturally. We don't set it. Wow. It just comes from data. It emerges. We actually saw in our lab, this was an amazing moment. We actually saw the emergence of just this kind of brain appearing.

21:38Zuzanna Stamirowska:It was like, whoa.

21:39Grant Harvey:I remember we all, it was like late in the evening, and we all just rushed into the office because we were doing something else. And then Adrian just calls us, hey, look at this. And then I see the brain. And he's like, whoa. And I remember that's my brother.

21:53Corey Noles:I almost have chills thinking about that. It's just like learning that on its own.

21:57Grant Harvey:I remember I immediately texted my brothers and investors and I, look at this.

22:03Zuzanna Stamirowska:Oh, my God. Yeah. That's awesome.

22:06Grant Harvey:It was huge. I mean, emergence also for complexity scientists, right? Emergence is what we love to see. This is a spontaneous order. You think that something's just so random doing, the gods know what, but you can strip it down to such very simple fundamental rules that, yeah, you will see this larger order appearing. And this is what we got, like the structure of the brain somehow appearing naturally from those very local, honestly, message passing between neurons. As we do it on social networks, for example, we say something to our friends, right? Imagine this rumor spreading dynamics. This is how kind of learning works here.

22:45Grant Harvey:So whenever two neurons were interested by something, the connection between them becomes stronger. and this is memory yeah

22:58Corey Noles:that's right because that's sort of like how the hippocampus works right where it's like I'm going to do a terrible job

23:06Grant Harvey:you use it more it becomes stronger I mean this is just a principle and it's only positive activation so there's no like positive and negative it's only positive it gets stronger something is not used you know over time it will start fading um but but journey speaking you the connections that were useful become stronger and and this is kind of it and then i mean this structure is you know it's actually very efficient because it's like a brain so it's computationally efficient it distributes nicely it gives so many nice properties that unlock a number of things you know that then for for us even from the engineering standpoint in terms of how it scales how it distributes how you can run it on many machines, etc.

23:51Grant Harvey:But it's like a scale-free graph structure. So point is, even if we go beyond the scales that we've seen in data and tests, we scientifically know how it will behave. It's very different from the transformer, at least as we see it now, because before transformer it hasn't been studied, it would be difficult to study. For this, because we know how the emergence works, we know that, I mean, Yeah, it's scale-free, same laws we'll be holding, you know, above what we've seen in tests and kind of data until now.

24:28Corey Noles:Does that mean that it's also more interpretable at some level? Like you can kind of understand what it's going to do or no?

24:35Grant Harvey:Yes, in a way. So specifically, we do see very precisely neural activity. And because we see the neurons when they care about something, right? We just see them.

24:45Zuzanna Stamirowska:Yeah, right, right.

24:45Grant Harvey:So like for LLMs right now, for transformers, I've also tried to build MRI machines to scan the brain, whereas we sort of have a CCTV inside of the brain.

24:56Zuzanna Stamirowska:I was just going to ask you about that. I remembered you making the CCTV analogy when we talked recently. What does it look like?

25:04Corey Noles:Literally, we see the neurons that fire up when they fire up on something.

25:08Grant Harvey:So in the paper, we're showing things that fire, like synapses and neurons that fire up for the notion of currency. And you have like a dollar one, right? That you fire up, firing up. Of course, there is two scientists who may be listening. As an end of the system, there is compression of information while learning. So, you know, it's not always super clear. Like some concepts may be fairly large and you will have like a lot of things firing up, for example. But dream speaking, we see this neural activity and we see even more. We see neurons getting bored. so when you keep on repeating something to them uh we just see their activity yeah whatever uh just going down in learning like think about this especially as you go get older i mean kids get learn very quickly but also because everything is new to them everything is a surprise it's worth learning um with as we get older like things you know we stop thinking about many things We stop noticing them because they're just so obvious.

26:11Grant Harvey:I mean, our connections are strong enough, let's say, for that one thing, like eating soap or whatever. We shouldn't be eating soap. Bad idea. So there's this element of surprise that actually somehow shows that something is valuable and worth remembering. So it was actually for us very funny to see this surprise effect literally on your activity.

26:32Zuzanna Stamirowska:So will it, in the same way that the brain over time, if there are areas that are not being used that they can weaken, will the same thing happen in a model?

26:46Corey Noles:And if that's a dumb question, please say so.

26:49Grant Harvey:No, no. So, I mean, yeah, actually, so you'd be getting some sort of like fading connections that they're not used for very long. But this is more a topic of, okay, how to transfer also to long-term memory, right? Yeah. So, yes, because there are some things that, again, it's not to work like a database, right? Okay. If there's a database, then in deployment, this is something you plug in. Right. If you want to store absolutely everything forever, right? This is less of a problem. But for reasoning and having, let's say, your space to explore when you reason, you want to build it in such a way that you have the most relevant and compact structures.

27:36Corey Noles:Well, then I guess what I want to know now is, So, you know, you've proven this BDH works at GBT2 scale, last I read, with 1 billion parameters. Is that correct? What's the path to scaling it to, say, 100 billion parameters? What needs to happen to get there or grow larger?

27:54Grant Harvey:Of course, first of all, we do it. There are no reasons actually for it not to scale. And like scaling laws are inherited from the transformer. But there's also no big need to scale. This is not the game of scaling of more parameters and more data, because this is fundamentally not where the value is to come from. The value is to come from faster learning how to solve problems that haven't been seen in the training data. Like, this is what we, this is where we want to get to. And actually, if we can show better learning out of smaller data, well, this is the kind of value that we want to prove.

28:37Grant Harvey:So actually, I hope that very quickly, you know, we'll be more looking at models that are very small but capable of producing results comparable to the big ones.

28:48Corey Noles:Love that.

28:49Grant Harvey:That's awesome. We're not looking at scale and root for scaling. We're looking at this getting better at puzzle solving and reasoning and hopefully in as general way as possible to get it closer to the way that humans reason, work and ultimately innovate. Because if you look at the real innovator, like the best ones that I know because I kind of have them on the team, right? It's not about seeing what's there, but it's seeing what's not there and what could be there.

29:20Corey Noles:Let's say, for example, it's learning something really complicated, like where it's, I mean, you're a complexity scientist. You know more about this than I do, but something like really complicated. does it at some point run out of brain power or how like because i'm used to thinking of you know parameters as this thing that's like oh this is like it can retain a lot more information like and if this thing is just continuously running at what point does it reach its limit of what it can think of or do we just not know that we don't know but i don't think those limits

29:50Grant Harvey:work in this way i don't think i mean right now we do cap the number of neurons that's like the problem number that you may get there you may imagine models where you would be adding them this is all right you could have a model that's that's growing technically um but but it's like right now in transformers it's not the reasoning power doesn't come so much from this size per se okay when you think about this we actually do have a lot of compute power because of how much place we have in the brain because of the structure so these models but if you if you look like the brain now i would need to check again but i mean for the synaptic synaptic connections the brain are in the trillions right and i think it could be yeah like one thousand one some folks would say could be having one thousand trillions or something like this of synaptic connections in the brain, this gives you a lot of memory and a very efficient structure.

30:53Grant Harvey:So effectively, you get to something that operation works like infinite context. I mean, to be super scientifically precise, yes, context and BDH is limited by the size of your brain. So the number of neurons and then connections between them. But this network structure allows you to encode so much. Well, even the human brain fits in a cereal bowl.

31:17Zuzanna Stamirowska:There you go. It's really not that big considering the amount of information it can store and its speed and its capacity, I would think.

31:26Grant Harvey:Exactly. And then when you think about this, you keep your memory close to the core. Actually, exactly at the core. So you don't need to do look-ups for technical people. You don't spend your energy on all of that. You don't need additional compute for this. So it becomes very efficient from this point of view. You have memory directly there. and yeah, it's like in memory on a chip. Okay. Wow, yeah. And then second thing is that you don't fire up the entire model every time, but you fire up only those guys who are connected and who care. Right, that makes sense. And for most cases, yeah, you don't fire up the entire brain.

32:04Grant Harvey:I guess when you fire up the entire, like I don't know how much brain people are using, but I mean, small percentage, right? Yeah. At any time, but that's exactly the point. Why would you be using the full thing? Not for every task, right?

32:17Zuzanna Stamirowska:I don't need that to pick up a soda or to, you know, answer the door and say hello. You know, there's definitely lightweight tasks for sure.

32:26Grant Harvey:Exactly. I mean, you may have some that are, you know, more specialized in something. Cool thing, however, like with our heads, what we're not capable of doing is that we cannot exactly glue two brains together. It would be very difficult. And with those models right now, since like they scale only with a number of neurons, So there's just this one dimension to something we're showing in the paper. We can actually glue two separately trained models together fairly easily. And they become one. So we showed in the paper, we have like one model trained in one language, the other one trained in another language.

33:02Grant Harvey:We just put them together, even without leaving them, you know, to train a bit. They actually start producing sentences that mix up the two languages pretty well. And then, of course, the value will be allowing them to train together a bit more such that they create really like a one unity because they also create links between each other. But yeah, this gluing is a bit like Lego blocks that you can put together. So you could imagine a model trained in finance, another one in legal, you know, in an enterprise. And then you put them together, you get this like super, well, I don't know what the person with, you know, good legal and finance background would be doing.

33:40Grant Harvey:Maybe we shouldn't venture there. Right. Too much power. I guess some superpowers that, I mean, we would definitely want. It's like, you know, us and having Adrian. It's a different story when you have, if I were to have a physicist, a mathematician, and a computer scientist on a team. Sure, cool, but still communication, different intuitions, and all of that. Versus having one person who kind of has all of this in one efficient structure, which is the brain.

34:08Zuzanna Stamirowska:Something I'd like to ask, and this will shift gears a little bit, but I know you already have some early adopters and people who are working with this. Like, I understand NATO, the French Postal Service, and maybe Formula One. Is that correct?

34:21Grant Harvey:So, these are like, we have history of actually working with pretty amazing accounts.

34:26Zuzanna Stamirowska:Wow.

34:28Grant Harvey:But as of now, they don't have the models deployed. They have the Dragon's Nest, if you wish. so I mean you have to understand that there is if you are to bring life intelligence you'd have life data as well they need the way to connect and you need your dragon to feel cozy you know in its environment so those guys I mean as as of now they're using like layers of technology that we have built for the dragon nest to make sure that we actually can feed data at low latency very efficiently and actually do cool things that are necessary once you put such models and live intelligence in production.

35:09Grant Harvey:Yeah. And yes, this is like, of course, cases are pretty cool, right? I mean, sometimes I joke that, I mean, how are we exactly getting only the coolest customers? Like, do we have a cool, cool factor and kind of regeneration? There's a lot of cool there.

35:22Corey Noles:It's the dragon factor.

35:23Zuzanna Stamirowska:I can't think of anyone with more data than what Formula One collects. it's unreal and definitely the cool factor

35:36Grant Harvey:it's really cool for them it's like from the strategy of the race you have so many things that can go wrong every car is like a prototype it can blow up of course NATO I can't talk about this freely but I mean this is very fundamental I imagine that there is if you played any sort of war game like I know harpoon or whatever and just imagine getting real-time information from the field and then kind of informing the strategy and stuff. So this is all cool. And then, you know, boys and girls like buses, ships.

36:13Zuzanna Stamirowska:Oh.

36:13Grant Harvey:Yeah. All of that. And, I mean, you know, we have these complex systems of moving parts and, like, this is how the world works, really, right? We have a lot of moving, living things that are interconnected it and ideally would like to predict what's happening out of it and somehow see the patterns in this chaos, hopefully be able to control it. And yeah, for this, you kind of need a number of technologies that come together. And we do have a nest. That's the point. Our baby dragons have nests.

36:47Zuzanna Stamirowska:That's awesome.

36:48Corey Noles:What's the roadmap then? And, you know, are we thinking we're going to be Lego blocking a bunch of different models together? I mean, as far as the company goes, like, where do you see this going in production?

37:01Grant Harvey:Yeah, so actually, just like a week ago, we announced a partnership with NVIDIA and AWS at ReInvent. Yes. So point is, the moment we're ready, it's going to be available to AWS customers kind of right away, all built on the infrastructure and kind of made in such a way that it will be easy to plug in and test it and adopt it. So this is one. I mean, this should happen sometime next year. So this is it. And then, I mean, we have our own roadmap. I mean, we believe we're on a faster way to AGI and working towards getting there as fast as possible and keeping all the focus on this.

37:42Zuzanna Stamirowska:On a philosophical level, let's go a different turn. Do you see this approach as, and you just mentioned AGI, as a step toward general intelligence as a whole? Are there any safeguards or things along those lines that you're thinking you might need, as it begins to think more like a brain, operate like a brain?

38:05Grant Harvey:First of all, I think it's important to say that we're looking at reasoning and we see reasoning as a primary function of intelligence. LLM says we've seen them for like literally language models, right? As we've seen them for child code use cases, summarization, some sort of like search. I mean, these are great use cases and brought us here, but they are not the primary function of intelligence. And we would say that reasoning is the primary function of intelligence. This is not just us, actually. I think by now, everybody from all the labs would tell you that reasoning is this. Yes. So we're looking at reasoning, then with reasoning capability of solving and inventing very tough problems.

38:50Grant Harvey:So the North Star is to get to an innovator who sees what's not there, as opposed to just seeing what was there and recomposing. Right. Yeah. So that would be the true generalization. And then you have generalization over time. So going towards solving problems in environments and over like periods of time that were not tested, that were not seen in data, etc. In terms of safety, I think I'm not yet there to give very definite answers. But I'd say that part of what we're doing is actually gaining a way better scientific understanding of how the models are working and why. For us, mapping, like very precisely mapping, understanding this passage from micro interactions and having something that we know we call the equations of reasoning.

39:39Grant Harvey:to having a scape-free structure of known laws that govern it, this is kind of important and it's important for safety. Then some discussions that we have, let's say more internally, are around getting to provable risk levels of how such system will behave and if it will be kind of do-what-I-mean-machine or not. because so the risk the risks that are we're looking at that are you know more controllable maybe for us is just making sure that yeah these models will not just out or just by themselves won't venture into doing something just completely silly because they're hallucinating right something so off policy and it will be just ridiculous because they would stop working like a predictable human

40:31Zuzanna Stamirowska:yeah because if you think

40:33Grant Harvey:about this if you're hiring someone you observe them for what couple of hours max yeah you see you kind of see their credentials you know how they were kind of taught at Princeton probably same curriculum

40:48Corey Noles:you know

40:49Grant Harvey:you get them and you give them tasks and you safely assume that they will perform in a certain way right without getting completely crazy and blowing up the planet earth yeah And this is a fair assumption. This is something we should get with AI, right? Agreed. So at least mathematically, we should have some sort of, you know, comfort that, okay, we know how those Princeton graduates function. And then, however, we still don't control the objectives, right? So if somebody puts AI to a bad objective, I mean, this is something that, I mean, as a startup for now, I mean, I have no influence over.

41:33Grant Harvey:this is definitely something that as we get closer to HDI we'll have to be resolved at many levels and I'm sure we won't be the only voice to take part in this discussion, I hope we'll actually have something to say

41:48Corey Noles:How do you prevent it from learning something you don't want it to learn? And maybe you're not at the point where you can't do that yet

41:55Grant Harvey:No, this is actually not too bad I think the easiest is that you can roll back to a checkpoint Nice. So this is fairly simple.

42:04Zuzanna Stamirowska:And you have observability that traditional LLMs don't have, essentially, with your CCTV.

42:10Grant Harvey:Knowing that physicists, I mean, it looks like a cascade. Imagine like an epidemic spreading in a graph, right? So you have a system, you have an epidemic spreading. This is how information is out of spreading, right? So if you have a very small cascade, very small epidemics, you probably don't care because it wasn't relevant enough for the model. potentially. But if it's small, you can still reverse it.

42:36Corey Noles:You can literally reverse it. You could quarantine it, essentially. But if it's big,

42:42Grant Harvey:this is when also the information is very relevant and you get to a point when you wouldn't be even able to say where this came from if you were somewhere in this graph looking around, you wouldn't be able to say where it came from. Then it's non-reversible per se. Just from a physics point of view. So, yeah, but you roll back to checkpoint and you could just checkpoint your kind of a little over time. So, you know, if you infuse something you didn't want to have in the data and the model, you take it out.

43:13Zuzanna Stamirowska:What is the one AI capability that you're most excited to unlock with BDH that isn't possible now?

43:22Corey Noles:With the current architecture.

43:23Zuzanna Stamirowska:Yeah. Yeah, with Transformers specifically.

43:26Grant Harvey:This is very easy, like generalization. Yeah. But I mean, for me... True generalization. True generalization.

43:33Corey Noles:Some would argue that it can kind of generalize, but...

43:36Grant Harvey:No, true generalization. So getting to innovator level. I mean, why? I mean, I think there is a path towards space exploration for us.

43:47Corey Noles:As in like your models in space?

43:49Grant Harvey:No, I mean, although I do know a guy who's kind of working TPUs in space. This is so cool.

43:56Zuzanna Stamirowska:That is cool.

43:57Corey Noles:That is cool.

43:58Grant Harvey:Yeah.

44:01Corey Noles:You mean the space more abstractly?

44:04Grant Harvey:Yeah, I mean TPUs in space, like where we'll be building data centers.

44:08Zuzanna Stamirowska:Hey, in space.

44:09Grant Harvey:That's a big discussion lately. Yeah, it's a very serious thing, right? And I think timelines are pretty short. And within two years, they assume they will have actually TPUs in space. Wow. So, yeah, but I think to just make space travel actually work, we will need so many scientific unlocks and especially on the energy front. So there's a couple of very fundamental problems in technology and science that we need to crack. And I see AI, in fact, not as an end in itself, but as a crucial tool that will help us lift a number of obstacles that will then allow civilization 2.0. So when I talk to friends, you can compare this transition to maybe the moment when people started agriculture.

45:05Grant Harvey:you know it's like before we're kind of like hurting and moving from places to place and then kind of humanity settled down and then i was able to build culture and civilization uh so like ai the moment we becomes you know everywhere and this powerful will allow us to build civilization

45:25Zuzanna Stamirowska:2.0 you know and we spend a lot of time talking about the next five years the next 10 years the next 20 and we forget that like time's going to go a long way what's going to happen a hundred years a thousand years five thousand years where will where will humanity be as a result of what we're experiencing right now at that point and that's uh i can ramble on that for days but look at the

45:52Grant Harvey:pace yeah sure look at the pace right yeah just i remember this time last year folks asking me for example about, oh, so what will happen in AI next year? And I was saying, well, reasoning models are going to be everywhere and they're going to be the main thing. And people by then didn't exactly know what reasoning models were. Yeah.

46:11Zuzanna Stamirowska:Correct. They look at you like you're crazy when you say it, don't they?

46:14Grant Harvey:Yeah. I mean, I'm happy my predictions work, but thing is just that this space right now, the speed of transition, let's say of shift is like unprecedented. In hardware, I know Railway, that was a huge one that changed the world. I mean, you have huge infrastructure projects that take just so much time to be built, right? So, I mean, it's like so, so slow. Whereas here, it's like from month to month, the landscape is different. I mean, you know it better than anyone because, you know, your job is to really get an understanding and translate it, right? Yeah.

46:50Corey Noles:So hard.

46:51Grant Harvey:Yeah.

46:53Zuzanna Stamirowska:So many times there's a thing that we think of as being way in the past, and then we go and we look at something and we're like, well, that was eight weeks. That was eight weeks to go. You didn't have those two months ago?

47:08Grant Harvey:At my PhD defense, I had a guy from United Nations who was there, and he actually commented and gave me a book entitled that every time I found the meaning of life, they change it.

47:25Zuzanna Stamirowska:So true. Well, Susanna, thank you so much for joining us today. We were both so excited to meet you and look forward to watching your tech grow for many, many years to come. If our viewers want to learn more, how do they go find Pathway?

47:41Grant Harvey:Yeah, so, I mean, please go to our website, pathway.com, and then you can connect with me on Twitter and LinkedIn. And then, yeah, I mean, please follow our research papers. This is what we do. We'll probably also kind of start some sort of, like, blog to get some of these messages kind of closer and maybe take them, you know, out of the paper. Yeah. And, like, paper is pretty long and deep. But, yeah.

48:07Zuzanna Stamirowska:This is it. And guys, thank you so much.

48:08Grant Harvey:This was so much fun.

48:09Zuzanna Stamirowska:Excellent. Thank you for having me. Well, everyone, that wraps up another episode. If you haven't yet, please take just a moment to like and subscribe today's video. That way we can continue bringing you more cutting edge interviews from some of the most fascinating people at the frontier of AI today. Also, don't forget to check out the Neuron newsletter that started all of this and join 600 ,000 or so other people who read it every morning. Thank you again to Zuzanna and Pathway. And that's it for today. Farewell for now, humans.