OpenAI Researchers on the Future of Mathematical Reasoning

26:28And the proof kind of explains where this is coming from. And because you understand this LP bound perfectly, this actually just gives a better bound on delta d. It turns out that this old bound can be kind of reinterpreted in this framework, and what the model does is it shows you the best possible bound you can get by this framework. So the model sort of made the connection. And what is the sort of like... I mean, so I think, so the model gives a function f, which, so first it constructs a function f, which gives you this down. And then it shows that there's no function f which does any better.

27:02So it's inequality, which is quite strong. So we now understand this problem in high dimensions very well. And that's pretty remarkable. And the model was just kind of told like, analyze this linear program in high dimensions. Go have fun. Got it. And to give an indication of how it was known, I think this conjecture was based basically only by doing numerics. Extremely clever numerics, but numeric. And so, yeah, you have to kind of figure out why this is the right thing to aim for, and it does. And that was pretty remarkable. Yeah, I mean, I had actually thought about this problem for about six months at some point when I was a graduate student.

27:42And yeah, I remember making absolutely zero progress on it. So it was very nice to be explained why it was true. So that was a pleasant experience. I think also in general, it was one of these solutions which I knew several people had tried the problem. It's pretty remarkable because like the model solution, especially for this being like the LP can't do better than this, was like quite short. It's a few pages of complex analysis, but it's kind of exactly the right approach. Like once you see it, it's kind of, it's like unbelievable. Like why hasn't somebody done this before? It was like, there are many types of good mathematics, but I think one of them is just like, you see it and you're like, oh man, why didn't I think of this?

28:20And it was really fun. And I mean, I sort of knew why I didn't think of it, but it was quite nice to see it and it was fun to see. That's why I like this problem a lot. Yeah, so this is the first of the 10 problems that Astro saw. But the second is actually closely related. So this was sphere packing. The second one is spherical and binary codes. So what's that? You should draw a code. You drew a packing. It's going to be the same picture. Okay, sure. Otherwise, we're going to have his picture. The hexagonal packing in our minds. Yeah, a spherical code is literally just a sphere packing, but on another sphere.

29:07Yeah, so I mean, a spherical code is basically just a sphere packing on the surface of another sphere. So, yeah. Yeah, it looks like a sphere.

29:22Okay, so same picture as before, except you're kind of on a curved surface. Okay, so why is it called a code? Well, you can... I guess the reason is because of binary codes, which is, again, the same sort of thing, but now it's on a cube. Okay, yeah, fine. Let me draw a picture of a cube and some simplest possible code on it. When you're sending... So this is really about error-correcting codes.

30:01So what are error-correcting codes? So it's like I send you some string of bits, right?

30:13And maybe I'm worried that some of the bits I send you get corrupted, right? So maybe like just because of some errors in my system, like this one gets changed. And we want some communication protocol so that like you can decode this like small amount of error and like recover what I was trying to tell you. And you know, like normal English language kind of has this sort of property, right? If I make a few typos, you're going to be able to understand what I'm saying. But if we have some like really brittle communication scheme, it's not going to work. So codes are kind of the way you solve this.

30:52And mathematically, it just means like, you know, what's a binary string like this is a fixed length. It's like a point on some hypercube. And we want a dictionary of allowable code words that are like separated from each other. So like in this case, if I don't want any two to be adjacent, I would kind of take these four vertices, kind of the like even ones if you sum up the digits, right? And like, okay, I guess, okay, in this case, I guess if I have an error, you can't tell which one it's from, but at least you can tell it's like not, at least you can tell there was an error. Oh, I see, I see.

31:28Yeah, because it's like kind of sparse in the, yeah, it's like it's not too adjacent, so that like, this is like when the hemming distance is not? Yeah, yeah, yeah, right, right. You want, yeah, so you want like a large hemming distance between any distinct in your dictionary. And yeah, I guess if you take two opposite corners, then if I have like a single bit error, I can always like recover which point it was coming from. One that it's definitely closest to. Yeah. So there's kind of the, you know, same question in both of these cases, like in a very high dimensional setting, what kind of rate can you get?

32:00And like for binary codes, it's really like, you know, an extremely practical question. It's sort of like if I send you like an n bit string and there's like, you know, 1 % error rate, Like, how much longer does my message have to become to tolerate that amount of errors? That's like some fundamental information theoretic limit of, like, communication.

32:26And, you know, but you can see, like, certainly this spherical case is, like, it looks very much like sphere packing. For example, if you, like, if you make all these little spheres really small, then, like, the curvature of the big sphere is kind of not going to matter so much, and it looks like just packing spheres in full space. And in fact, yeah, like these problems turned out to be very related. So for these problems, there were similar bounds coming from these KL authors and there's like something for the sphere and something for the cube, but it's all kind of the same stuff. And our models found better bounds for these cases as well.

33:09And, like, I mean, the techniques look pretty different, actually, if you, like, write them out. So this full space analysis of this linear programming was using, like, just complex analysis. But if you, like, the method for these cases, we're using representation theory. Like, both the sphere and the cube have a lot of symmetry. and basically the idea of the proof was to really leverage this symmetry. Like there's some amount of this in the previous like existing method and really the improvement is to like lean into the representation theory like really hard and kind of make the algebraic symmetry like enter in a more sophisticated way.

33:57And then it like turns out that from the representation theory formulas, if you kind of take this like small sphere limit in the spherical code case, you recover part of this result and you recover this value. So this result isn't a special case. You kind of only went one direction of the bound from looking at it from the code's point of view, but there's a very close connection. Okay. Yeah. You guys let this run in parallel, so it's kind of discovering, because you're not sort of feeding it. So actually, this was the one case where there was some interactivity involved. Oh, interesting. So except for this pair, it was just, you know, we had some problems, we fed them in, and the model came back with some solutions.

34:44What happened here is actually pretty interesting. So we first asked it to improve the bounds for the codes. And it came back with an improvement that used some amount of representation theory. And then we kind of asked it, hey, can you push this further? Like, you know, what happens? And then it came back with some, like, much more sophisticated representation theory. And, like, it turned out that you got this conjectured value for full space sphere packing, like, out of that method by pushing it as far as it can go. So then we kind of asked to directly analyze the sky and try to complete the picture.

35:24Okay. Yeah. So the relationship, like, isn't a coincidence. Yeah, yeah. It's like interesting when you're saying the first prompt, which is, you know, maybe so basic, which is like, can you push this further? It does require some judgment from mathematicians, but like eventually you would imagine by scaling the models, you don't need to do that. Or there's another view that the harness actually does matter and this is kind of part of the harness apparatus. Do you guys have any views on that with your working with Astra? Especially generations of models and how much you have to kind of input or how much the harness matters versus not?

35:59I mean, yeah, I guess there have been some funny quirks like this that just come from exactly what you asked the model to do, basically. In this case, what the model was asked to do originally for codes was to improve the bounds by some exponential factor. So it really shows up in this leading constant up here. And it improved the bounds and it didn't try to push things too much further. Sometimes you see it do, but sometimes it just doesn't bother. But yeah, you just. ask it again and it goes further. So it wasn't like a capabilities issue. It just kind of didn't feel like it at the time. Do you call that judgment or like what is the, because there is a, yeah, what do you call that?

36:41Models tend to be pretty task-oriented. If you tell it to do a task, it accomplishes the task. It's pretty happy. So yeah, the task-orientedness, it's like, but do we expect that level to kind of ascend up to, it's not that they will be less good at being task-oriented. is like they'll ascend to the level of like okay no let's let's go in this direction you'll have the judgment too because you guys had the judgment too like okay this is pretty promising looks like you're using a lot of representation theory it doesn't seem like there's a limit so far um but it doesn't have that context yet but like i guess what i'm trying to say is like this one it's hard to maybe harder to extrapolate but from like previous generations when you had to give it more maybe prompting more of that harness work but eventually probably had to give it less so it probably gives you some confidence that there's this like really fast ascension.

37:30And do you see, yeah, like what are some problems here? Somehow solving a harder math problem is like you have to solve many smaller, like somewhat less hard math problems. And the fact that the math problems are getting harder is kind of an indication that the model is able to take on more and more work in like a single continuous unit. And I think that's the thing that looks very promising. Somehow, like any of these solutions, it's not like one idea, then you're kind of home free. You need several pieces to kind of interact and talk to each other. The model doesn't come up with all the ideas at once, right?

38:05It doesn't pull everything out in an instance. So kind of the fact that it needs to sort of see how this piece interacts with another piece, that's kind of like solving a problem in itself or piecing together many problems in itself. It could just be that, okay, when you're telling it, okay, push this even further, that was of the same order of like magnitude as like all the smaller things it's solving as well in between. And so you don't think that this is kind of like a privileged direction. It's just sort of like, hey, let's give it like one more help. Or you actually think that there's, I guess what I'm trying to get at a bigger question is like, is there a good sense of like, you know, taste?

38:40Because like when people talk about, for instance, how well the models are getting at like doing research, for instance, we want a little bit of RSI. And sort of like there's surprising things about how that improves. And then there's like, oh, you know, maybe right now it's at a level of still like a junior researcher. It's like not really asking like the right problems. And so I'm just trying to get like maybe a sense of like where you're seeing that progress through the model advancements each generation. I mean, what is taste even? Yeah, I think I tend to be pretty utilitarian in my view of taste.

39:19And if you're able to solve problems faster by making better judgments, like I think that's like the best like general proxy I have for a taste. And somehow the fact that solving harder problems means it has kind of by definition means it has better taste. I think there are these, yeah, I think occasionally because they are task oriented, you do occasionally get these symptoms of like, oh, it clearly has made a breakthrough. It kind of understands it's made a breakthrough and then it doesn't kind of push all the way to the limit because that's not what you asked. But that seems, yeah, that seems rather minor compared to the state of progress we've seen so far.

39:57Okay, yeah, I think it's pretty clear. I think it's like maybe you're liable to get confused if you're trying to do a concrete long horizon task and show taste kind of at the same time. but like you know if you if you have like one model that's responsible for taste and one model that's responsible for going out and like you know working for a long time at solving a hard problem kind of as the as the like you know underling of the supervising AI I feel like that's kind of going to be fine currently. Oh interesting because that is like saying that these two things are somewhat if not separate, at least they shouldn't kind of pollute each other's context, which is a little bit, I mean, it could be potentially like a stronger statement than, I guess, you know, it's just kind of interesting because it might just be, like to your point, it's, you know, let's take the utilitarian answer.

40:57It's solving harder and harder problems. It's doing a lot more than just like, you know, brute forcing something. It's making choices. It's like pruning, you know, a vastly large space of possible paths into something that's like really, you know, it's both tractable, but then ends up being like it's a diminishingly small path within that space. But like having like why would would be like a separate model, a separate generation of something that's a different version of the model that would contribute to taste? Or maybe that's totally like it's too abstract, doesn't make any sense. you know we should just let the actual this might like the related question be like you know what is what is the thing that gets us to a better version of intelligence the harness and the model or is it just the model and it's like we see this in you know at least in applied AI or you know startups where it's like it's a continual battle of like you need the harness but then the harness adapts very poorly to a new model because sometimes like a very very minimal harness is still the best way to expose to the raw power of the model.

42:01But then now we also have these like training regimes where we require the harness to be, you know, trained with them. Maybe part of this is to keep things more proprietary and harder for other people to use it. But I think partially it's maybe actually that it helps have more control on like the reasoning traces you care about. It's a long rambling way of saying it's like, yeah, I don't actually, like this is so interesting to see how the models have gotten better at math. And maybe something that's like very abstract and hard to describe like taste is a way to tease out like what is actually necessary here.

42:33I think my only like non-trimural thought here is that like when you're working, I mean, just when you're doing any tasks, occasionally you get pigeonholed and you like work really hard and just having a friend look over your shoulder and be like, what are you doing? And then just like just having that one bit of like step back for 10 seconds, like this is often very useful. Yeah, yeah. I mean, I see no reason why humans would be so different than models somehow. Yep, yep. having, or models would be so different than humans, having a few humans working together is often more powerful than just having one.

43:03Yeah. It's like in this kind of collaborative thing, you actually you kind of, yeah, artificially created it, but it's very similar and dynamic. But I think a lot of taste is also like having a sense of what problems you or like some method you have in mind are going to be good at solving. Like, it's I mean, certainly there's some amount of like absolute aesthetic point, right? But there's also just like, you know, having a nose for what you might want to pursue because you'll be able to make progress. And, you know, I think for that, like, there's, you know, you would expect that as a side product of being good at completing tasks, you would get there sort of, right?

43:45Let me know if we still want to do like a section on Soffit groups, because I think, you know, up to you guys, it's definitely super interesting. So maybe the first question is, what is a group? Let's remind ourselves. So a group is a set of elements with some multiplication operation.

44:11And basically, this is how mathematicians think about symmetry.

44:24So, basically, if G and H are elements of your group, then GH is some other well-defined element of your group, and you have associativity, and you have an inverse. so for every G there's some inverse and there's some like specific element in the group that is kind of the identity okay so it's some like abstraction of like composing operations so these could be like numbers they could be like multiplying matrices they could be like rotating something which is a special case of multiplying matrices and a group is so thick. What if? Well, there's some, you know, precise definition, but, you know, roughly it means it...

45:36So I should say, like, groups that can be finite or infinite. So, like, you know, if you have, like, a square, like, all the rotations of it form a group with, like, four elements. If you have, like, a circle, then the rotations form a group with, like, uncountably many elements. And so SOFIC groups are either finite or countable. You should think of them as being countably infinite, so there's, like, the same number of elements as, like, the integers. And if it's SOFIC, if in some sense it can be approximated by finite groups. So we didn't know if there was a non-sophic group. So the result that Astro proved is simply that there exists a non-sophic group.

46:25Yeah, and without like, I mean, we can, you know, before going to that prove, it is like, you know, I feel like a lot of the programs in math is like, okay, we are such finite creatures. Let's see how well our finite approximations do. And in this case, especially for the countable case, maybe you'll be relating it to the Aldous Leon's thing. It's just like it helps kind of anchor the picture of like it seems like such a, I mean, it's a nice result if it were true, but it's not. And it seems almost like reasonable. And so, yeah, I actually didn't go, I would love to hear the explanation of like how it found a counterexample.

47:05Yeah, I mean, I would say that like, you know, the hope that there was no non-Sophic group, so every group has this kind of approximation. Like, maybe this is sort of like people hoping that there's a miracle. Because it turns out that groups like this have a lot of nice properties because you can run certain proofs for finite groups and then, you know, kind of approximate them in whatever way the definition of being Sophic lets you approximate them and get the results. So there's this notion of being a surjunctive group. So there's some fact that any group which is SOFIC is also surjunctive. Surjunctive is some property of dynamical systems on the group.

47:57And I guess the original question was whether every group is surjunctive. This is some question of Gottschalk from the 70s. And this fact that follows this pattern of prove it for finite groups and then do this approximation is what motivated the question about if there's a non-sulfic group.

48:20Yeah, maybe I'll say a little bit about this Aldis Lyons conjecture. Yeah, sure. Yeah, yeah. So I guess I had heard of this a little bit beforehand because there's a related stronger conjecture in probability that was made popular by Aldous and Lyons. This conjecture, roughly what it says, is like

48:44any infinite graph with some nice property called unimodularity.

49:00a modular random graph can be approximated by large finite graphs.

49:14So maybe the way to explain what these kinds of things are trying to say without getting into technical weeds is to say what they mean about the integers. So how would I draw the integers as a graph? So this is called the Cayley graph. You're just going to connect nearest neighbors. So there's some kind of canonical way in which this is like the graph that represents the integers.

49:44And there's some sense in which you can approximate this by finite graphs. Why? Well, if you look at integers mod n, then you kind of get the same picture but like you have like a big circle instead of an infinite line. And the point is if you like look at any point here and any point here like in nearby things look the same. You have to go like very far away to kind of see this global geometric structure that you have a circle and not a line. And in fact the integers and integers mod n are both groups just by like adding numbers or adding numbers mod n. So these integers mod n are like SOFIC approximations for the full integers.

50:35So like this approximation is kind of why the integers are a SOFIC group. So the statement that every group is SOFIC is sort of a generalization of the fact that you can do this approximation with groups. and this Aldous-Leyens conjecture is kind of a broader conjecture that like any network, you can do this and you don't require as much algebraic structure roughly. So it's kind of a broader conjecture. So this conjecture was disproved earlier, like two years ago. And it was kind of a really tour de force work. Like it was like 250 pages building on another 200 pages. It uses like quantum complexity theory.

51:22So it really builds this very complicated bridge. And I think not many people could understand this. So since this is a stronger conjecture, the disproof is weaker than disproving this statement that all groups are Sofic. But it turns out that the direct proof that there's a non-Sofic group was much shorter and easier than this really amazing disproof of the Aldous Lines conjecture. It's like 15 pages, maybe. And it doesn't have any of this very complicated connection with quantum complexity. It just kind of stays in group theory land. I mean, it uses some important existing results by other mathematicians, like Kuhn and Kuhn and Tom, but it's like a very reasonable, normal kind of proof.

52:12Yeah. And kind of spelling maybe out the obvious, but like the connection between the Sofit group statement is just you take the Cayley graph and that's the one that is like what they use or for the Elvis Leon. Yeah, yeah. And so that's why it's like a subset of. Right, so yeah, basically what happens is, yeah. So for a group, you can take exactly a Cayley graph. So you take some like elements that like generate the group and you kind of connect elements that are adjacent. So in this case, like this is a Cayley graph of the integers. so so right so when you do that um from a group you get like a deterministic graph right you just get like a single graph right you fix some set of generators so this conjecture is stronger basically because um it allows a broader set of graphs that aren't deterministic it allows them to be random but have some extra uh you know um you know modularity property that constrains exactly how it can be random but yeah basically that's that's the difference um like uh Here you kind of have to give a deterministic network instead of a random one.

53:17Yeah, anything kind of interesting, surprising about the results? I mean, you mentioned some things, which is like it stayed within group theory, the techniques. I mean, I think maybe it's like a nice example of this general pattern that theorems produced by AI have generally been like, like the proofs are pretty short generally. they're like um like with the counter examples so far yeah but yeah this one it's like okay it's sort of a counter example but like there's some you know there's some like stuff you have to to do to analyze the difficult part here is that like the property of being a sophic group is not so easy to get your hands on so you have to find like a concrete way of saying like like producing a way of saying this group cannot be so big.

54:10And the proof is actually, it's very short. It's a combinatorics argument, but it's a very delicate combinatorics. Somehow you need to both have the right statement and know what piece of the literature and then execute it correctly. That's very nice. The difficulty of this problem is that it's just really, really hard. It's very hard to get your hands on being approximated by any possible finite group. Yeah, I was going to say, it's like, what is happening at that countable infinity that's resisting this approximation? Do you guys kind of give a sense of... Do you do post-mortem when you're like, okay, Astra, explain to me.

54:43What is the... What was a good explanation you got out of it? I think there's some concrete combinatorial obstruction. Basically, it's hard to explain, but there's some concrete combinatorial obstruction, which if you read the previous papers, you realize that that's what they couldn't rule out. And Astra found a way to kind of say, okay, no, no. If you add this one extra algebraic fact, this weird conspiracy can't happen. It's very clearly trying to rule out a conspiracy that previous authors had implicitly written about. And those were the actual suspects, it turned out. So they were sort of on the right track, and then this did the last mile of, well, whatever.

55:26However you quantify that. But I think it's, like, a year ago, I would have been very surprised to learn that, like, all of these AI proofs are, like, very short and elegant. Yeah. Like, they're, you know, you're kind of, like, afraid that they're going to, like, generate all these thousand-page things. Yeah, like, no, let's verify that. I'm never going to be able to understand it. But it's been kind of the opposite. Yeah. Like, only humans can generate, like, 200-page proofs right now. Yeah. Yeah. Well, and also I was like asking, like, if you do that postmortem, it ends up usually engendering more mathematics.

55:59Because when you do that with humans, like, that's what, you know, breeds new mathematics. So maybe if you kind of alter the prompt a little bit and be like, how would you, you know, generalize this or something? Like, yeah, I don't know if that's been a technique for you guys to like have it explore and exploit what it has already developed. Well, there has been some, there has been follow-up on this already, actually, by Kuhn and Tom, who this was always built on. So, they like... So, math community is coming on. Yeah, yeah, yeah, which is kind of what we're hoping. You know, we don't want to be, you know, writing lots of follow-up papers ourselves, but if there's some interesting follow-up that, you know, it's like, we're very, very happy that there's some follow-up building out these ideas more and giving, like, more examples of non-selfic groups in this case.

56:43Yeah, well, actually, maybe that's a great segue into like how, you know, what's the ideal way that this is being taken up by the math community? Because I feel like there's a spectrum of answers from working mathematicians, sense of like some, you know, probably most at this point are like, okay, AI is obviously doing some non-trivial stuff. It would be a disadvantage not to admit that in my workflow. I've definitely heard some stories where people are kind of, you know, would find it hard to either take AI as a co-author or like how do you even do kind of attribution this way? But I don't know, like what, maybe to paint the more optimistic picture, so you're saying you want the mathematicians to be building on these results.

57:24It definitely generates a lot more results to be verified. So, you know, it puts pressure on the community and the profession. Like how do you kind of expect the evolution of kind of uptake and collaboration with mathematicians? I mean, given that the fact that the models can produce sophisticated mathematics means that they can help you understand sophisticated mathematics. I mean, I don't know, occasionally I enjoy looking at the archive and I want to understand some proof and I could read the introduction, but in practice, it's just much faster. Take the PDF, put it into my favorite model and then get an output of what is the rough proof strategy.

58:03And so I have this like, along, I mean, of course, models are going to help us produce exponentially more mathematics, but they also make it much easier to absorb it. And right now, okay, it's still a bit of a challenge back and forth, but I think it's, for me at least, much, much faster at understanding. It's much, much faster to understand a piece of mathematics with a model than without it. So, it's helping solve the problem it creates anyways. Yeah, I feel like that at least, and it's, you know, I don't view it as creating much more problem, but again, I don't have such high stakes in like, okay, I'm going to get I'm not going to get tenure, et cetera.

58:40So, like, I agree. Like, making it more accessible. Like, if I'm not spending so much time absorbing an area, I can, like, put it into chat GPT and then expect to, I mean, you guys have an even more powerful model, hopefully releasing, for other people to enjoy as well. But, like, it's, I think, like, the positive version of that is actually more people can participate in mathematics. It's like people might be coming with other intuitions and they could actually maybe generate good mathematics. Is that sort of, like, closer to the vision of what you're hoping this is, you know, pushing towards?

59:13Or like what things do you think mathematics should be wary of to kind of adapt fast enough to take advantage of AI? Yeah, I mean, I think certainly there will be a lot of changes, right? Like I guess in math, like there are a lot of things that are kind of important for like a given result, right? You need someone to come up with it, but you also need people to understand and absorb it and like, you know, internalize it enough to do more with it and like figure out where it fits into like humanity's understanding, right? And like a couple of years ago, like the proving the result was like so hard that kind of the other stuff was just kind of coming along for the ride, right?

1:00:00You know, like if you manage to like prove this thing yourself, you're automatically going to understand it quite well. You're kind of responsible for maintaining it in some sense and explaining it to other people. And yeah, now this kind of, what was the main bottleneck before is kind of much less of a bottleneck. And these other kind of constraints come into play. So yeah, the optimal structuring for, you know, organizing the knowledge could look rather different. Yeah. How does that look? I mean, does this make the field a lot more kind of empirical? Will people do sort of the hard, like the first thing that was scarce, which is like all the reasoning, and then more, I mean, not that it's like a bad thing to make it empirical, but it's almost like it functions as a very different discipline.

1:00:54Like a lot of the fun stuff is understanding, you know, and so understanding, communicating, maybe assembling, having still the human taste, Does that sort of remain rarefied and that's how, you know, current mathematicians need to adapt and reward, you know, contributions? Or is this too much of a caricature? It's like something else. I think certainly understanding how to put, as we get more and more mathematics, put it in like a proper framework and sort of how sort of like being able to explain it to other humans so that they can also appreciate it. I mean, so implicitly we valued this, but it was usually because you were the person proving the results that gave everybody else the understanding.

1:01:34But I think increasingly it would be a function of like, you're sort of helping, you're the human who can sort of give this understanding to other people and sort of help them with it. I think that more sort of, that communal understanding will, I think, become, it was much more implicit in how we viewed math and genes, but I think it would be an increasingly more explicit and valuable part of the subject. I mean, a nice thing about math is that the ceiling for difficulty of a math problem is pretty high. So even if AI continues getting exponentially better at math, it might, you know, plausible will never solve something like P versus NP.

1:02:14And it could be that the field kind of becomes more, you know attached to like like these big mysteries and less to like smaller mysteries that are more like routine now yeah yeah i think that's a positive vision of the i mean also like i don't know there are things i spent like months or years of my life wondering about not getting to know and and now we get that yeah some portion of them i'll get to know the answer to it i'm pretty happy about that. No, exactly. No, I'm excited about this, like, renaissance of results and understanding, and I feel like, I mean, this is such an infinite, you know, field.

1:02:55Like, no pun intended, but, like, it's just, like, it's just, there's so much that you can actually create here. So, I mean, especially for somebody like me who's not going to have the time to actually, like, practice mathematics. Now there's, like, a lot more that you can actually do in the activity of math. So, yeah. Yeah, I think the, like, the ability of someone who's not working on math is, like, their literal job all the time to, like, understand what's going on and, like, you know, learn about some of the mysteries they might have wondered about will go up quite a lot. Also, you know, if you're, like, if you're working on something that requires some math, you know, suddenly you don't need to, like, find a world expert on this topic to be able to, you know, use it in your own work.

1:03:40Sorry, Matt. No, it's true. I mean, I think there was just like a dearth of actual people who could do that. And so I think this is helpful. Maybe it's helpful for theoretical physics. Like, we'll see. But a lot of other applied areas as well. It'd be nice for the world if applied mathematics went a lot faster. Yes. I mean, I'm over that. Well, thank you guys for joining. This is a lot of fun. And I'm, you know, just so excited for how much the models are advancing. So maybe we'll have you guys back soon. Thanks so much for having us. Yeah, thanks for having us. Thanks for listening to this episode of the A16Z podcast.

1:04:18If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

1:04:53Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures. We'll be right back.