OpenAI Researcher Dan Roberts on What Physics Can Teach Us About AI

0:00In the 40s, the physicists went to Manhattan Project even if they were doing other things. That was the place to be. And so now AI is the same thing and basically set open AI as that place. So maybe we don't need a public sector organized Manhattan Project, but it can be open AI.

0:37Joining us for this episode is Dan Roberts, a former Sequoia AI fellow who recently joined OpenAI as a researcher. This episode was recorded on Dan's second to last day at Sequoia before he knew that he would go on to become a core contributor to O1, also known as Strawberry. Dan is a quantum physicist who did undergraduate research on invisibility cloaks before getting a PhD at MIT and doing his post -doc at the legendary Princeton IAS. Dan has researched and written extensively about the intersection of physics and AI. There are two main things we hope to learn from Dan in this episode. First, what can physics teach us about AI?

1:13Whether the physical limits on an intelligence explosion in the limits of scaling laws, and how can we ever hope to understand their own nets? And second, what can AI teach us about physics and math and how the world works? Thanks. Thank you so much for joining us today, Dan. Thanks, delighted to be here on my probably second to last day at Sequoia, depending on when this air is and how you're going to talk about it. You will always be part of the Sequoia family, Dan. Thanks. I appreciate it. Maybe just to get started, tell us a little bit about who is Dan. You have a fascinating backstory. I think you worked on Invisibility Cloaks back in college.

1:52like what led you to become a theoretical physicist in the first place? Yeah, I think, and this is, you know, my stock answer at this point, but I think it's true. I was just an annoying three -year -old who never grew up. I asked why all the time, curious how does everything work. I have a, I have a 19 -month at home right now, and I can see the way he followed the washing machine repairman around and had to look inside the washing machine. So I think I just kept that going. And when you're more quantitatively oriented than not, rather than going to philosophy, I think you sort of veer into physics.

2:27And that was sort of what interests me. How does the world work? What is all this other stuff that's out there? The question that you didn't ask, but maybe I'll just answer it ahead of time. The sort of inward facing stuff felt less quantitative and more in the around with the humanities. So what is all this stuff? That's pretty physics -y. What am I? Who am I? What does it mean to be me? That felt not very sciencey at all. But with AI, it sort of seems like we can think about both what does it mean to be intelligent and also what is all this other stuff in some of the same frameworks. And so that's been very exciting for me.

3:05So we should be trying to recruit your 19 month goal right now is what you're saying. Oh, yeah, absolutely. He's got his, he grew out of one of his, His Sequoia won Z, but he fits into his Sequoia toddler teacher now. And so he definitely is ready to be a future founder. I guess at what point did you know that you wanted to think about AI? At what point did that switch start to flip? Yeah, so I think like many people, when I discovered computers, I wanted to understand how they worked and how to program them. In undergrad, I took an AI class. And it was very much good old -fashioned AI. A lot of the ideas in that class actually are coming back to be relevant, but at the time it seemed not very practical.

3:47It was a lot of, you know, if this happens, do that, you know, and there was also some game playing there that was sort of interesting. It was very algorithmic, but it didn't seem related to what it means to be intelligent. Can we just pick up on that real quick? Like, what do you think it means to be intelligent? That's a great question. I can see wheels turning. Yeah, well, this is one of those questions where I don't have a stock answer, but I think it's important to not say nonsense. One of the things that's exciting to me about AI is the ability to have systems that do what humans do and to be able to understand them from how, you know, what are the lines of Python that caused that system to do something to, you know, to trace that through and understand what are the outputs and, you know, how the system can like see and classify what it means, you know, what is a cat, what is not a cat, or can write poetry.

4:46And, you know, it's just a few lines of code. Whereas if you're trying to study humans and ask what does it mean for humans to be intelligence, you have to go from biology through neuroscience through psychology up to other higher level versions of ways of approaching this question. And so I think maybe a nice answer for intelligence, at least the things that are interesting to me is the things that humans do. And then now, if you pull back the answer that I just said, a second ago, that's how I connected to AI is taking pieces of what humans do, and we're understanding at least. And an example that's kind of simple and easy to study that when you might use to understand better what it is that humans do.

5:35So Dan, you mentioned when you were, you know, studying AI in college, a lot of what sounds like kind of hard -coded logic and brute force type approaches. Was there a moment that kind of clicked for you of like, oh, okay, this is different? Was there a key result or a moment where it was like, okay, we're going bigger places than kind of the if this than that logic of the past? Yeah, actually it didn't click. I sort of wrote it off. And it would be great if there was a separation between like 10 years between then and the next thing that I'm going to say But actually the writing off maybe lasted a year or two because then then I went to I went to the UK for the first part of grad school.

6:12I spent very long time in grad school and I discovered I discovered machine learning and and and more statistical approach to artificial intelligence where you have a bunch of examples of large amounts of data, or at the time maybe we would have said large amounts of data, but we would have said you know you have you have data examples of a task that you want to perform and I mean there's different ways that machine learning works, but you write down a very flexible algorithm that can adapt to these examples and start to perform in a way similar to the examples that you have. And that approach borrows a lot from From physics, it also, at the time I started, so graduated college in 2009, discovered machine learning in 2010, 2011, 2012 is the big year for deep learning.

7:06And so there's not a big separation here between right off and rediscovery. But I think, and machine learning clearly existed in 2009, it just wasn't related to the class. Classly, I took, but this approach made a lot of sense to me. And it started to have, I got lucky in that it started to have real progress and seemed to fit in a framework that I understood scientifically and I got very excited about it. Why do you think there are so many people who come from a similar background or similar path to you? Like a lot of ex -physicists and working on AI. Like is that a coincidence? Is that heard behavior?

7:39Or do you think there's something that's particularly, that makes physicists particularly well -student to understanding AI? I think the answer is yes to all the ways you do. All the above. You know, physicists infiltrate lots of different subjects and we get parodated about the way that we go about trying to use our hammers to to tackle these things that may or may not be nails. Throughout history there are a lot of times that physicists have contributed to things that look like machine learning. I think in the near -term, the path that physicists used to take when they don't go didn't remain in academia often was going into quantitative finance, then data science, and I think machine learning was Because the, and its realization industry was exciting because, again, it's something that feels a lot like actual physics and is working towards a problem that's very interesting and exciting for a lot of people.

8:32You're doing this podcast because you're excited about AI. Everyone's excited about AI. In many ways, it's a research problem that feels a lot like the physics that people were doing. But I think the methods of physics actually are different from the methods of traditional computer science and very well suited for studying large -scale machine learning that we use to work on AI. There's traditionally physics involves a lot of interplay between theory and experiment. You come up with some sort of model that you have some sort of theoretical intuition about. Then you go do a bunch of experiments and you validate that model or not.

9:10And then there's this tight feedback loop between collecting data, coming up with theories, coming up with toy models, understanding moves forward by that. And we get these really nice explanatory theories. And I think the way that big deep learning systems work, you have this tight feedback loop where you can do a number of experiments, the sort of math that we use is very well suited to the math that a lot of physicists are familiar with. And so I think it's very natural for a lot of physicists to work on this. and those tools, a number of them differ from sort of the traditional, at least theoretical computer science and theoretical machine learning tools for studying the theoretical side of machine learning and maybe also differentiation between just being an awesome engineer and also being a scientist and there's tools from doing science that are helpful in studying these systems.

9:59Dan, you wrote this, what I thought was a beautiful article, Black holes in the intelligence and explosion, and in there you talk about this concept of sort of the microscopic point of view and then the system level point of view and how physics really equips people to think about the system level point of view and that has a sort of complimentary benefit to the understanding these systems. Can you just take a minute and kind of explain sort of microscopic versus system level and how the physics influence helps to understand the system level? Sure, so can let me start with an analogy that I think is like very, you know, goes even further than an analogy, but going back what year is it?

10:40Maybe like 200 years or so, there was around the time of the Industrial Revolution, there was steam engines and steam power and a lot of technology that resulted from this and ultimately power industrialization. And in the beginning, there is a lot of engineering of these steam engines. And there was this high level theory of how this work called thermodynamics were. And imagine everyone's seen this in high school, perhaps, where there's the ideal gas law that tells you that there's some relationship between pressure at volume and temperature. And these are very macro level things. Like you can buy a thermometer.

11:18You can also measure the volume of your room. And you can buy a barometer as well. maybe people don't, or look up on the weather report. But these are things that, these are like measurements that we use and we talk about. But then underlying this, and it took us a little bit later to like validate this and understand it, there's the notion of atoms and molecules. They air molecules bouncing around. And somehow we now understand that those air molecules give rise to things like temperature and pressure and volume, I guess, is easier to understand that the gases the molecules are confined to room.

11:54But there's a precise way in which you can start with the statistical understanding of those molecules and derive thermodynamics, like derive the ideal gas law from it. And you can go further than that, derive its ideal because it's wrong. It's just a toy model. But there are corrections to it. And you can sort of understand from the microscopic perspective, which is the molecules, which we don't really interact with. We don't see them. We don't interact with them on a day -to -day basis. but their statistical aggregate properties give rise to sort of this behavior that we do see at the macro scale.

12:27And part of, to get to your question, I think there's a similar thing going on with deep learning systems. And I wrote a book with Choyeda and Boris Hanon on how to apply these sorts of ideas to deep learning and at least an initial framework that allows you to start doing this in an initial way. And to answer your question, the sort of micro perspective is you have neurons and weights and biases. And we can talk about in detail how that works. But when people think of the architecture, there's some very specific, some people say circuits. There's specific ways in which these things, there's an input signal which might be an image or text.

13:13And then there's many parameters. And it's very simple to write down. and it's not that many lines of code even taking to account the machine learning libraries. But it's like a very simple set of equations. But there's a lot of weights. There's a lot of numbers that you have to know in order to get it to do something. And that's sort of the micro. That's like the molecule's perspective. And then there's the macro perspective, which is, well, what did it do? Did it produce a poem? Did it produce a solve a math problem? But how do we go from those weights and biases to that macro perspective? And so for statistical physics, the thermodynamics, we understand that completely.

13:53And you could imagine trying to do the same sort of thing, literally applying the same sorts of methods to understand how does the underlying micro statistical behavior of these models lead to the sort of macro, or as you said, system level perspective. Dan, maybe speaking of scaling laws, And I think you were at our event AISN, Andre Carpathy mentioned that current AISystems are like five or six orders of magnitude off in efficiency compared to biological neural maths. How do you think about that? Do you think scaling laws get us there? Just combination of scaling laws plus hardware getting more efficient?

14:34Or do you think that there's kind of big step function leaps that need to come in research? There's maybe two things that could be ment here. One is that the way humans seem to work at a similar scale to AI systems is much more efficient. The amount of, we don't need to see trillions of tokens before we speak. We see a much, you know, my toplar has, is already starting to speak in sentences, and he's been exposed to far less tokens than a typical large language model. And so there's some sort of disconnect between human efficiency at learning and what large language models do. Of course, they're very different systems are designed, you know, the way in which they learn is right now very different.

15:16And so in some sense, that's to be expected. So there's this gap here that you could imagine bridging. There's another thing that I think is not what you meant, but I think is sort of the thing to answer about with respect to scaling laws, which is, and I talked about this a bit in the article, but lots of people seem to talk about this, which is what is the final GPT? You know, there's GPT for right now, and it could be other companies as well, but since I'm going to join OpenAI, let me represent my new company. So is it going to be six? Is it going to be seven? At some point, if assuming we have to scale things up, there are things that are going to break, whether they're economic.

15:53We're going to run out of, you know, we're going to try to train a model that's larger than the world's GDP or GWP, whatever the, however the D works for the world. And, or we're going to run out of, you know, we're not going to be able to produce enough GPUs or we're not going to be able to put, you know, it's gonna cover the surface. You know, a lot of these things are going to break down at some point. And so probably the economic one happens first. So how many, you know, how many more iterations do we get before we run out of actually being able to scale practically? And where does that get us?

16:26And then I think to tie those two perspectives together, there's sort of scaling on its own. And of course it's impossible to just tangle this because people are making things more efficient. But you know, there's like, you could imagine and there's the take literally what GPT -2 was, which was the initial big model, and keep scaling it up, is that going to get us to some, you know, super different, exciting, economically power, or however you want to define what the end state of AI research and AI startups and AI in industry is. So, or do we need lots of new, exciting ideas? And again, of course you can't really just disentangle these, but I think the general scaling hypothesis is that it's just the scaling, and it's not the ideas that matter.

17:09Whereas the how do we get to efficient like humans, I think requires like non -trivial ideas. And to answer your question, the reason I'm excited about joining OpenAI is that I think there is high leverage to be had in the ideas in going beyond scaling. And that we will need that in order to get to the next steps. And I have no idea what I'll be working on. But when this air is, I guess I will know what I'm working on. But that's what's really exciting to me. Dan, is there almost like a pendulum that swings back and forth between scale and ideas in terms of how people apply their efforts in the world of AI?

17:48Like, transformers came out, great idea. Since then, we've largely been in this race to scale. It feels like things are starting to asymptote for a bunch of practical reasons that you mentioned. Is the pendulum swinging back toward ideas as the currency? It's less now about who can have the biggest GPU cluster and more about finding new architectural breakthroughs, whether that's reasoning or something else. Yeah, that's a really good question. I think there's this article by Richard Sutton called the bitter lesson, bitter pill. And it basically gives the argument that ideas are not important.

18:30That scale is what you need. That all the ideas are always trumped by scaling things up. And that's a bunch of things. But maybe that's a high level takeaway. And there is a sense of this where there are a lot of interesting ideas that came out in the 80s and 90s that people didn't really have scale to explore. And then I remember when after AlphaGo and DeepMind was writing a lot of papers, people were rediscovering those papers and re -implementing them in deep learning systems. But this was sort of still before people realized, no, the thing that you need to do is scale up. And even now with Transformers, people are exploring other architectures, or even simpler architectures that we knew before that seemed to be able to, There's a notion, maybe scaling laws don't come from, as long as the architecture isn't sick in some way, they come from sort of the underlying data process and having large amounts of data rather than from having a special idea.

19:28I think the real answer is that there's a balance between the two that scale is hugely important and maybe it was just not understood how important and we also didn't have the resources to scale things up at various times. You know, the things that have to go into producing these GPU clusters that are producing these models are, you know, you guys know this as well But like there's a lot of parts along the supply chain or along the product chain whatever you actually call it in order to make those things happen and to deploy them and even You know the way GPUs were originally they've now co -evolved to be well suited for these models and Trans the reason In some sense, you can think of Transformers was a good idea, was because it was designed to be well suited to train on the systems that we had at the time.

20:11And so sure, these other architectures could do it, you know, at an ideal scientific level, but at a practical level, it was important to get something that was able to reach that scale. So I think, you know, if you brought in ideas to be that sort of thing that's married with scale in some way, then I still think ultimately, someone came up with the idea of deep learning. That was an important idea. There's pits in McCullough came up with the original idea for the neuron and then there's lots of frozen blood came up with the original perceptron. There's a lot of people from going back 80 years of making important discoveries that were ideas that contribute.

20:55So I think it's both, but it's easy to see how, you know, if you get to apply, if you're bottlenecked and then all the sudden you get to apply, if you're bottlenecked people think about ideas, and then if you unlock a new capacity of scale somehow, then you just see a huge set of results and it seems like scale is super important. I really think it's more of a synergy between the two. Maybe in the topic of the race to scale, Dan, you mentioned kind of the just the economic constraints and realities, which I guess are more practically a ceiling in the private sector. You also mentioned the Manhattan Project earlier in terms of things that physicists have been involved with.

21:33Do you think we need a Manhattan Project style thing for AI, like at the nation state or at the international level? Well, one thing I can say is that part of the process that led me to open AI is I was talking with your partner, Sean McGuire, who brought me to Sequoia in the first place. And trying to figure out, is there a startup that makes sense for me to work on that has the right mix of sort of scientific questions, research questions, and also as a business. And I think it was Sean that said, and I don't mean the analogy in terms of the impact of, in terms of the negative impact of what the men think of them and had in project.

22:13But just in terms of the scale and the organization, He said, you know, in the 40s, the physicists went to Manhattan Project, even if they were doing other things. That was the place to be. And so now AI is the same thing, and basically set open AI as that place. So maybe we don't need a public sector organized Manhattan Project, but it can be open AI. Open AI is Manhattan Project. Yeah, well maybe that's not a direct quote that we want to be taking out of context. I think in terms of... Is the metaphorical, man, in front of you? Yeah, in terms of scale and ambition, in terms of, I think, I mean, I think a lot of physicists would love to work at OpenAI for a lot of the same reasons that they probably were excited to...

22:56Well, okay, there's a number of different reasons. Maybe we just have to leave it as a nuanced thing rather than making broad claims. Can we talk a little bit about this, like, can we ever understand AI, especially as we go to these deep neural nets, or do you think it's a hopeless black box, so to speak? Yeah, I think within the, this is my answer to the, what are you are continuing about, although maybe, you know, on the internet, everyone takes every side of every position, so it's hard to say you're having a continuing position, but I think within AI communities, I think of my contrarian positions that we can really understand these systems.

23:34And in the physics systems are extremely complicated. And we have made a huge amount of progress in understanding them. I think these systems sit in the same framework. And another principle that, that's shown I talk about it in our book, and that's a principle of physics is that there's often extreme simplicity at very large scales. Basically due to the statistical averaging or more technically the central limit theorem, things can simplify. And I'm not saying this is what happens exactly in large language models, of course not, but I do think that we can apply sort of the methods that we have and also maybe hopefully have AI that can help us do this in the future and by AI, I mean, tools not like individual intelligence is just going running on their own and solving these problems.

24:21But I guess I feel at the extreme end that this is not going to be an art that the science will catch up and that it will be able to make extreme leaps in really understanding how these systems work and behave. So Dan, we've talked a bunch about what physics can teach us about AI. Can we talk a bit about what AI can teach us about physics? Are you optimistic about domains like physics and math and how these emerging models can probe further into those domains? Yes, I'm definitely optimistic. I guess my perspective is that math will be easier than physics, which maybe betrays the fact that I'm not a mathematician.

25:09And I'll say, I can give explain why I think that in a second. But I still have a lot of friends that work in physics. There's like a growing sense and maybe even approaching a dread that, and maybe this is actually the answer to why the physicists work on AI. Because if what you care about is the answer to your physics question, and you want to make it happen as soon as possible, what is the highest leverage that you can do? Maybe it's not work on the physics question you care about, but it's work on AI to make the, you know, to, to, because you think that the AI might end up solving those, those questions very rapidly anyway.

25:51And I don't know the extent that anyone really takes it seriously, but I think within the theoretical physics community that I come from, that this is sort of a thing that, that someone gets thrown around and, and, and discussed. I think, um, maybe to give a more object level answer, I think what's exciting about math and, And maybe when you have no umbrella on, if you have a mod, I'll talk about this, but this is something that he's talked about for a while before he joined OpenAI. I think that we have, you know, we meet a lot of progress in terms of solving games by doing more than just looking up what is the strategy that we should use to play the games, but also being able to simulate forward and, you know, the way that if I'm in a very hard position in a particular game rather than just playing with intuition, I might sit and think about what I should do.

26:46Yeah, sometimes this goes under the name inference time compute rather than training time compute or pre -training. And you know, there's a sense in which what it means to do reasoning is very related to this ability to sit and think. So we know how to do it for games because there's a very clear, witting and lost signal, so you can simulate a head and sort of figure out what it means to do good or not. And I think math in And some parts of math, again, I'm not a mathematician. And we'll always scared about talking about math publicly and saying something wrong to all of the upset mathematicians.

27:19But it seems like certain types of math problems are not as constrained as games, but are still constrained enough where there's a notion of finding a proof. There's different problems in terms of search in terms of how do you figure out what is the next move in the proof. But the fact that we might call it a move suggest that there's things in math that feel a lot like games and so we might think that the fact that we can do well at games maybe means that we can do well at certain types of mathematical discovery. Well, I was going to say since you mentioned Noam, he likes to use the example with test time compute of whether it could help to prove the even hypothesis.