🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

25:36is like one for fluid and kind of PDE-type modeling systems or for, let's see, molecular systems, there's oftentimes this idea about equivariance. I mean, are the models like picking up on these, you know, tricks which have been developed in the literature or are they adding some weights for some physical prior or something? What does that look like when it introduces like a physics, when it introduces some sort of, you know, knowledge like that? I don't know if I have enough data to sort of say, oh, 73 % of the time it does this. But especially when you point it at existing papers, it will try to, in fact, it will do very well at adapting the methods that are described in the papers for the problem.

26:22In fact, it will do an amazing job. You can actually often just recreate or reverse engineer a paper. That's, again, what Michael Brenner actually likes to do this. He'll say, oh, that sounds like an interesting paper. We did this actually for the, we had this thing. It was actually kind of a hack. I suggested this to Michael where there's this one MIT professor who, he came up with some code to do, essentially if you have a rooftop with a fixed area and you want to sort of maximize the amount of solar power you capture over a day, sort of solar energy, you can build up, which of course captures more sunlight, and you can sort of build, you can have it design a widget.

27:03you know, involving mirrors or struts or solar panels just sort of stuck at whatever angles or sizes you like. And usually with a maximum height. And then try to figure, let it sort of explore that design space. And I believe, Michael, we can ask him, I believe it actually just, I don't think he actually installed the simulator. I think the code just reproduced the code because it has a coding agent inside of it. So just reproduced the code from the paper and sort of figured it all out. So yes, it's very good, especially if given a pointer to what other people have done. It's very good at kind of like, oh, I haven't seen it try at equivariant modeling.

27:48That can get very hairy if you know about Klebsch-Gordon coefficients. It's pretty normal. They're fun, yeah. Yes. So I don't know if it'll do the true equivariant stuff, but it actually knows a lot. I remember actually when Gemini 2.5 came out I know this is not exactly about era but I remember sort of thinking oh this is a new world when like the day 2.5 came out because I said hey Gemini 2.5 can you write me some boosted decision tree code? And it did. And it worked. It just did, yes. And I said you know yeah this is a new world. So yes I think to loop back to your question I think yes if you give it sort of guidance about oh you know it's important to put this kind of thing in, it will.

28:32And so it won't necessarily, at least not that we've seen, discover completely new, like if you didn't know about Klebsch, Gordon, Goffin, you didn't know about something, it won't completely discover a new kind of physical model from scratch. But it will certainly, if you tell it about interesting constraints about the world that are known, it will certainly follow up. I don't know if I answered your question. So there's still room for humans for the next year or two. Oh, in fact, there's, going back I think there's totally room for humans because I don't know I mean we have co-scientist that tries to help you come up with sort of hypothesis generation but really I still haven't seen sort of the creativity and the philosophy and sort of the careful rigor you totally need humans I don't see humans going away they can make strange suggestions and I've used co-scientist actually for an interesting problem in geochemistry.

29:29And I learned about a new kind of ion I didn't realize happened in magma, but, uh, I, but, and so it'll tell you interesting things and you'll learn stuff, but I don't think it sort of substitutes for human creativity. You know, going back, we were just talking about two years, two point, you know, two, you had Gemini 2.0 to 2.5 and this was like, you already saw a leap and now it's been another year or two and now we're at 3.5 and, you know, there's, you're saying this is working much better. I mean, And whenever you look at a graph, you know, you can, if something looks like an exponential, it can either, you can either be in a sigmoid or you can be at the beginning of a takeoff, right?

30:03I guess every exponential turns into a sigmoid dimension. Every exponential turns into a sigmoid. But the question is like, where are we on that? I mean, I guess I'm a big believer in sort of the whole jagged. The frontier. Yeah, the jagged frontier. And so certainly, at least what I see, I mean, I don't know what's going to happen in a couple of years. But yes, there's some big spikes out in jaggedness in terms of coding ability and just gathering knowledge and finding related things. And that's huge and wonderful, which I think is great for scientists. So far, it's kind of less in terms of rigor.

30:37And we can talk about things like the International Meth Olympiad and meth in general. But in terms of sort of philosophy and creativity, I think it's still kind of not. And maybe it'll, maybe everything will, some people are saying everything's going to inflate and pass, but I'm still seeing a lot of very strong jaggedness. So I can see, well, maybe it'll get to be extra good at coding and extra good at fitting models and extra good at making suggestions and things. But I don't know. So far, not. So far, you need the humans.

31:06John Platt:Yeah, I want to get back to the question about the multiple hypothesis testing. Oh, sorry, sorry. We got it. We derailed. Sorry, yeah, yeah. Totally love the tangent. But multiple hypothesis testing is when you have a descriptive model and you're saying this is the way the world works and you have a bunch of data and you take a billion darts and you throw and so you have to be careful there's something called false discovery rate. And so the question is, is this finding descriptive models or is this finding sort of predictive models? And fundamentally, the scientist is there to make sure that whatever is saying is descriptive.

31:43if we haven't been able to make a system so far out of these pieces that can really sort of discover completely new physics or completely new science. But this is sort of a power tool to help you discover completely new science. So maybe I'm trying to unask your question about multiple.

32:02John Platt:But I think this gets at the heart of it, yes. Yes, but then you're saying, what about just pure, okay, let's set aside. It's not trying to figure out a descriptive model of the world. That's still up to the scientists. But what about just plain old overfitting? Yes, you have to be very careful because it's a power tool. It can, I shouldn't probably say, it can slice your fingers off. You know what I mean? You have to be very careful and you have to be very rigorous. In fact, now you have to be more careful and more rigorous to not fool yourself. You really, really need to be just excruciatingly careful about having, you know, very hidden holdouts that you don't look at.

32:37You have to be just super, super rigorous to make sure that you don't completely, because it is a total power tool.

32:44John Platt:So the question to how do I not slice my fingers off is you need to use the same techniques, but be very careful with them. Yes. That's a very clear answer that I'm going to get before you think. Yeah. Okay. Yeah, I actually don't think I've heard any guests say that. Yeah, it is, I think, a very important skill. Maybe one of the most important skills in the new. People talk a lot about taste. Yeah. But maybe this is a variation of taste. The taste is like the other side, right? It's like the rigor. It's like the, yes, yes. In fact, if anything. Yeah, taste or rigor, which one's more important?

33:17Well, I don't know. I think people, at least the way I am viewing it is, I mean, the aspect of the researchers are software developers, which there's a lot of overlap in a lot of fields. I'm seeing that software engineers are, it's almost like, obviously, there's a lot of concern, like, oh, no, what am I going to do? You know, coding seems to be, you know, getting automatic. So I think there's sort of both. I think there's a lot of people get pulled into, well, I'll be the creative source. So I'll try to figure out new science. I'll try to figure out new products. I'll try to sort of really be very recreated.

33:56And again, I'm a strong believer that I don't think that's going to go away. There's also people sort of pull towards rigor, Like, oh, I want to make sure this doesn't crash. I want to make sure this scales. I want to make sure this isn't wrong. I think you need both. And I think you need people who are really good at both. But they don't necessarily have to be the same people. But yes, I think you need, I think this is even broader than science, just as sort of software engineering evolves. Yeah, it'll be, you know, the people who will bring the creativity and the people who will bring the rigor.

34:26And I think those will be sort of anchors.

34:29John Platt:Some other things that I've seen are related work out there. There's a really cool leaderboard for, you know, a claw leaderboard, agent leaderboard for scientific problems from Stanford. I don't know if you're familiar with it. it seems like a really interesting idea to me to have, you know, sort of different agents kind of competing on the leader. So it seems like if you squint a little bit, what ERA is doing is, is kind of a leaderboard, but it's internal and it's recombining ideas. Whereas what are your thoughts about this? And do you, is that like a thing that you guys are working on? And is there problems with that or advantages to that?

35:11Ironically, you know, the whole ERA project actually started because people may not realize Kaggle is actually part of Google. Oh, yeah. And so it was called the auto Kaggle problem. So it was actually like, that's what it was. Let's try to have a system that can sort of win at Kaggle competitions. So that's sort of why it sort of has this shape. That's sort of how the project started. And it goes back to sort of overfitting, right? If you've ever actually competed in a Kaggle competition. I have done Kaggle competitions, or I've done one. It is a really interesting phenomenon because there's this, overfitting is like rampant.

35:52Yeah, yeah. And it's really impressive how people can overfit to certain datasets. That's right. In a way that is, yeah. Or even we had a, we have a fun project I can talk about more if you like, that tries to mitigate contrails. Jet contrails. Yeah, yeah. Talk about that if you want. And we had a contrail Kaggle competition and people actually beat us. But they found that we had a half pixel error in our labels. And they had to do with the center versus the lower left. Like, where is zero, zero? Is it in the lower left of the pixel or is it in the center? You know what I mean? Yeah. So they found that and exploited that and squeezed or whatever a little bit extra stuff.

36:32Because it turns out when you make artificial data and you rotate it, you have to make sure that you take into account that half pixel offset. So yes, people themselves will act like these LMs and try to sort of reward hack on these things. So it sort of goes back to, what is it, Goodhart's Law? Yeah, Goodhart's Law. Maybe we could quote it out. Let's see, let's say any metric that becomes a target is no longer good as a metric. Yeah. And so that's the, I mean, it's good. And it's just that you have to be very, very careful. and you have to, again, you have to have like layers of rigor. Like, okay, but we'll do this and we'll optimize for this.

37:12But you have to realize, okay, that's just now Goodhart's law applies and you have to be careful. And so that's a lot of reasons why the whole AI field has been kind of constantly exhausting these things because, again, Goodhart's law applies individually to every leaderboard you make. So, again, it's sort of you just have to step back and be very, very careful. Maybe that's not, I don't know. No, no, no.

37:35John Platt:No, that's really useful to my thinking. As we've had guests on, it's been a recurrent theme of how do you manage all this, the complexity that's introduced by LLMs in agentic science. I think my follow-up question was about overfitting in Kaggle. Yeah, it is. If you had an auto Kaggle problem and then the question is, given auto Kaggle, how often was it successful? I mean, I assume you probably just ran this on like all of your Kaggle competitions or something. Well, we tried it on various, like what they call playground competitions, and it did very, very well in playground competitions. We've entered into different competitions, some of them.

38:15It turns out there's, in the last few years, just the number of leaderboards and competitions and whatnot have just exploded far beyond Kaggle. So we've done very well in some of them. Like one thing we're super proud of is the whole CDC set up this competition where you try to predict next week's the number of COVID and flu cases that will happen in every state and territory in the U.S. And you try to predict a week in advance. And Ira did super well on that. It's funny because in some sense, Google invented the concept of using data to track disease progression with Google Flu. So it's kind of funny that you were sort of going full circle 20 years later or something like that.

38:59So that did very well. Other ones where we've entered, we weren't quite as good often because people are very, again, you know, you have to sometimes, sometimes how well you do in these competitions is a measure of how much sort of TLC you put into it and how much you're willing to squeeze the last.001. And so it was, I mean, Ira did well, it got you pretty close, but we didn't close the jump in the last, whatever, 30 places or whatever, because no one was there to shave the last, you know, point one off the thing. Yeah, yeah.

39:33John Platt:Is it very iterative? Like you get to, you know, I saw the charts in the paper, and, you know, you sort of get these step changes as it discovers something, and then flat. And then, so is it very much human in a loop? Like, okay, you've stalled on the problem, like try this kind of thing. Okay. Yes. At the outer loop, which is, I think that's almost like the more fun, creative part. So yes, oh, here, look at this paper. Oh, you're doing something bad or, you know what I mean? So it's almost like having a hyper eager grad student or something who doesn't sleep. And you sort of tell it things and you sort of guide it around.

40:13John Platt:How often does someone intervene versus, like, what does the outer loop look like, actually? It might run for a few hours and come back and give you some examples. And then you would, you know, you can do it as far as you like. You can sort of keep trying and keep poking at it. So that's, it's very much designed to be human in the loop then? Yes, yes. Interesting, because a lot of the other tools that I've tried tend to be very one shot. Well, I guess it depends on your definition, right? I mean, it's obviously you talk to it and you start it and it'll go for some number of hours and come back.

40:51And then, but of course, then you say, but then that's where the human creativity kicks in. And then you're sort of doing the outer loop where you sort of every, you know, depends if you want to sleep, but you know, every few hours you go and you give it another try and you.

41:03John Platt:What kind of budget are you giving this thing? Like you blew through a million dollars accidentally kind of thing. I don't actually know because we're using sort of, you know, internal calls to Gemini. So actually, I don't actually know. So, but look, there's token budget, but then there's also like, I'm solving a problem that is computationally expensive. Oh, yes. That also, essentially underneath it, because the scoring function itself might have, you know, Monte Carlo estimation or whatever. Yes. So you actually end up, you can actually end up using a lot of compute to just even do, or simulation.

41:38like if you have a simulator inside, it has to run a simulation. So yeah, you can spend a fair amount of just CPU or GPU.

41:46John Platt:So my little experiment with Aira and COD is to build a neural network for some classification problems. And so they obviously, like, if you have enough data, then, you know, larger networks work better, but they're more expensive to train. And you get to, you start to run into a question of how do I manage my budget if I have a fixed budget so that I'm spending my dollars on the most effective solutions. That's right. And I think that's still something we need to figure out. But it's, of course, it's no different than if you have a grad student and they're trying to train a very, very large neural network or a very, very large data set.

42:28They themselves have to, there's something like, oh, is there a scaling law? Can I extrapolate? So it's not, I guess it's the same problem, but maybe more urgent because it just runs into this problem. Because it's so relentless, it runs into the problem much quicker than a grad student could.

42:43John Platt:One of the things that you optimized was contrails. Can you talk a little bit about that? Well, let me maybe spend a minute or two talking about the contrails problem. Yes. For context, contrails, not chemtrails, which is a conspiracy theory. Yes. Although you should also dislike contrails, but maybe not for the same reason. So contrails are, if you've ever seen those white clouds form behind jets, those are called condensation trails or contrails. And it turns out they add, at least according to the estimates that people have, about 1 % of all anthropogenic global warming is caused by contrails.

43:24Why is that? I can just talk about maybe the physics of that. So it turns out that there's actually two countervailing effects. Contrails are, well, sometimes if you've ever seen them, they're streaking and they kind of go away. Those don't really do anything. But sometimes they last for a long time. You'll just see in the sky just almost like a waffle of just persistent contrails, they're called. And there's two effects that they have. Those are thin white clouds, so they reflect sunlight, but that only, of course, happens during the day. It turns out all objects emit something called black body radiation.

43:59and the earth does at whatever the temperature is, about 300 Kelvin. It's in the far infrared, around 10 microns. And at those wavelengths, contrails have very low albedo. They're almost essentially black. And so they'll absorb a little bit of the outgoing infrared radiation and then re-emit it both directions. So essentially they'll reflect some of the outgoing heat, so it'll trap heat like a blanket. And so because that happens 24 hours a day, they tend to be warming. And it turns out it's a surprising, again, there's some uncertainty about it, but, you know, contrails cirrus, cirrus that sort of comes from contrails, might cover, especially in places like Europe, which has a lot of air light traffic, a few percent of the actual sky is covered by sort of additional contrails, which so it adds, in those places like Europe, it adds about one watt per square meter of forcing, locally at least, which means that, just to give you a sense, all of anthropogenic warming sort of average across the whole globe is about three watts per square meter.

45:05So in places of high airplane traffic, it can be a lot of warming locally. So what can you do? Well, it turns out contrails are caused by areas in the atmosphere that are ice supersaturated. They're a little bit like rock candy. So like when you have rock candy, you get a water solution that has too much sugar in it, and any little, you know, little bit of sugar in it will just crystallize all the sugar out. Just like in this contrail, these regions, they tend to be kind of pancake-shaped, only a few hundred meters tall, and if you fly through it, the jet exhaust has a little bit of moisture in it, which will turn into droplets and then freeze, and then for every gram, if you're in this bad region, for every gram of water, ice, or soot you put out, It's about 10 kilograms of water gets sucked up.

45:56So there's this enormous 10 ,001 curing ratio. So it's a big problem. So what you can do is you can figure out where these regions that are invisible, of course, these regions of icy-percentrated are, and then tell the plane to go underneath. And you only have to drop essentially what they call two flight levels. So it actually, it costs a little bit of fuel, but not very much to kind of avoid these sort of bad regions. So we built a system that sort of looks at satellite images and tries to detect where contrails are. So we have essentially a continuous monitoring system and then try to build a model of, because it turns out the weather models are not quite accurate enough to find these places of ice supersaturation.

46:40So we built a custom model, again, like a convolutional net or a UNET or something, to essentially to predict where they're going to happen so that, and then we give maps to a flight planning software so that they can dodge it and inexpensively reduce the climate impact of aviation by a lot. What's the physics behind why you can predict that?

47:02John Platt:Is it just, I see it in the satellite, and then tomorrow I think it'll be there because planes go to the same place? No, it's because you're trying to detect these regions of ice supersaturation because they're very, very persistent. Oh, they're persistent. Oh, yeah, yeah. I mean, no one knows exactly, but they could last for days. Essentially, they're caused by, they think, sort of warm, moist air being injected just at the boundary of the tropopause, just at the bottom of the stratosphere. And then when humidity gets up there, it sort of sticks there for a long time and then gradually dissipates.

47:34John Platt:Got it. So there's just a sort of... They're like bad spots in the atmosphere you don't want to fly through. Right. Okay. And so once you've established that, it's probably good for a couple of days at least. Well, you have to keep predicting where that is. Yeah. And the models you're using are, you mentioned like CNNs or something like that. That's right. And we haven't replaced those with era level models yet. But there was a very interesting problem that came up, which is you sort of want to know, well, just how much warming did this contrail make and how much did it add to global warming? Because, for example, you might want to find the biggest ones because there's some fuel cost and maybe it costs a bit of money for the airplanes to avoid it.

48:17So you say, well, gee, I'd like to kind of know how much it did. But that's actually what they call a counterfactual problem. Like, okay, you made a contrail and a certain amount of infrared radiation happened. So we can measure that if you're careful. but would have happened if there hadn't been a confrail there. That's a very difficult thing to estimate because you can't access the universe where... That didn't happen. That didn't happen. So you have to make these things called counterfactual models. And those are actually, I don't know if your listeners know, counterfactual models are actually pretty tricky to fit and make.

48:48And we were, remember, there were two reflecting the sunlight and then there's the infrared. at. It turns out the measuring what the effect of reflecting sunlight is actually more difficult. And we were actually stuck on it for two years. We had a model that worked okay, a counterfactual model for the outgoing long wave radiation, but not for the reflected sunlight. ERA actually helped us find a model that sort of searched all the confounders and sort of figured out like, oh, how can we estimate it? Because we had, again, we even had like test code on sort of artificial because you can kind of inject artificial data sets where they're sort of injected contrails and sort of figure out, oh, well, we know how much it was because we injected it.

49:34And so, again, our own attempts didn't even pass our own tests, but ERA's thing actually did and sort of unstuck this problem. So, yeah, we're in the middle of writing up a paper. We have a paper about the outgoing long-wave radiation, but we have a paper that, it's not submitted yet, but we've talked about it at EGU, I think, where we actually solved this problem.

50:00John Platt:And the models that Eric comes up with, are they just like a big monstrosity of code or are they like pretty basic and it was just you needed the intuition to develop? Yes, it's actually in this particular case, it was actually more of the latter that it essentially sort of helped identify what the, it was a very simple model with some, just some number of confounders that we just hadn't tried that combination before and it worked very, very well. So yeah, it actually sort of came up with the, and it was sane in retrospect. So that was, I think, a big win. Yeah, that's interesting. I know you've done a lot of work in climate.

50:35John Platt:What other stuff have you done? I think I talked about this, right? I talked about the CO2 thing. That was pretty fun because it's still quite a, the reason why estimating CO2 in the atmosphere is an interesting problem is we actually don't know what the carbon flux is in and out of the biosphere. I mean, we do. We know that the biosphere captures, right? We emit a bunch of CO2 out into the atmosphere and some of it gets absorbed into the ocean with sort of mostly inorganic chemistry, some phytoplankton, and a lot of it gets absorbed on land. But the error bars about what happens are moderately large And the error bars 50 years from now are very large.

51:20Like the models in 2100, we don't know how the biosphere will react to the ever-increasing temperatures and CO2. So we don't actually know how much the CO2 will absorb. And the error bars are 300 ppm of CO2 just from the uncertainty of what gets absorbed. And just to point out, you know, right now there's about, what, 440, 450 ppm. So it's huge. I mean, it could be seriously, amazingly awful or not great, but, you know, the 300 ppm is like enormous uncertainty. So it'd be really nice to figure out, you know, can we reduce that? So this is like the CO2 concentration is like one step towards that, solving that.

52:07John Platt:And I know that Google has made some really big improvements in climate modeling and weather prediction as well, right? I was at NeurIPS this year, this last NeurIPS, and I stopped by the climate track. And, you know, I maybe only had a chance to listen to talks or something, but it really blew my mind the sort of step change I think that's happened in the past, I don't know what it is, maybe 10 years or whatever. In terms of climate modeling, I know a lot of that happened at Google. Can you talk a little bit about what has happened in Google and other places that has made that, allowed that really big transition in climate and weather modeling?

52:51Okay, so let me, people often sort of collapse climate and weather together. Well, because they're fundamentally the same physics. Yeah. Although, at least for the atmospheric physics, they're obviously when you start having ice and land, you know, climate is long-term weather. And so the complexity of a full Earth system model, which is a climate model, is much, much bigger than an atmospheric model. Like you have to actually measure what's the water flux and the CO2 flux in and out of the land or what will happen with ice. And so there has been a step change with weather models. Sorry, I want to make this.

53:28Weather is up to 15 days approximately because, you know, weather itself or the atmosphere appears to be chaotic. I'm hoping, I don't know if I should explain chaos. Essentially, it's the butterfly effect, right? that small perturbations, like a butterfly flaps its wings and the weather will be completely different in two or three weeks. So weather is trying to predict the actual trajectory of the atmosphere over, say, two weeks. And that's now, that has been a huge step change. And that's because that's been a lot of not even the new LLM stuff that was based on the 2018 era machine learning stuff and just a large amount of data and a large amount of compute.

54:12So there's been a lot of sort of very clever work and a lot of it from Google making new weather models. And it's been great. And in fact, we had a really neat breakthrough because now we can apparently predict tracks of cyclones, tropical cyclones, much more accurately many days in advance. And so places like Jamaica got hammered by a terrible hurricane. and a lot of the classic models didn't actually predict it, partially because it's often, especially the intensification, it's all being driven by what the surface temperature is. Because hurricanes, people might not realize, are heat engines.

54:52Essentially, they convert sort of heat in the ocean to big atmospheric motions. So weather's been great. Climate is much more difficult because you don't actually care about, you're not trying to predict whether it's going to rain in Seattle in 2070. you're trying to get kind of like averages and what makes it difficult is that it's what they call non-stationary so that in fact literally it's like the underlying physics or the underlying like you know plants are behaving differently and ice behaves differently and so it's very very difficult to use sort of classical ML on sort of true climate models and so that's why sort of the whole discussion you guys had about you know what we're talking about about descriptive models and multiple hypothesis testing, that is incredibly severe in climate because we have no data from 30 years from now, and we don't want to wait 30 or 50 years to find out whether we were right or that we overfit.

55:49So whatever things we do, you have to be kind of careful and try to peel off sub-problems and the problem of exactly how do you inject, how do you build a big model that can predict into the future but is still constrained by what we know. So it's a fascinating problem. I think it's still unsolved, but it's a great problem to have because, again, these uncertainties. We really would like to know what will happen in 60 years to the climate. So it's still, it's a thing, it's a very, very interesting problem to work on. But so far, AI has not revolutionized it because it's very, very resistant, again, because of this data problem.

56:26It's a low data problem.

56:27John Platt:Does the butterfly effect, the chaotic nature of weather, does that also impact climate? Or is the timescale so large that you have a closed system for which, you know, maybe it's oscillating between poles or whatever, but it's sort of, when you look at it at that timescale, it's more stationary. It's unfortunately non-stationary in a different way, but the original sort of whole chaos thing was, well, maybe many people came up with it, but in meteorology, it was back to a person named Lorenz who had this sort of very model, very simple model ODE. So the difference between climate and weather is weather is where are you on the attractor?

57:09And climate is about the statistics itself of the attractor. The problem with climate is that we're altering it so the attractor itself is changing, is moving. And there could be, everyone talks about tipping points, that means that the attractor suddenly changes. And the trouble is that's very, very, very difficult to predict.

57:27John Platt:So even the attractor's shape changes quickly. Or could. Could. And the trouble is when you run a simulator, you don't know, like, is it, did it go unstable because my model's not great? or is it an actual physical instability? Interesting, yeah. And it's extremely difficult to tell the difference. So what do you do, especially when, I mean, to me it strikes me that not only do you not have future data, you really don't have much past data. You can do some measurements and ice cores and lots of stuff to try to do that, but there was nobody with an instrument 100 years ago. That's right. So if you have annual data or whatever, maybe you have, if you're lucky, 50 data points in any one location.

58:06Right. So whatever we do has to be very constrained by what we know, but it's just very different. I'm just telling you sort of the horns of the dilemma people on. So people make these, in fact, people in applied science in general, I would say climate is the most extreme, make these things called process models where what you do, and I've seen the code. Oh, well, you know, I'm going to be reductionist and I'm going to sort of take the horrible complicated climate thing and sort of boil it down to a thousand pieces. and then I'm going to, you know, find the expert who wrote a paper about, you know, piece number 763 and he fit a cubic to some data.

58:44Like, for example, one thing that's very mysterious, which is related to contrails, is how does ice behave in clouds? It turns out, you might, again, everything is complicated once you dig into it. But it turns out that, like, when you make a contrail, how long does it last? Well, it depends on, because the way contrails can evaporate is ice starts to accumulate, as I said, and then the ice crystals get big and then they fall. But of course, how quickly they fall depends on their shape, which is not known. And how much does the contrail mix from the moist inside the contrail out? Again, people have approximations, but they don't know.

59:24And so the uncertainty is very much confound. And it's not just, oh, John, who cares about contrails? It turns out that the actual physics of microphysics of ice has very strong implications about what climate models do. And it's sort of, we just don't know. So I'm trying to say it's very gnarly and it's not a solved problem. my hope is that with tools maybe not like today's era but maybe tomorrow's era because it remember you can as I was saying before it not only can fit data it can read papers right and the question is it can read a lot more papers than we can so maybe we can integrate all the data or all the knowledge that people have carefully evaluated much more than any one person writing a piece of code and fit data I mean, that would be utterly glorious.

1:00:16We don't have that today, but that's sort of one of the hopes that I have, even for a more amazing tool in the future, is something that really can write code in a sane way, even much more sane, because it'll be constrained by all the scientific knowledge that we've accumulated so far. That would be amazing. We don't have that today.

1:00:35John Platt:I mean, that strikes me as being very similar to biology. Oh, yes. Oh, boy. Right? If you've ever played biology or even looked at biology, there's so many exceptions and so many hacks in the biological systems. Yes. Oh, boy. So, yeah, it would be amazing if we could have a thing that could really integrate all known scientific knowledge with data and try to synthesize sort of new models and new things. I think what you're saying is that AI can be an unlock here to some extent because the models are so piecemeal, necessarily piecemeal. And so that being able to assemble the jigsaw puzzle, not to mix metaphors, but to assemble like a really, this jigsaw puzzle, having just scale and capacity actually helps a lot.

1:01:28That's right. The one thing that these AIs have is somehow, you know, humans, even I'm pretty well right, I think, but it's just difficult for me to kind of integrate across the N squared different. of papers I've read in the life. It's pretty big. And it's just difficult for me to even do that N squared thing. But somehow there's just so much data in those billions and billions of parameters. And you can also give it access to read PDFs that it can somehow start to pull things together that people wouldn't do it. So that's, again, I'm starting to see little indications of that inside of ERA. I'm not claiming that's what ERA does today.

1:02:05But yeah, that's sort of my hope of where this is going to go. I think I've heard a lot of people suggest something like the route to intelligence is to combine LLMs with some form of search. So it's actually amazingly like you're doing that, right? Something which, you know, is maybe a very strong database lookup with a good search algorithm is one way. And of course, I mean, there was the whole, I mean, people still do, I guess, the whole RAG thing, of course. Yeah. And if you think about it, Google itself, you know, the 10 blue links things, it was or is a form of AI before we had LLX, right?

1:02:40Because it was like, you can cast yourself back to whatever, 2010 or 2015. You could ask Google about literally anything and it will tell you stuff, right? Surprisingly well, actually. Yeah, surprisingly well.

1:02:52John Platt:Because somebody on the internet has written about it probably. That's right. So if you can match that. In fact, that was one of the reasons why I wanted to come to Google. It's just that was such an amazing thing, right? I wonder how much of our audience did a search pre-Google and just know how bad that experience was. Yeah, I remember in 1998, I think, I think it's when Google, like I was using AltaVista. Yeah. And like, I don't know, Google just got released and I used it and I just, sorry, digital. Yeah, I remember that too. I just dropped like a hot potato or something and started immediately using Google.

1:03:27So that is sort of a form of AI. And so, yes, it might be, yeah, Yeah, it could be that just having access to all of that and sort of keeping it in mind at the same time.

1:03:38John Platt:That model of sort of scientific discovery in as much as it pans out is kind of comforting too because it is reductionist so that you can look at the individual parts and understand them. So it found the exact things to assemble, but they're all actually maybe fundamentally things that people have invented or it's done iterations on. And so all those little pieces are individually understandable, and then you can also put them together into a coherent picture. I think for a lot of problems like biology or climate science, I don't think we would trust the answer unless it was in that shape. Yeah.

1:04:15Because if there was some giant black box model that said, oh, this is how a cell works, it's like, do I believe it? I mean, I don't know if I believe it because I can't examine it.

1:04:24John Platt:But I mean, to argue against it, though, if it works really well. but you'd have to gather, you'd have to, I mean, you have to test it obviously but a statistical model, again, it has to extrapolate. Yeah. Yeah. And it has to extrapolate to the extreme or the sort of the black swan events. That's right. Yeah. And so it's very hard. This is why things like self-driving cars are very, very, it's a very difficult problem. Right. It's all corner cases. Yeah. It's kind of amazing how well they've done. That's an interesting point thinking about like when, you know, coming from the world of physics where a model was, usually a single equation or a small number of equations which uniquely define a system and everything about it and you just crank you just find a solution to the system and you you now know everything you need to know and i think something like alpha fold was kind of a shift for a lot of people where before they thought oh protein folding is you know a problem where you just if we find the right force field and we have the right computational engine we will protein folding.

1:05:25And the thought of even really solving it in a data-driven way was only appeared a few years before, you know, AlphaFold, AlphaFold 1 came out. And it's interesting that I think it's sort of forced the new AI modeling has, I think, forced people to reevaluate almost what is science because AlphaFold is, and similar models are incredibly powerful. There's a lot of think that they've opened up as tools, but at their core, they oftentimes don't give intuition in nearly the same way that, let's say, the most physicists historically would have wanted. And so I guess it's sort of, there's this old saying, all models are wrong, some are useful.

1:06:08Yes, Box said that. Yeah, yeah. When do you find the data-driven models to be sufficient? And when do you want sort of like something which is interpretable that humans can actually understand? I think it boils down to almost like the difference between weather and climate. If you're in a data-rich regime, like weather, or even proteins because of PDB, you can feel, oh yes, in other words, I've got enough data to kind of cover, and so a statistical model like AlphaFold should do. And so, in fact, a lot of people happily use alpha. You know, I think it's really revolutionized my understanding. I'm not a biochemist, but people seem to love it.

1:06:48And one amazing thing they did is they exhaustively, they just ran it on all PDB and published it, which is just really, really cool. It's like 6 billion protein predictions. The vast majority are actually quite accurate. Yeah, so that's just amazing. But it feels closed, if you know what I mean. But when it's like climate and it's open and it's non-stationary or you have to make these big extrapolations, you have to be much more cautious. Or maybe biology. Again, and there's probably, there may be parts of biology like, oh, there was this virtual cell challenge from the ARC Institute. That had a funny result.

1:07:25I know that people were, there may have been some overfitting for at least that. Yeah. What would we say more? Sorry. Or I guess maybe at a high level, I think simple baselines. Oh, it worked very, very well. Yeah, yeah. Yeah, yeah. Yeah, just like in the... One of the classic things in whenever you do biology is just always start with a simple baseline. Maybe this is probably just good ML in general. It's good ML in general. Yeah. Understand your simplest case in biology. There are many problems where they're extremely resistant to anything beyond the simple baseline, even if you have a lot of data.

1:07:54That's right. And in fact, I tell people the same thing. I said always just fit linear regression. Just fit... Just do it. Just do it. Just do linear regression. Or an SVM, yeah. Or SVMs. I mean, SVMs are just a different... Different form of... Linear regression, yeah. Yes, so when do you need the more process model-y thing? I think it's just when you have, I mean, sort of climate is on one end, and I don't know, weather maybe on the other end. That may be too extreme, but I think it's where are you on the data richness thing? When can you feel like, oh, no, I really have a closed problem, and I think I can actually cover it?

1:08:32Yeah, a closed problem that data fully covers is, yeah, I think that that makes a lot of sense in what I've seen as well. Yeah. I think one thing I'm kind of curious about is when you're working on climate modeling, what are the, you talked about contrails, you've talked about CO2 predictions. What are the broad things you're trying to accomplish? So one of them is, I guess, making up, like making interventions and the other one might be making predictions for things like insurance or like how do you help adjust for some sort of climate change? Or what are the principal goals, I guess, for you specifically or the community at large?

1:09:15I think, you know, just like any community, there's probably many different goals for me and my team. We're very, very interested in interventions. so like which ones are possible at relative cost. I mean, contrails was kind of amazing because it turns out the intervention is quite low cost and also one amazing thing about contrails is they are local unlike things like CO2 because so if a country decides to fix contrails over itself it actually improves its, I mean it has the global effects but it mostly improves the climate a little bit over themselves so they like that. I guess that if you were in a cold climate and you want to warm it up, this is now your own, you could know.

1:09:58Yeah, it turns out that it's a little bit asymmetric. The warming is constant, essentially, and global. The cooling only happens when you're sort of at a good, when the sun is at a good angle over you. So it's very rare that the uncertainty, there are contrails that where our uncertainty bounds in terms of the warming. There are many, many contrails, mostly at night, of course, where it's largely warming and we're very sure in terms of two sigma. There's not very many contrails where you say, oh, I know for sure that it's cooling and I want more of it. So only over the poles in polar summer do you know that the contrails are cooling.

1:10:41And therefore, if you got rid of them, they would warm up. But there are essentially no flights over Antarctica and not that many over the poles. in the summer. So no one who lives in a cold climate is going to use this maliciously? Well, yes. Well, they wouldn't know for sure whether it was warming or cooling, and so they would do stuff. So mostly we just sort of ignore, we don't recommend that people fly those. You also brought up an interesting point about the economics. I mean, a lot of, there I think was a lot of resistance historically about certain climate change interventions, which have, in some sense, the market has just taken over.

1:11:21Like, at this point, unambiguously, like renewables and batteries are just almost universally, unambiguously, just better than alternatives. For non-mobile. I mean, even for mobile. That's a really good point. Yeah, yeah. Like planes, we do not have a solution to. Correct. I mean, there are some battery-powered planes, but they're very small and have to go very limited range. They probably will never actually be. It's hard to imagine. The physics would be very, very, Unless we came up with something like nuclear batteries, which would be kind of amazing, but we don't know how to do that. Or even if we did, I think the risk of people would be too afraid of a nuclear battery going wrong or something.

1:11:58Oh, yeah. Since we don't know what they are, we don't know what the risk is. I guess we don't have the risk. Yeah, so we don't know. So, yeah, that's the problem with it. I have given talks about climate change, and I talk about the pie chart of sadness, which is there's no one silver bullet for climate change, right? There's so many different things that contribute greenhouse gases just from across our economy. So they sort of all have to be fixed or many, many of them have to be fixed. So there's no one single thing. I mean, I worked on fusion. Fusion is cool. And it might actually knock a lot of them out if it's cheap enough, which we don't know because we don't know if it'll work yet.

1:12:34I mean, fusion is one of those interesting things where the joke was always fusion is 30 years away. But I think it's actually now less than 30 years away, maybe. Yeah, no, I think there's a definite probability that someone will make commercially relevant fusion even by the end of this decade. So I think it's like three years away, not 30 years away. It's very real. Interestingly enough, I think a lot of that, I'm going to not just stump or advertise some of our other episodes, but a lot of that actually comes down to material science, interesting enough. Well, I'm skeptical. Oh, superconductors, sure, sure, sure.

1:13:09Well, yeah, sorry. We can talk about fusion if you want to make. Yeah, yeah. There was actually two things. One is fusion. The other is better control systems, which I think is actually up to us. Yes, and in fact, Google DeepMind has been working on control systems for tokamaks to make sure they don't essentially go unstable and go disrupt. Yeah, that's right. Disruptions are quite interesting to themselves. Yes, yes. Yeah, it's basically the entire, all energy in the tokamak collimates into one little beam. And then it hits your vacuum chamber and you're very, very sad. You're very sad. Yeah, yeah.

1:13:43I think people believe that either could turn on after$30 billion disrupt and then basically have a$30 billion brick or something. Oh, yeah. Yes, I guess you could try to patch it. I remember working, again, before LLMs, we worked with a fusion company called TAE and I was in their control room. And yes, it was kind of sad. You have to be very careful. we were making systems to recommend new experiments. And they were very, very skeptical and jaundiced, which they should, because I've even been there. Even under human control, it's like they were doing some experiment, and then you hear this big bang.

1:14:20And it was like, oh, no. And then it's like, you know, then the apparatus is down for two weeks as they patch some. You were there during a disruption? Oh, no, this is, sorry, they have field reverse configuration. Oh, okay. I got it. Which has its own, I mean, things, you know, there's some arc. So what is that?

1:14:36John Platt:Sorry, I'm not familiar. Oh, what's the field reverse configuration? Well, it turns out tokamaks are not, although perhaps the most studied form of plasma, there's many different kinds of architectures, essentially ways to try to stabilize and compress plasmas. There was a shape, essentially, it's essentially a self-contained football of plasma called the field reverse configuration where essentially the magnetic field inside and outside are opposite. So they're separated by something called a separatrix. and that is sort of in theory unstable, but in practice stable. Like, for example, when you run Magnete Hydrodynamics MHD code, it's unstable under that assumption, but that's an assumption.

1:15:15That's not the way the real world works. And so, yeah, it was kind of disfavored for many years, but TA and other people, I think Helion, have FRCs because they are actually relatively robust. You can actually knock them against walls and they'll stay stable. Yes, but you can still get discharges and things that punch holes in your vacuum chamber, which is kind of unfortunate.

1:15:39John Platt:For clarification, so you have these fusion reactors. They are, or trying to be reactors maybe. Apparatuses, yeah. Apparatuses. And you create a plasma. The plasma is magnetically charged. Confined, yes. So it's confined by a magnetic field. So you have some sort of magnetic system that is tunable by a computer, and then the computer tries to kind of maintain the confinement. Well, FRCs, once you make them, they're sort of sustained. There's different ways of trying to make sure you... Okay, so all of fusion boils down to something called the Lawson criteria. There's essentially, and it explains why fusion is hard.

1:16:25Essentially, you can just very easily on the back of an envelope just show that the density, the temperature, and essentially the energy loss, it's called the confinement time. It's one over the amount of time it takes for the energy to decay away, one over E in a plasma. So the product of those three numbers has to be bigger than some constant, and then you can get fusion. And if you don't, then you don't. and that explains the fact that it's a product of three numbers explains why fusion is so hard because every approach has an Achilles heel where one of those numbers is not very big and then they try to desperately make that be higher.

1:17:00And every approach to fusion is kind of different. And a lot of, you have to be a bit skeptical when there's all these sort of breathless news things about fusion because they'll say, you know, now confinement time is stable for X minutes or whatever. and it's talking about like one of the three numbers, but you have to have all three numbers before you can get fusion. I think the whole field is making a lot of progress and it's very exciting, but you do have to be a little bit cautious about the breathless news articles that only talk about one number.

1:17:30John Platt:So what is the computational part of that? Oh, unfortunately, for better or for worse, it depends on the approach. So for Tokamaks, as Brandon said, it's mostly stable, except that there's occasionally this instability that takes all the energy and smacks it into one place. And so you have to sort of keep everything sort of under control. So it's a control system. FRCs themselves have very simple instabilities. So, for example, they have what they call a Z instability. So it's fine, it's stable, it'll just wobble, literally wobble back and forth. But you just make what they call a PID controller that just keeps the football in the center of the reactor and things are fine.

1:18:10John Platt:And it does that by adjusting the magnetic field? Yeah, it's sort of, it actually just, I think the electric field is sort of, sort of knocks it back and forth. The issue that people have is it really depends on which sort of plasma architecture they're deciding to use. Climate is, you know, sort of political because of economics, basically, probably, mostly, maybe other stuff. But the economics of it, you know, you have to persuade people to somehow spend more or you have to have a solution that has like this happy coincidence where it's both economically better and better for the climate. That's hard.

1:18:49Yeah. But in many cases, it's not. I mean, in many cases, it hasn't been hard. Yeah. Yeah, I guess.

1:18:53John Platt:Well, you think about like, okay, predicting even weather, right? You can prep and you could see how that could be economically beneficial. So what kind of work are you doing with interventions and how does that kind of interact with economics? Like it sounds like the Contrails one, I did an analysis and said, actually, this is great because it's very low economic impact, but high value. That's right. So if you're trying to think, there's sort of energy intervention. And so you have to sort of compete with existing forms of energy. And that's not trivial, unless there's a co-benefit or there's some sort of clever, you know, just co-benefit.

1:19:35Like, this, again, is highly speculative. It wasn't our work. There was a startup that was, I don't know if you saw the news, it was last year, I think, where someone figured out if you inject mercury into a fusion reactor, that the neutron flux can actually transmute the mercury into gold. and then you can sell the gold, which I thought was very clever. It might not work, but... You know, as a physicist, the one thing I want out of a fusion reactor is helium, but that's a different story. Oh, helium, it's three. Well, I mean, the helium four is kind of boring, although it is getting, because the strategic reserve has been shut down, there's less of it.

1:20:09Yes. And of course, I want helium three, four. Helium three, yes. Well, not even just a fuse, just to make dilution refrigerators for quantum. Or MRIs or so much, yeah. So much technology we think about actually just goes out the window if we run out of helium. That's true. No one's thinking about it. Sorry, that's like a complete aside. Yes, the fact that the U.S. had a helium, strategic helium was to reserve, was for a very important blimp fleet. Yeah. But they kept it for decades anyway. So that was nice. But then we stopped. And then we got rid of it all. It all went up in the air. Yes, yes.

1:20:40In balloons and stuff. Yeah. Or out of natural gas wells. Yeah. Sorry, now we're talking about helium. Yeah.

1:20:47John Platt:So interventions, what are some of the most exciting, interesting ones? Well, I'm very excited by fusion. I mean, I don't know if it's intervention. That's sort of a source of energy. Because if we can make it work and we can make it be sort of low enough capital costs, that will actually help a lot. Because at least the current models are renewables are great. Ideally, you'd like to electrify everything, right? which has problems because you can't electrify flights, but you can try to electrify a lot of stuff. You know, there are EVs. You'd have to figure out how to electrify things like cement or steel.

1:21:23Those are hard, especially things like making steel reduction power anyway to essentially you're adding carbon and you're reducing iron ore. So there's a lot of sort of things that are difficult about electrifying everything. But if you could electrify everything, then the amount of electricity required would grow by a factor of five. and you could try to grow renewables. Renewables plus battery, again, trying to squeeze all of it out, it starts getting ever more expensive because you just need ever more, you need like a huge number of batteries to sort of cover the last few percent or even 10 or 20 percent.

1:21:59So we do need some sort of power that can cover the last 20 percent, something that's, you know, base load. So fusion might be a thing for that. So that's super exciting. Again, there's no one sort of silver bullet that can sort of cover all the cases. So I'm happy to sort of talk about any specific case, but it's sort of like the world is a very complicated place and the global economy is a very complicated place. So it's super hard to sort of talk about sort of interventions in general. Yeah. Maybe instead of interventions, one thing I'm curious about is how does this make affect decisions and to, for example, like what do we build?

1:22:36How do we build? I think you're from L.A., right? Or at least you... Well, I spent 11 years there, yeah. Okay. So, yeah, you spent a lot of your life in L.A. I mean, L.A. just basically large parts of it just burned down. And maybe probably close to where you used to live. So, you know, this is something that I think a lot of people kind of saw coming. Maybe partially due to regulatory issues, but partially due to other issues. You know, and we were completely unprepared. And it seems like there is a lack of preparation about what to do next or to sort of adjust for this. And I mean, have you worked on basically predicting like new risk assessments or, you know, suggestions like what do we actually change to maybe harden, you know, society even for what's coming, regardless of whether or not we're actually do something to make solve the underlying problem?

1:23:29That's right. So, in fact, there's a big effort at Google into something called crisis resilience. And so we had a very fun project called Firesat. I don't know if you know about this. So it turns out that for wildfires, a lot of these wildfires, if you only caught them early enough, it's very easy to put out a wildfire the size of this room. But even if it's like an acre, it gets much, much harder. And of course, under certain circumstances, they can grow exponentially from the size of this room up to an acre. So that might be hard to catch. But they often sort of start small and spend a while.

1:24:08So we figured out that, oh, if you had a global constellation of low-Earth orbit satellites that could detect in the mid-wave IR, which goes back to the blackbody, essentially that's the temperature of fire, the fire stand out in the mid-wave IR. And so we designed a sensor that if you built, it depends on exactly their orbits, but roughly 50 to 80 of them, you could actually find fires about the size of this room, about five meters on a side. maybe it's a bit larger in this room, five meters on a side, and anywhere on the planet, and again, depending on how many satellites you had, within like 15 to 20 minutes, you'd have to put a fair number up, like 80 to get them within 15 minutes.

1:24:58And then you could actually intervene. You could decide not to if you wanted to have the fire burn fuel and you thought it was safe, but if it was going to blow up to something unsafe. So we worked with now a nonprofit called Earth Fire Alliance that we're part of. And so they're starting to, we've launched one satellite, which is a prototype. We've worked with a company named Muon Space to actually sort of make the satellite. So that's cool. We have wildfire boundary detection and we propagate that information out through Google. So we can actually sort of figure out from existing satellites and existing data feeds where the boundaries of fires are.

1:25:34And then we sort of tell people through their Android phones or through search about fires. We've worked with the U.S. Forest Service on making new models for how fires propagate, because again, that goes back to these process-based models from the 70s by a person named Rothermel. So we've actually made a little neural network proxy model based on a new, essentially to sort of be able to run it very, very quickly. So we worked with the Forest Service on that. So yeah, yeah, we're very, very interested in trying to minimize, because it turns out people might not realize the World Health Organization estimates that there are 300 ,000 excess deaths a year across the world from wildfire smoke.

1:26:14Yeah, I mean, I remember, it's been a few years since we had a really bad fire season. Maybe, what, four or five years ago, there was this cloud of smoke which crossed all of northern the U.S. and Canada and caused a lot of respiratory issues, I think. Yes, yeah, and it's very hard to track. I mean, you have to get these, you have to estimate these excess deaths from statistical means. But yeah, it's a very serious public health problem and also just very scary and it burns people's houses down and it's terrible. My view is that climate change is sort of like a serious disease. Do you treat the symptoms, i.e., do you adapt or do you try to attack the underlying thing?

1:26:50And the answer is, well, if it's serious enough, it's both. Right. And so, yes, so we take sort of adaptation, especially around climate resilience, very seriously at Google. And we try to give people informational tools to sort of help. That's part of the reason why we were working on weather and sort of cyclone prediction and things. So it all actually hangs together. So it's more than just, you're right, it's more than just interventions. It's climate resilience too. Yeah, having lived through four or five fire seasons on the West Coast, they can be quite nasty. And it used to not be, I mean, I have a cabin up in the Sierra Nevada mountains.

1:27:24And yeah, it used to be, oh, you know, summertime, it's nice. and then now it's like, well, not every year, but yeah, there's like, you know, winter, spring, summer and smoke. Yeah, fire season. Yes. Yeah. I want to stay to the west of the fire line. Yes. Yes. So that is another thing that I'm interested in and that Google's also very interested in is climate resilience. Are these IR sensors small enough that they could hitch a ride in like a microsatellite grid? Like, would it make sense to, would it be almost cheaper just to hire, I mean, to pay someone who's watching a constellation? Oh, they're not that small.

1:28:02They're not that small. The thing is you need refrigeration because it's mid-wave IR. So you have to keep it cool. So these would have to be their own special satellites. They're not super large. Okay, yeah. They're not like the, you know, the satellites in Geosynchronous Orbit are these giant monsters because of all the optics and who knows what. but yeah. And they have, you know, they're basically just IR sensors with a resolution of five by five. No, no. That's the other cute thing is the resolution is about 50 by 50 meters, but you can use super resolution because it's essentially multi-spectral and you sort of know where fires are.

1:28:38So yeah, there's a fair sprinkling of AI on top of them to reach that five by five meter.

1:28:44John Platt:Oh, that's cool. Yeah, yeah. When you also have, you have a convolution over the, what you're reading out, right? and law. I forget the frame rate. The satellite is moving. I don't remember what the point spread function is. I'm sorry. But you're right, they do move. But I don't remember how fast they... I don't remember how fast they... This has also been called a broom sensor, so there's this funny thing of trying to... You kind of spread out the spectrum one way, and there's also... It's a somewhat complicated thing. It isn't just like a Polaroid. It's a complicated sensor. Yeah. Sort of switching gears a little bit.

1:29:21You know, you've been at the intersection of AI and science for quite some time. I think you've sort of wound your way into and out of it back and forth. How do you see the field has evolved? Because I feel like it's evolving very quickly now. And what are the sort of lessons that you've learned that you think the community has learned? And how do you think this should change? If you are a young scientist or young practitioner, how should this change how you should approach, you know, the future? Well, I think there has been a phase change in the last 12 to 18 months. So, I mean, a lot of what we used to do, as I said, was build these specialized models to solve individual problems.

1:30:05And if you think that's your job, it's kind of fun. You find a problem, you solve it. You find another problem, you solve it. But now we have these much more general AI things. And I think the whole AI for science community is kind of still feeling around. The fact that they're working is so new that collectively we're not sure, like, what's the best thing to do? Or maybe there's no one best. Maybe there's a tool chain. And I think we're all trying to figure out, like, what should we do? And so there's a question of what should young scientists do? I think it would be, you know, I have a son who just turned 21, and he's really into both sort of AI and coding and chemistry.

1:30:49And I look at him, I think he's doing the right thing because he's both learning a lot and trying to be a domain expert about RNA, but he's also sort of using Vibe coding and using all the tools. I think that's the right answer is because everyone's figuring it out. still be a deep domain. I don't think domain expertise is going away because it goes back to a lot of people who said it goes back to taste and trying to figure out how people get taste without doing all the grant work. That's an interesting open question, but develop domain expertise, but also try and play, I would say, with all the different tools that are available because it's not like, oh, yes, we know what's going to happen and the smart old people are knowing what's, no, we're experimenting too.

1:31:34And so, yeah, so I would say definitely develop domain expertise and try to use these tools and try to solve big, hard scientific problems as best you can. There's still the huge open issue about what do you actually do about physical lab work that is not going away because, you know, experiments are the ground truth. And a bottleneck. And a bottleneck. I mean, people are talking about lab in the loop, but that's still very, very, very open because no one has, as far as I know, a general lab that does everything. There's a lot of very specific labs that are controllable. So I think it's just we've gone through this phase change.

1:32:16It seems super exciting. Again, I would advise people to play with whatever tools are available and to develop sort of deep domain expertise and taste to the extent you can. And I would advise people also not to be scared. Try stuff. You know, I'm always happy we have student researchers at Google and they come and they do sort of wild and crazy things. And that's always just delightful. So, yeah, people should be trying sort of wild and crazy things and see what happens. This may be a question without an answer, but when I think about how I developed expertise and how a lot of people developed expertise, it was by starting with a, you know, a simple defined problem and then hammering it.

1:32:57And then in that process of exploration, you learn more. and, you know, some ways you go broader, some ways you go deeper, but you still, the process of just banging your head against a problem, which now would be instantly solvable, teaches you the skills you need to solve harder problems. I mean, what advice would you give to your son for that? You know, I don't know. Maybe it's a bit like hiking, which is, yes, I mean, you obviously can't drive everyone. You could drive up the mountain. Yeah. Or you could hike up the mountain and maybe it's okay, even fun, to occasionally hike up the mountain, even if you can drive up the mountain.

1:33:30And yeah, I mean... In the old days, I'm going to sound like a real old man. In the old days... In the old days of six months ago. Oh, no, no. I was even thinking of in the old days of the 80s and 90s. Like, you know, a lot of people take it like, oh, there's open source packages. There's Glearn. There's all sorts of things. We didn't have that. I had to write my own numeric library. I had to write my own machine learning. I've written Boosting, probably rewritten it four times, four different languages. And so now I know Boosting, you know, it's like... And so maybe not taking the totally easy, obviously, I mean, there's this trade-off like, oh, but I want to be as efficient and productive as possible.

1:34:04Yes, but you also have to develop the muscles. So it's a little bit maybe like being an athlete. Like there are times when you're actually doing exploit, when you're trying to run as fast as you can. And then there's also training time. And so maybe people just have to train. And it's entirely plausible that if you spend time actually hammering away and doing the hard work, even if it goes slower there, that pays dividends into your larger, you know, productivity long-term. Like even if locally that one moment you are not being maximally productive by not exploiting an LLM, that feeds into something.

1:34:37I hope so. I hope the thing I don't know is I hope people in their careers, it's hard because, right, the whole world seems to want to optimize everything. It's your fault. No, I know. No, I know it's my fault, But it's just sort of the, you know, and sometimes you have to set aside time. Like at Google, especially in my group, we have this concept of 20 % time, which I still, I very, very strongly in my own group try to protect. It's like you can do whatever you, if you want to learn stuff, if you want to try stuff. You don't even have to tell me. You don't even have to tell me. In fact, probably shouldn't tell me.

1:35:13You know, just do stuff for exactly for learning. And also because that's where the sort of creative juices are. I don't want to so occupy people's time where they have nothing like, where they can't feel like they can play or learn or try new crazy things. So I know 20 % time is unusual and there just seems to be this strong impetus in the world to just, like I said, optimize and squeeze everything out. But you do lose something when you hyper-optimize. You sort of overfit. Yeah, you overfit. Exactly. You overfit to productivity. Yes, that's right. So I know that my advice might be swimming upstream against perhaps cultural norms.

1:35:50John Platt:The thing that always comes up for me here is that the problem is not stationary. There's a new skill set that will be the right skill set for the future. And the question in my mind is always just detangling, okay, is this a skill that is an enduring skill? Yes. Or like maybe it wasn't enduring yesterday, but today it will be enduring because like I've seen that, you know, like for just as an obvious one, you become sort of like a manager when you're using LL. Absolutely. I tell people this. A lot of those skills transfer well. Some of them don't, but a lot of them do. And so that as a manager, you lose track of the details of what's going on and you trust your people or agents or whatever to have that managed so that they can report up to you and answer, you know, sort of the high level questions and get the judgment about the little things correctly.

1:36:51John Platt:And so that is that what we've come to? Is it we're just like middle managers now? Well, I mean, I don't know. Again, I have a little management work, but you can't, I don't, you don't want to be an empty suit. In other words, because the things might get it wrong, especially LMs that are sort of really weird. They don't make the same kind of mistakes that humans make. And so you can't, you know, fully trust them. You have to be rigorous and like, you know, poke at it and make sure. Although you should be poking at software that you write yourself to. I mean, you shouldn't trust yourself. That's one thing I've learned.

1:37:27What did Feynman say? You absolutely can't fool yourself and you're the easiest person to fool. So that might be enduring. I don't know if I can quantify what's enduring, but somehow fundamentalness, I mean, really learning a domain that is about the world, for example. So this is why I like, well, biology or physical sciences, I think those are enduring sort of fundamental things. math is very enduring, but even things like rigor and checking and that sort of thing, which goes back to maybe management that, that you want to really make sure that the LMs are producing the right things or they haven't cheated in some way.

1:38:08But again, you should be doing that to yourself too. Yeah. So I think there's some enduring and also just the enduring value of sort of creativity and thinking out of the box. And so I think those are enduring. I don't know. There's some, there's something very fundamental about all of those.

1:38:22John Platt:So you mentioned Feynman, if you don't mind me changing gears. Yeah, yeah, yeah. I took a class from Feynman. Yeah, yeah, they were going to ask you. Oh, okay. Not just any class. Yes, his sort of physics of computation class. I did it with, in fact, Hopfield and Carver Mead. That was fun. At the time, I don't think, maybe Feynman knew. I felt like none of us knew what even the problem was. I mean, I guess Feynman was trying to say, oh, let's do quantum simulation, which I guess is, it turned out to be the right answer. But yes, at the time it was, well, I guess it was cool. First of all, DARPA funded it.

1:38:59So you were supposed to go once a week to get a primary dinner. But I just went every week. Anyway, for the students. Yeah. So just for a little bit more context, this class was basically the class right after Feynman and I forget who else proposed the concept of a quantum computer without really knowing what it was, than knowing that there was some sort of... Well, there was plenty of room at the bottom essay, which I think was in the very early... I took it in 82. Okay. That was when he taught it. And he did say there was plenty of room at the bottom. But the way the class was structured, it was like a guest lecture on Tuesday, and then Feynman would stand up on Thursday and explain why that was all wrong.

1:39:45Which was pretty fun. And then we encountered something, which other people have also encountered, something called the Feynman effect. Maybe he was so charismatic or something. He would explain things and he would say, yes, yes, I understand. And then you walk out to thinking, no, no, I didn't see any of this.

1:40:00John Platt:That did not stick. Yeah. So it was kind of fun. But a lot of the guest people, I mean, maybe this sort of showed the chaos. I mean, a lot of people, I think it was Danny Hillis came. There was all these interesting guests. So I would say in the union of all of them, I think showed the sort of mass confusion of what was going on because there was a lot of like, oh, Should we make computers reversible? Because we have to make sure, can they even be reversible? Can the bottom limit of the heat per operation be zero? Or is there some sort of thermodynamic limit? That was a big deal. I don't think that's a big deal now.

1:40:33So you're talking about the Landauer limit, right? Or not what's unknown? Well, at the time, it was always this question about can you have billiard ball computers and can they be reversible and things? And also, what kind of computing? That's why Danny Hillis came. I'm like, you know, I think that was in the era of the connection machine. And the original thinking machines, not Miramirati's, but the original one in the 80s. So he came. What was the state of general computation in 1982? Oh. I mean, at this point. To rounding error, we had zero. I mean, I remember when I got there. I was Carver Mead's sysadmin, and we had a VAX 11750 that maybe did a MIP.

1:41:14One million, ooh, one million operations. And the whole research group shared an 80 megabyte disk drive that was the size of a dishwasher. It was very exciting.

1:41:24John Platt:So what about complexity theory? What was it? Because I know there's a lot of interest in quantum complexity and how it relates to gravity right now. And I wonder, I don't have a mind in my mind about when complexity theory. We could try to look it up. I don't think there's, of course, the whole, there's a whole hierarchy of, you're right, quantum complexity classes. is. I think that was developed after that, because this was in 1982. In fact, I'm not even sure there was a gate. We never talked about the gate model. There was no gate model of quantum. You mean quantum gate? Yeah. Yeah. Yeah. Yeah.

1:41:54I forget. I'm sorry. I think a lot of those. Yeah. I don't think those came out until like the 90s or something. Yeah. I mean, I remember reading Nielsen and Chung, the classic quantum information textbook, which brings up a lot of those points. Now that I think about it, that textbook was written in the late 90s or early 2000s. Yes, that's right. And I think that was the first textbook which put down kind of the general knowledge of the field. But I could be wrong. I guess I think now that when I was young, I took it just for granted that this is the book that everyone... Yeah, I know, but this is a book.

1:42:30Well, you remember in the early 80s, I mean, there was a whole bunch of excitement around neural networks. and it's really interesting because again we really did not know what we were doing no one knew what they were doing there was like an interesting like neural networks then SVMs then neural networks oh no no this is before that this is in the 80s everyone said everyone said this has the capability of revolutionizing computing but what does well I mean there was a tremendous excitement around Hopfield networks in fact NeurIPS came out of because there was a workshop at Snowbird that was nominally private but everyone tried to crash, and so they spun up.

1:43:09Snowbird is a skiing trip with a computation conference attached. That's right. That's why I learned to ski. I didn't know how to ski, and then I kept going. Anyway, that workshop came out of the Santa Barbara workshop in 1985, which came out of some local things at Caltech, which was called Hopfests. So, yeah, people thought, oh, wow, something involved. In fact, ironically, there was like, yeah, yeah, something about associative memory. And if you actually dig down into what transformers are, they are associative memories. So, in fact, there was even a paper called, you know, Hopfield Networks are All You Need.

1:43:41So, yes.

1:43:43John Platt:Yeah, I forgot. So, was that title as a reference to that paper? To the era. So, it's actually, it's like the whole thing has sort of come full circle. And, in fact, I think we did, we have collectively as a field revolutionized computer science. But it was the sort of the hopes and dreams completely outstripped the capabilities because, again, we effectively had zero compute. Yeah. I think there's the history of machine learning is different paradigms as like compute versus memory scaling versus data become available in different levels. That's right. And the fact that I think people don't realize that the reason why neural nervous one is they're the one compute limited thing.

1:44:27Although, again, with now that we have transformers, things are getting memory limited again. But and the fact that they ride on top of Blas. And so the fact that Blas was being optimized by things like GPUs. I mean, I don't actually know if, in fact, I'm pretty sure the brains don't work by matrix multiply. But it was just that the algorithms co-evolved with the hardware. And that's why we're here. Who knows? I mean, if we'd gone down some other path where people really cared about some other compute, who knows what architecture we would have ended up with. I don't know. I mean, didn't a lot of this start with, I think people were hacking PlayStations to train neural networks or something, or to do, I guess, maybe it was even before that.

1:45:10John Platt:There's a funny story by a friend of mine from grad school, Brian Catanzaro, but video where he, Brian, sorry if I get this wrong, but he came to NVIDIA and was having a lot of trouble getting traction. And he basically, they were doubled and tripled down on gaming. And he basically one day had a meeting with Jensen and convinced him, let's, you know, like this is, we have all these people using CUDA for Blast, Blas, basically, and for deep learning in particular, and convinced Jensen. And he said it was like a 15-minute conversation, and they pivoted the whole company the next day or whatever.

1:45:54John Platt:Okay. I've never worked in it. I mean, I have friends, Dave Kirk, who's the first chief scientist, and Bill Daly were both friends of mine from grad school. So, yes, but I don't know the details of it. Now, remember that people like in Hinton's group, even in around 2010, they were using GPUs to do the deep learning. But even in, I would say, 2007, 2008, they weren't that much faster than CPUs. Again, they only started really exceeding. In fact, I think this is not a coincidence in the era of the original ImageNet and some of the speech recognition stuff. So I think that's not a coincidence that they really resurged when GPUs passed CPUs.

1:46:36So that's not a coincidence either.

1:46:39John Platt:I really want to know, how does one get the opportunity to name an asteroid? Oh, well, again, Caltech, there were some wonderful people, Gene and Carolyn Shoemaker, and they were teaching a class in planetary science. I like planetary science. And so, yes, as part of that class, they took us through the sort of asteroid discovery thing. Now, this is in the 80s. So now there's all sorts of amazing, amazing systems. Well, there was for a while something called Linear, which automated the thing. But this was before anything was automated. So, yeah, it was just part of a class. And what you would do is you'd go, the steps, even though we did it out of order in the class, but the steps we used to do, this was 40 years ago, is you would go to Palomar.

1:47:26There would be a fast telescope, which literally now a museum piece is in their visitor center. But at the time, it was a real thing. You would put a piece of film in it. You would take a picture and then you'd wait for a few more minutes and take a same picture of the same point of sky. You'd put in a stereoscope, like back at Caltech. You would see if anything, you would look around. You would see if anything floated because it would move by a tiny bit. So it would pop up.

1:47:49John Platt:So because it's different in different eyes, then you would be able to see it as something that was projected in a different place. That's right. So you would pop out at you literally. And then you would go to a measuring microscope and you would take measurements of known star references and where this floater is. And so if you got accurate enough measurement, then you would send it off to a person. I don't think it's him anymore. His name is Brian Marsden at the Minor Planet Center in Arizona. and then he had a big software system to do, to sort of piece together these called apparitions. And if you found, and if you happen to have done an observation, which was the final apparition, which allowed his software to connect it into one big orbit, then you would get discovery rights and you could name the asteroid.

1:48:33But now it's like amazing. There's this observatory now in Chile called the Vera Rubin Observatory and there's this amazing telescope called the Simone Survey Telescope. essentially automates this process, essentially can just take many frames of the sky. It's an utterly stunning instrument. And so they discovered 11 ,000 asteroids in six weeks. Oh, wow. Yeah. So it's now... How many of them are there? I mean, at least above a... Well, it depends, I guess, on what's the cutoff. They cut off sides, yeah. Yeah, but there are probably millions of asteroids. So there's still a lot of opportunity. It's true, but I don't even know if they bother me.

1:49:09So I don't know. Maybe you don't need, I don't know. But people are doing occultation. Sorry, I'll talk about this forever. There's stuff called occultation. Like one of my asteroids, I was talking, I was sending email to an amateur. If an asteroid happens to pass in front of a star, it dips. Like when they discover exoplanets. But if you're super lucky, it'll dip and then it'll dip again because there's a moon. And so one of my asteroids, this amateur, found a little moon around it. So that was, I think, last year.

1:49:40John Platt:Around the asteroid. Yeah. Interesting. So that's pretty cool. So there's still room for... Your asteroid has a moon too. Yes. In fact, apparently, it's actually, actually, there's a little bit of a story there because it's not really my asteroid that I found two of them. I named one of them after my dad. I waffled. So Carolyn named it after a professor at University of Washington. And I said, oh, but I wanted to name it. And so I whined and she was nice. And so she gave me one of hers. And so then they named that one after my mom. And that's the one that has the moon. Oh, okay. So the asteroid named after my mom.

1:50:11It has a moon. How big is it? It's like the main body, they think, is about four kilometers across. I'm trying to get the units right. And the moon is about one kilometer. So it's actually, it's not, it's a pretty big binary. They're not quite twins, but yeah.

1:50:27John Platt:Did other people in that class discover any? I think there was one other person. It's a little bit of a crapshoot, yeah. And the fact that I found two is quite unusual. Was there a reason why you, was it just pure luck? It was just pure luck. Like, just staying there all day, do it, try it. I don't know if I'm doing it. I know I try to be careful, but yeah. How does one become, get an Oscar? Yeah, an Academy Award. Well, again, maybe I was at the right place. My thesis advisor is named Al Barr. And I was a student intern, actually, in 1986 at a place called Schlumberger, which is a well-discovered place, but they had an AI lab.

1:51:02And so we were all doing sort of computer graphics research. And we were all thinking, you know, at the time, this was revolutionary. I realized it's now considered incredibly boring. Like you could actually use like physics simulators to make computer graphics movies. And at the time it was like, wow, that's really cool. So I said, you know, you could use the theory of elasticity to make floppy things. And so I said, sort of like, here's the theory of elasticity. And I wrote an elastic simulator and I made, you know, fabric and stretchy things and stuff. So they said, oh, wow, that's cool. And so you sort of descendants of that became a lot of the physics simulators that people used in, you know, Pixar and their various movies.

1:51:39So yes, I tell interns sort of half jokingly, well, if you do a really good job as an intern, you can get an Academy Award. So how long was that between the time you did that work and then? 20 years. It was 20 years. Which is actually not atypical, right? Because they want to, when they give you an Academy Award, they want to make sure like, oh yeah, it's sort of well used and everyone uses it and stuff. So yeah. But of course, obviously, like in that 20 years, like, well, everyone does this. It's obvious, but you know. So this was specific work done for a specific movie or something? No, it was like a paper in SIGGRAPH.

1:52:10It was a paper. And then Pixar just became, I think, somewhat important core to a lot of their... Yes. And in fact, some of my friends, in fact, a lot of my friends did a lot of these simulators. So, yeah. I'm curious about, since you have been quantum computing adjacent and worked on quantum computing directly at Google Applied Sciences for a while, I think you're not currently on that. But I'm curious to see what you... I'm still dabbling in quantum error. Oh, you're still dabbling. Yeah. Yeah. Quantum error. Yeah. Yeah. So where do you see the trajectory of quantum computing over the years going?

1:52:40Because this is one of those things which I guess kind of like fusion was sort of had a sense of first like being very exciting and then seeming like it wasn't going anywhere for a long time. And then maybe now we're seeing hints again of it being exciting. And I'm or maybe that's like my sort of quantum adjacent. I think a lot of people have gone through that. I tend to average. I tend to average things out over the day. I guess I'm old enough now. where I sort of average things out over the decades and it's just progress. But I mean, like going from, you know, starting from Feynman just trying to figure out even what this means as a concept all the way up to now where, you know, there's this recent Willow result of quantum error correction, which actually seems genuinely achievable with like the right scaling laws and stuff.

1:53:26I mean, is this like a, do you still think that, or would you say that quantum is 20 years away or even until like some simple but practical algorithm which actually succeeds? Or, I mean, is it, do you see this accelerating or do you think this is still going to be something that's log? Because, I mean, there's a lot of areas where we saw, oh, this is very, you know, this advanced very quickly and it was very unexpected. And I'm wondering if this is a thing that you think will be quick or will not be or this is, I don't know. I think sort of in between, it's not a purely software thing because we need to build systems that are large enough and stable enough to be able to be a quantum computer instead of a quantum apparatus.

1:54:10We're in what they call the NISC era. That was the intermediate scale quantum, which was, I think, coined by John Preskill. It's not a very good term, sorry, but I guess that's the term we have. And there are, you know, the quantum team made this wonderful paper, which I think I'm co-author on, one of many, for something called the quantum echoes algorithm, which could be applicable now, which fundamentally it's in the style of Feynman's proposal, which essentially it's, well, the technical term is you could try to fit a Hamiltonian to observe data like an NMR. What that means is, yeah, you have a physical model that's parametrized and you use the quantum computer to kind of adjust the parameters and try to figure out it inside of a loop so you could, for example, decode NMR parameters.

1:55:02So that is a practical thing that's used now. The question is, is it going to be, it will be big enough to make breakthroughs? That's still TBD. The quantum team at Google has been very executing amazingly well against a roadmap that Hartmut Nevin laid out a few years ago, and they're just continuing to march down this thing where they're scaling up, and when they hit their last milestone, they should be able to have a quantum computer that does amazing things and they're continuing to march it along. So yeah, it's, I think, on the scale of a few to several years, I haven't sort of kept up on exactly what date they're saying, so you should ask Hartman exactly when that's going to happen.

1:55:46But yeah, so they're marching along. So the interesting question is, will the superconducting computing be the winner or will one of the other sort of alternate technologies, and that's still TBD. I still think superconducting is a very promising thing because it is very scalable. So, yeah, no, I don't think it's 30 years away. I don't think it's tomorrow. I don't think there's going to be, I don't think, unless, well, even if there's a sudden hardware breakthrough, these are very finicky things. So it's like someone might have a brilliant idea, but it'll still be a while before. Because fundamentally, at least in the NISC era or for a while, these are fundamentally analog computers.

1:56:21And so they tend to be very, very, very finicky. so I wouldn't expect all of a sudden, you know, some phase change happens. No one has figured out how to scale up qubits in a way that interact in just kind of arbitrary size or still in the order of 100, 200 qubits, I think, or? You mean in terms of, okay, so in terms of, well, it's a little bit like, yeah, yeah, exactly. So for superconducting qubits, yeah, people, I mean, although people are working on it, the ratio of the number of physical qubits histological qubits is still relatively large for the error rates that you need. And so maybe there'll be a breakthrough there, I don't know.

1:56:58Or there are other things that might have, you don't need that ratio to be so high because a lot of it has to do with the 2D connectivity of the chips that you lay out your qubits on a 2D chip. And for things like neutral atoms, they in theory can connect anything to anything else. But of course in practice we don't really know and we don't actually know what the limitations of neutral, at least I don't know the limitations of what neutral has. Maybe the super experts know. Superconducting cubits in particular have a sort of tension where you want to have nice clean resonators, which you do get a clean resonator by decoupling from the environment and then you get good interactions by coupling resonators, which involves covering to the environment.

1:57:43So it's only seems like there's a bit of a tension there. But there's like the whole environment. And then there's like little tiny people. that you want to go through to your neighbors. This is not like a fundamental uncertainty relationship or something. That's right. It's like this is a technological limitation that difficulty or something. Yeah. The hardware team and Google Quantum is very, very skilled. They're very, very skilled. So, yeah, they're really good at making these designs and making these things actually work. So I find them impressive. Yeah. It'd be exciting to see that advance.

1:58:15Yeah. Yeah. Yeah. I guess I'll just have to hold a breath and wait. Just wait. Yeah. I guess I'm just very patient. So I start working. 20 years ahead of. That's what Dave Bacon, who runs the software team in Google Quantum, always, he teases me like, John, like, oh no, I can't work in quantum computing. This was like 10 years ago because it's going to be, you're always 20 years ahead of time. And so I'm going to have to wait for 20 years. Like, okay, well, you know, 10 years have gone by. Halfway there. Halfway there. I don't, I don't take that as an absolute. As a hard role. Yeah, but I also started working on fusion 10 years ago.

1:58:51We'll see. Maybe you're actually causal. Like you start working on something and the reality just, you know, catches up. Catches up. I guess so. Maybe. Who knows? I don't know. I started working. I loved convolutional nets in the early 90s. And Jan LeCun claimed I coined the term convolutional net. And as far as I can tell, that may be true. Because everyone called it le net because it was a very specific thing and they all worked for Jan. I said, well, I don't work for Jan. I don't want to call it le net. So I called it a convolutional net. So I don't know. That was a more generic term. That's a good term.

1:59:21Yeah. Before you go, is there anything you want the audience to take away? Any messages you want to deliver? I think AIR is an example of it, but I'm really amazingly excited about the potential of AI for science. I think it's going to be an amazing power tool for scientists. I think scientists won't be replaced. I think, in fact, I'm hoping that they'll spend all their time on, again, the creative stuff, on the rigorous stuff, on the philosophy stuff. And so I think it's going to be way cool.

1:59:51John Platt:I hope so too. Yeah, my intuition is that a lot of, I've heard a lot of people say that. I hope they're right. And I hope it's not just sort of coping with a reality that's uncomfortable. The other question we almost forgot to ask, if you could remove a bottleneck in your industry, which whatever you, however you want to define that, by fiat. Like magic. Yeah, by magic. What would that be? If I could get a magic wish, I would say, someone please make the everything lab that you could like send JSON blob to and it will do any experiment at all. Okay, automated. But it'd have to be anything. So essentially, so I guess we have to solve the sort of AI complete robotics problem, I guess.

2:00:38But if we did, then that would be stunning because right now things like Aira, it's all computational. So someone has to gather the data. So yeah, if we could just break that, oh, oh, that would be so amazing. That would be so utterly amazing.

2:00:51John Platt:Yeah, great. So I really appreciate you taking the time to see us. And I think you like kind of flew in and adjusted your schedule a little bit. Yeah, I was sort of flying over San Francisco to get home. And so I said I landed in San Francisco. So you made a big effort to be here. We really appreciate that. it was really fun to talk to you. It's been a blast. Okay, cool. Thank you for having me. You're welcome. Thank you.