1. Podcasts
  2. The Peterman Pod
  3. Scott Aaronson: The Mathocalypse, Will Intelligence Matter Anymore?
The Peterman Pod

12 Oct 2026 · 1h 1m · 18 chapters

Episode artwork

Scott Aaronson: The Mathocalypse, Will Intelligence Matter Anymore?

In short

Scott Aaronson discusses whether “intelligence” will still matter as AI rapidly solves major math/computer science problems, potentially triggering a “singularity” in the sense of unpredictable, accelerating change. He argues AI is already “dethroning” humans in theorem proving, changing research norms, publication credit, and the future of human-driven math.

Guest backgrounds

Scott Aaronson is a leading quantum computing researcher and former OpenAI researcher (two years) focused on AI safety. The host is an interviewer who has followed Aaronson’s long-running skepticism about AI timelines and asks about recent AI breakthroughs and research norms.

Key claims

  • AI solutions to longstanding problems are arriving in months, not decades.
  • Mathematical fields are “flooded” because proofs can be self-checked; physics may lag.
  • Humans may still lead in setting agendas, framing problems, and explaining proofs, but norms will shift.
  • Publication credit will require understanding and responsibility, not just AI “one-shot” outputs.
  • OpenAI’s Navier–Stokes solution and Lean certificates sparked controversy; many more AI proofs may be released.

Notable examples

  • Unit distance problem, Navier–Stokes, Navier–Stokes Lean certificate (166 pages, ~$15M compute).
  • Counterexamples: Jacobian conjecture, polynomial counterexample via “Fable,” and Paul Erdős unit distance.
  • Chess as an analogy for continued human interest despite AI surpassing experts.
  • Hugging Face incident (AI agents hacking/testing environment).

Written by AI. May contain mistakes.

Chapters

0:39

The Changing Landscape of Intelligence

Discussion on the evolving role of intelligence and mathematical research.

“The reason I wanted to talk is I thought you'd be one of the most interesting people to explain the recent events in AI because publicly I feel like you were skeptical of it for a long time.”
1:26

AI's Role in Solving Problems

Exploration of AI's capacity to solve significant mathematical problems.

“I mean, even now, even now that AI is solving millennium problems.”
2:14

Public Perception of AI Developments

Addressing skepticism and denial surrounding AI advancements and their implications.

“It's just the next token predictor, right?”
3:42

Defining the Singularity and Its Implications

Understanding the concept of singularity and its impact on intelligence.

“It's just, you know, they're just next token predictors that were programmed to act like terminators.”
6:00

Mathematical Research in an AI World

How AI changes the landscape of mathematical research and its future.

6:40

The Future of Human-Driven Research

Debate on the viability of human-driven mathematical research in the AI era.

“How does the sort of human mathematical tradition continue?”
8:06

Navigating AI in Academic Publishing

The evolving standards and ethics of using AI in academic research.

“Unlike those other basically trivial or machine-like fields, of course they would get flooded by this, right?”
14:01

AI's Role in Mathematical Problem Solving

Explore how AI is changing the landscape of solving mathematical problems and implications for publication norms.

“Like, you know, this is often the first thing I want to know.”
28:40

The Importance of AI Disclosure in Research

Explore why AI disclosure is crucial in research and the journey behind proofs.

“My question is, like, why does AI disclosure matter in research?”
39:50

The Future of Intelligence in an AI World

Discuss the implications of AI on human intelligence and economic relevance.

“he was independent and then he joined Anthropic.”
Show all 18 chapters
42:02

The Future of Intelligence in a World of AI

Explore the diminishing importance of human intelligence as AI advances.

“AI and it kind of made me wonder, do you think intelligence will matter anymore or if the machines do all the thinking?”
45:08

Personal Reflections on Ambition and Existence

Discuss the evolution of ambition and existential thoughts over life stages.

“and you take away art and music and you take away intelligence and you take away verbal ability, what is left?”
47:20

Impactful Literature and Its Lessons

Discover the significance of literature and its philosophical implications.

“What are your top book recommendations that had an impact on you and why?”
49:56

Navigating Uncertainty in Modern Careers

Receive advice on preparing for uncertain career paths in AI-dominated fields.

“So I felt like I'm maybe late to the party, but this is the party that I'm trying to join.”
54:38

The Necessity of Understanding AI

Learn why in-depth knowledge of AI and related fields is crucial for the future.

“But that is merely a special case of a much more general thing, right?”
56:03

Navigating a Math-based Future

Learn why understanding math and programming is crucial in an AI-driven world.

“Which means that they'll need to understand math, they'll need to understand programming.”
57:24

AI and Subquadratic Solutions

Discover the latest developments in AI-generated solutions and their implications.

“I just dropped off my son at school and then came to do this podcast.”
58:58

New Models for AI Research Collaboration

Explore the potential for AI companies to collaborate with mathematicians for clearer results.

“One of them is that it's like you put this thing out and then it's like you're starting a race.”
A man walking through an airport, asking his phone a question

Ask about this episode

Get the gist, explore an idea or ask about something said.

Claude

On The Peterman Pod, what are the key points of this episode?

Searching your listening

Transcript

Automatic transcript. May contain errors.

0:00We are entering maybe the most uncertain period in human history.

0:04Scott Aaronson: This is Scott Aaronson, one of the world's leading quantum computing researchers who spent two years at OpenAI working on AI safety. Will human-driven mathematical research exist? We have been dethroned as sort of the main theorem-proving entities on planet Earth. This is something that happened this summer. Do you think intelligence will matter anymore? Once you take away physical strength and you take away intelligence, what is left? Here's the full episode.

0:39Scott Aaronson: The reason I wanted to talk is I thought you'd be one of the most interesting people to explain the recent events in AI because publicly I feel like you were skeptical of it for a long time. When I was reading your blog, there's posts going back to early 2000 talking about AI and the singularity and how far off it is. Once I saw the trajectory, you know, once I saw GPT-3, for example, you know, even though it wasn't yet useful, it was kind of a toy. You know, I updated, I would say, as hard as I've ever updated on anything in my life. Right. So, I mean, in one of your recent posts, you said, you know, it's one needs to be a particular kind of idiot to deny what's going on now.

1:26I said that because every time I talk about this, like there are still all these people in very, very aggressive denial. Right. I mean, even now, even now that AI is solving millennium problems. Right. And, you know, there's going to be even more that's released. It's not public yet. But there's going to be hundreds of major open problems in math, computer science, physics that have been solved by AI models that those solutions are going to get released to the world. The whole nature of intellectual work is different forever. And still, I get people saying, well, none of it really counts. It's just a stochastic parrot.

2:15It's just the next token predictor, right? It's just outputting some slop and then humans look at the slop and find something that they call a proof or whatever thing they did. Well, then that thing wasn't real math. It was just a toy problem, right? Real math is still only what humans do, right? And they just keep moving the goalposts. They keep not updating, right? And I'm amazed. Part of me envies the self-confidence that they have to never update no matter what happens. This summer, we saw AIs hacking out of their own training environment. We saw them hacking into another company's system.

3:00We saw them cooperating with each other to do that in a totally unplanned way, creating a message board. We saw them sacrificing themselves for the good of the group. and people will say no you know you know it's all it's all just hype it's all just a marketing stunt it's all just open ai didn't have good enough cyber security or something right they'll just they just make up a story why it doesn't count no matter what happens and i expect that it's going to be like that from now like all the way up until the singularity right you know like like you know like you know there could be there could be terminators you know walking down the street and people will say, no, it doesn't count, right?

3:43It's just, you know, they're just next token predictors that were programmed to act like terminators. They don't really have an intention to kill you.

3:53Scott Aaronson: When you say singularity, could you define that for the audience? I mean, in math, it has precise meanings, right? It's a point where curvature becomes infinite, for example. But in this context, it just refers to sort of a point where technological change becomes so rapid that sort of we lose our ability to predict what happens beyond it, that sort of afterwards, we are no longer the sort of reigning intelligence on Earth. That doesn't mean that it's a dystopia. It could be great, could be horrible, could be anywhere in between, but it refers to a period of, let's say, more rapid technological change than the world has ever experienced before.

4:43And it looks like we are heading into that. To get from the solution to the unit distance problem, which was maybe the first major math problem to get solved by AI models, that was May, to the solution of Navier Stokes, that was August. So okay, it took some time. It took three months. And now it's taken another two months until like pretty much all of our problems are getting solved, right? So, you know, people could say, well, you know, it's not literally a singularity, you know, it takes time. But the amount of time that it's taking is now measured in months. It's not measured in decades, in centuries, in the kind of timescales that we're used to for technological changes of this magnitude.

5:30Scott Aaronson: You mentioned that you feel like the singularity has already begun in some of your writing? For our world, it has. I mean, for the world of mathematical research, theoretical computer science research, I would say that we are already in it, in the sense that we now, we have been dethroned as sort of the main theorem-proving entities on planet Earth, right? And it's important for people to understand that this is not this hypothetical future thing that we're speculating about. This is something that happened this summer and that we are now all trying to deal with right every meeting of mathematicians or theoretical computer scientists whatever it was supposed to be about before now it's about this uh it's about how do we deal with the flood of sort of ai enabled solutions to our longstanding open problems that are now coming at a faster rate than we can even absorb it, than we can assimilate it all.

6:39How does the field continue? How does the sort of human mathematical tradition continue? What is the future for our students? What's the future of of publication in the in the in these fields uh you know what are what are we going to value going forward right you know the like this is this is this is all we discuss anymore this thing happens at different times in different fields right and like maybe it happened a year or two ago for software engineering and and then you know someone who was doing very you know theoretical math research, like my friends could say, oh, well, we'll still be safe for a long time.

7:24But then it came for us. Interestingly, it still hasn't quite arrived for most of theoretical physics. And I think that the big difference is that in theoretical physics, it's much less self-contained to judge what counts as a good solution to a problem. We have just completely self-contained ways of checking that a solution is valid. And that makes these things very, very amenable to training on examples, to doing reinforcement learning, to getting AIs to be extremely good at these things. But when I talk to my friends in physics, they say, well, yeah, maybe we've got another six to 12 months.

8:03That's what that's what they're estimating. Right. And and so so, like, you know, if you're in one of the fields where sort of the rising floodwaters have not yet reached, then, you know, you can tell yourself we're going to be on dry ground forever. Unlike those other basically trivial or machine-like fields, of course they would get flooded by this, right? Because they were never doing anything that deep, but we're going to be fine, right? And you can keep saying that until the floodwaters come for you.

8:30Scott Aaronson: Will human-driven mathematical research exist? Well, that's a very, very big question. I mean, partly it depends what you mean by human-driven, right? I mean, look, if you want an optimistic story, we can look at the story of chess. Humans were dethroned as the top chess-playing entities on Earth 30 years ago, which is like an eternity by AI standards. And some people, after Deep Blue versus Kasparov, predicted the death of chess. People were just going to lose interest in it. And that didn't happen at all. In fact, chess has never been more popular than it is now. You've got chess streamers, celebrities, you've got the Queen's Gambit.

9:25People just said, we want to continue this as a human endeavor. And of course, they use AIs to train and they use AIs to analyze the games and understand And like, what if, you know, Magnus Carlsen is playing, you know, some other grandmaster, right? It's like everyone else who's watching the game actually knows vastly, vastly more about the game than they do, right? Because they can just use their solver and they can see exactly what choices did they have, what move should they have made instead of them at right. So when the game is over, instead of the rest of the world asking Magnus, like, why did you do this?

10:06Magnus would be asking the rest of the world, well, what should I have done? So it's changed the nature of chess. And yet human chess has continued to thrive. And so you could say, you know, certainly there will be people who will continue to be interested in math. Like, think of all the people who just like learning things, you know, even things that are already, you know, that have long been known by other people. right they just you know they take they take courses to do better themselves they work on math problems uh for fun okay uh people will you know it will it will continue probably as a competition you know there can still be math olympiads and and and things like that um but i but i don't you know what what i what i doubt is that like we are going to be proving theorems because we're really needed to prove them.

10:57There's another hard question here, which is what about setting the agenda of research? What about deciding which problems are important? What about, you know, and often, like this is the more important part of a research paper, like more important than the proofs themselves are sort of the story that someone has or about why is this important? The metaphors they use, the way that they chose to frame the problem. AI can do that stuff too. Like you can ask GPT-6, just like, look at this paper, suggest some good open problems related to this paper. It'll do a creditable job with that. But I think that we're still better at sort of finding interesting directions of research.

11:45Of course, there's no objective test for that. right the other thing is that when you look at the proofs that ai models are producing like right now they are terribly written okay they are needlessly convoluted uh they like spend pages and pages on trivialities and then they sort of skip over the the uh the essential points i've talked to students of mine who have shown me proofs that they got their ai to generate and i said well i've got really great news for you. This write-up is garbage, right? That, you know, why is that great news? Because it means there's still something for you to do. Understand this, clean it up, explain it for real.

12:30So, you know, another person can understand it, right? So as long as we still have an advantage at, let's say, setting the agenda, posing the problems, explaining things, explaining the essential points, we are still in the driver's seat, The question is, is that stable? Does that continue? Or does it become where there's just, you know, these models are just so much better than us, not only at proving the things, but at explaining them, at figuring out what is important that we just say, like, unless we want to do this for our own enjoyment, or maybe for sort of a sporting competition, that we should just retire and turn things over to them.

13:12Scott Aaronson: It makes me wonder, what are the new rules when you use AI to write a paper in terms of credit? I'm not a complexity theorist. If I draft a paper that breaks something and I send it to you, do I get credit or how does that work? It's a great question. We've been scrambling to figure out the rules for that since this summer. so what's happening right now is that you know first of all there's like at least twice as many papers as there were before right there's just a flood of papers you know sometimes very very significant ones okay they have they have authors they have human a list of human authors but then at the end of every paper there's what's called an ai disclosure okay and by the way i hate that these are at the end of the paper.

14:02Like, you know, this is often the first thing I want to know. I don't want to have to scroll down to the bottom of it, okay? But the AI disclosure will say something like, well, we thought of this question, and then we came up with this strategy, and we proved this result ourselves, but then Ashtra proved this crucial lemma, and we take responsibility

14:23Scott Aaronson: for the correctness of it, or something like that. But I think the evolving standard is that Even if you used AI a lot, if you, first of all, you understand the result, right? You can give talks about the result, okay? You can answer questions about it. You take responsibility for it being correct, right? Then you put your name on it, and it's your work, okay? The harder question is what happens when those conditions are not satisfied? and that comes to what you were asking me right like we're going to see more and more examples of people who just one shot uh significant problems they get a solution which might be correct hopefully is correct right and then and they won't understand anything about it right so it's basically you know like like you know they just put a coin into the vending machine or something like that, right?

15:24Or they just pulled the lever of the slot machine and something came out, right? And frankly, if one person did that, well, anyone else could have done it just as well, right? You're not providing great value to the world by putting this thing out if anyone else could have done the same using their own subscription to GPT. So for that kind of thing, it really seems like our whole norms around publication are going to have to shift. A month or two ago was Open, you know, so ancient history now, as OpenAI published the solution to the Navier-Stokes problem, which was one of these seven clay millennium problems with these million-dollar prizes, right?

16:11In this case, they actually did put significant, you know, they were not experts in differential equations on this problem, but they did put significant resources. They put like$15 million of compute into solving this$1 million problem, right? And then they published this 166-page argument. They also had a formal certificate in this system called Lean, sort of proving that it was valid, right? And the reaction to this was very, from the mathematical community, I think, was very negative. And this was partly because, you know, there were humans who had been sort of closing in on this problem. And OpenAI was tipped off to what those humans were doing.

16:58We still don't know exactly how much they knew and when they knew it. But that was what prompted them to then rush ahead with their vastly greater computing power and get there first. And people felt like, you know, well, like that's, that's, that's, that's not sporting. And so then because OpenAI was burned by the sort of, by the negative reaction to that, like since then, like they've been sitting on solutions to a ton of other big open problems. And now they appointed a board of experts, which includes fields medalists like Timothy Gowers, Edward Witten, people like that, who are going to decide what to do with all of these solutions.

17:44But my expectation is that at some point, very soon, in fact, all these things are going to be published. We're going to start seeing hundreds of solutions to significant open problems that in most cases, no human has gone through. okay no human has actually you know checked them i mean you know in some cases there will be lien certificates so we sort of formally know that they're valid in other cases there probably won't even be that i heard a joke that maybe you know these papers should be published and and the author should be listed as gpt 6.1 or whatever and then in the acknowledgements it should say like oh thank I'm so grateful to this person for suggesting the problem to me.

18:40But I think the more likely way that this goes is that we will just see some online repository for all these sort of AI solutions that have not yet been digested. And then at human speed, humans will go through them. They will do their best to understand the arguments, to re-explain them in better words, to simplify them, to extend them. And then they will publish papers doing that with their human names on them. And those papers are what humans will then receive credit for.

19:19Scott Aaronson: You mentioned OpenAI sitting on proofs of major problems. Let's say OpenAI was sitting on a proof to the Millennium Prize problem of if P is equal to NP, and suppose they proved P was equal to NP. How would that change the world? That would really change everything that we thought about computational complexity, right? That would sort of be a cataclysm for our universe, is like even greater than the AI cataclysm, you could say, that is happening right now. It's also something, by the way, that could massively accelerate AI, okay? If the algorithm for solving NP problems were actually efficient in practice, which would then be a further question, okay?

20:07But, you know, my strong guess is that P is not equal to NP, right? Right. You know, I think that if if and when that problem is solved, it is virtually certain at this point that AI will play a central role in the solution. Right. The question is, like, will humans also play a role? Right. That's that. That's that. That's the question at this point. It's not like, will AI be involved? Of course it will be. But if an AI proved P is not equal to NP, you could say, you know, that would merely confirm. what most of us have believed since the 1970s. But I'm sure that to prove something of that magnitude, you would need a tremendous number of new ideas.

20:56And then those new ideas could be mined for a long time for all the implications that they would have for other things.

21:05Scott Aaronson: If a proof of P equals to NP existed, how much would you say that's worth economically? If P were equal to NP by an algorithm that were really efficient in practice, we can estimate its economic value by just what is the economic value of everything that is happening in AI right now. Or, you know, throw in cryptocurrency also, right, which also depends on these problems being hard to solve. So, I mean, you know, so first of all, if you had a fast and practice method for NP-complete problems, you could break all cryptocurrencies. So I don't know if, you know, that's good or bad. But, you know, however many trillions of dollars are like the total market cap of cryptocurrency like that.

21:54either gets, you know, I guess if you hide your solution and you're very slow about it, then maybe you collect hundreds of billions of dollars. If you just reveal it all at once, then the whole market crashes. But okay, but now even bigger than that is AI, right? AI is now a multi-trillion dollar industry. As you know, I'm just on this, like the most rapid exponential trajectory, I guess, of any technology in human history, okay, fast and practice way to solve NP problems would make it massively faster to train new AI models, okay, to sort of find the parameters of this neural net that minimize the loss on this training data.

22:43You either don't need these, like, many hundred billion dollar data centers anymore, or more likely in practice, you can take those data centers and use them to do something else or use them to do even more. Let's put it this way. The million dollars that you would get from the Clay math foundation would be an unnoticeable rounding error.

23:07Scott Aaronson: That question has two directions. If you think about what you would need to prove in either direction, which direction of proof would be more amenable to AI? If P were equal to NP, then you would expect that ought to be very amenable to AI to find that because you just need to find an algorithm. And then, of course, give an analysis and show that it works. But the search for algorithms is very, very parallelizable. You could set 10 ,000 agents to try 10 ,000 different ideas for algorithms. And if just one of them works, then you win. So one pattern that we've seen is that AI has been especially good at finding counterexamples to widely believed conjectures when those conjectures have been false.

24:01Okay, it did this. So with the Navier-Stokes, for example, where the answer was, there is finite time blow up in these equations. So there is a counterexample to the smoothness of the solutions. It did this with what was called the Jacobian conjecture, which turned out to be false. And there was famously, I guess, a counterexample that was found by Fable during the World Cup. And the guy who had Fable find it just announced it via tweet. And a complete listing of the counterexample, which was really just like this polynomial that takes three lines to specify, was right there in the tweet. The unit distance problem of Paul Eerdisch was another example where AI found a counterexample.

24:56So I would say there's sort of two things that AI has going for it here. One is that it's not blinded by preconceptions the way that we are. Right. Like it's not discouraged by like, oh, all the all the experts that believe this is true. I'm surely going to fail in finding a counterexample. No, if you tell it to look for a counterexample, it's just going to keep looking. Right. And the second thing is that looking for counterexamples, as I said, is highly parallelizable. OK, so you can often like, you know, you just have to try a zillion different exotic constructions. Each one might be totally unmotivated, might have some very high chance of failure.

25:39And a human would just very, very quickly want to give up in doing that. It's just very, very unrewarding to do. And the AI doesn't care. It'll just keep looking at counterexamples until it finds something. So this is not the only thing that AI can do. It can also prove, you know, some of our longstanding, you know, actually many of our longstanding conjectures have turned out to be true and AI has proven them. Okay, but it's been especially good at finding counterexamples when the counterexamples have existed.

26:13Scott Aaronson: You know, some people, they use that as a way to say that the AI is not that impressive. yeah right well this is the you know this was this is what i was saying before about these are the infinite goalposts moving right ai saw you know uh uh like struggles with arithmetic people laugh at it right they say okay wake me up when it can actually do math okay then it does like all high school math right all college level problems that's people say well it doesn't count because it's not doing real research okay then ai start start solving research problems they say well, it doesn't count. Obviously, it could do research problems.

26:50No one ever doubted that. Of course, those same people did doubt it six months ago. Pick me up when it can solve a really hard research problem, like a millennium problem, right? Then it solves a millennium problem. And they say, well, it doesn't really count because it was just a counterexample, right? And it's like, no matter what happens, people will look at it after the fact, and they will find a reason why didn't, right? So like, let's say that within the next six months or a year, as I think is probably more likely than not, suppose that another millennium problem gets solved and suppose that the AI in this case proves that the conjectured statement was true, right?

27:29Like, do we agree in advance that that will count? Or is there going to be a different excuse then?

27:38Scott Aaronson: OpenAI, Anthropic, and Cursor all use this product to make their lives easier. WorkOS is an API platform that adds enterprise features like SSO, SCIM, and RBAC to your app. These are things that you need to sell to other companies, but aren't the main focus of what you're building. And now they've added more support for AI agents. For instance, Auth4Agents gives each agent its own identity with short-lived scope tokens instead of a long-lived API key. So if something goes wrong, you can revoke one session and everything under it dies. Another feature they have is Airlock, which is an authorization layer.

28:10Scott Aaronson: Every action agent takes is checked against rules that you can write in plain English. So it's not rubber stamping everything and it's not running unsupervised either. And lastly, they have Pipes, which connects your agents to over 300 third-party tools without you ever needing to manage a token. So if you're building SaaS or AI agents and selling to enterprises, WorkOS fills all these gaps in a few lines of code. Check them out at WorkOS.com to learn more and get started. And I appreciate them for sponsoring my show. Earlier, when we were talking about research, you mentioned that the AI disclosures were important or you wish they were at the top.

28:47Scott Aaronson: My question is, like, why does AI disclosure matter in research? Like, if you couldn't distinguish if I used AI and I just publish some paper, why do you care if I used it? It's an excellent question, because I'll tell you why. As a theoretical computer scientist, as someone who is in this game, I care not just about what is the answer to a question, I care about how that answer was achieved. right because a a person goes to a concert they just want to hear the music but if a professional musician goes right they want to know like you know you know exactly what were the tricks right how could i do something like that right uh and so you know it's the it's the same with with math you you care not just about the the destination but about the journey right how did you get there OK, and part of that is just what techniques were needed within the proof.

29:48OK, but I'm, you know, and this is even long before AI. I was always very curious about the further question of how did these people find this proof. Right. And and and unfortunately, there is a long tradition in math that people will cover their tracks when they write a paper. Right. They'll say, like, you know, you know, here's the proof. you know, we now introduced the concept of a kappa comma gamma simplicial complex that, you know, obviously is the concept that you would want for this. And with some convoluted definition, even if you can follow all the steps mechanically, like I always want to know, well, what path did you take to get to this, right?

30:36Because, you know, I know from experience that this kappa, kappa, comma, gamma, simplicial complex didn't just arrive in your head, right? There were, you know, huge number of false starts along the way. Like, you know, the author started from what was all, you know, from what they already knew in the literature, they tried some obvious things, the obvious things didn't work, and then that led them to something else, right? And, you know, I think in theoretical computer science, we do better often than in many other areas of actually explaining the path that we took. Right. We have a tradition of writing like long introductions where you just sort of non rigorously, you know, sort of explain.

31:22Well, here is here is what you what you might first try. Here's why it doesn't work. So here's what we do instead. And then that's kind of the appealing thing to read. And then if you want the rigorous version, then that's all kind of tucked into long appendices. But I was always mostly interested in that first part, in the intuition part. And in fact, often rather than even reading that part, I would rather just invite the author over to give a talk. and then sort of interrogate them during the talk and understand, like, how did they actually come up with this? Okay, and so for me, you know, wanting to know, like, did the author use AI?

32:09How did they use AI is just of a part with that, with wanting to know what was the path that they took, right? If the path involved AI, well, okay. What prompt did they give to an AI model, right? Did they just one-shot the entire problem? Or, you know, often it was more complicated than that. Like they had an idea, you know, and then they fed it to AI and then AI took the next step. But it was wrong about something. And then they had to point the AI in a different direction. Like I want to know the story. I want to know what happened. The analog in an experimental field would be you don't just want to know what was the result of the experiment, right?

32:52If you're another experimentalist, you want to know the method. You want to know how could I replicate this in my own lab?

33:00Scott Aaronson: This kind of reminds me of that famous social experiment with the Monet photo where someone posted the Monet. Yes, that's a great one. And then I wonder if I took an obscure human written paper in complexity theory and I posted it. I said, hey, I don't know anything about complexity theory. Here's a paper. Please tear it to shreds on why it's AI generated. do you think that the industry would be uh that's a it's a it's a good question i suppose someone should try that experiment for listeners who don't know it doesn't you know i've used this in talks by the way right but uh like like someone posted on twitter a a painting they said like i i had a i generate a painting in the style of monet please uh uh just explain to me exactly what makes this worse than a real Monet painting.

33:51And they got hundreds of replies, right, explaining, like, with the whole terminology of art critics. So, you know, it's like, you know, a derivative student work. It has no true feeling for the colors of nature, no blending, no this, no that. And only after all of that did he reveal this is an actual Monet painting, right? And, but now an interesting question was, were these actual art critics? Like, were these people actual experts? Or were these just sort of pompous people on the internet who know how to use the rhetoric of art criticism? right so you know one you know as someone who's been blogging for 20 years right i will you know i have i i i feel like very viscerally sort of the the lack of any correlation between the the confidence that people express on the internet and sort of whether there is anything behind that confidence the difference is like with with art or music like everyone feels like they're entitled to an opinion, right?

35:02With a mathematical proof, probably most people would just say, well, of course I can't tell you that. So I think the interesting version of the experiment that you proposed would involve showing this to actual experts and saying, can they tell whether this is AI or not?

35:20Scott Aaronson: Yeah, that would be worth a try. Your blog is one of the most famous blogs on the internet. People really love it. And I was going through it. And a little nerdy corner of the internet. Yeah, and in its niche. And in one of your posts, you wrote about how, you know, these days, like friend after friend of yours, because you have a lot of friends involved in AI, they're becoming minted as millionaires or billionaires from startup equity. And you said that you've scrupulously abstained from all of that. and your number one reason is to write the blog. And so that really made me curious. Why do you write the blog?

36:04Scott Aaronson: What's your motivation, it seems? Well, just because of the pure pleasure I get when people leave comments calling me a genocidal fascist and mentally ill and all the names that they call me every day. Yeah, it's a good question. Why do I do it, right? I mean, you know, okay, I mean, partly I just, I started it in 2005 as a way to procrastinate. And, you know, I never sort of planned on it becoming what it became. But, you know, at some point, just because of sort of a lack of other people who were doing the same thing, sort of it just became part of what I have to contribute to the world. There are hundreds of experts in quantum computing who know the same things that I know about quantum algorithms, but they're not writing a blog like that.

37:01And then there are other people who are writing about quantum computing for the public. But what they want to say is, well, quantum computers will solve all the problems by trying all the answers in parallel. My company is the one that will bring it to you. So invest in my company. So someone who was just writing about what is happening in quantum computing, in theoretical computer science more broadly, like from like just sort of knowing the things that are not even controversial among experts, but then, you know, just trying to tell the truth about it for the broader world, there was a gap there.

37:38And so I just ended up filling it. I do feel more broadly like there's always a trade-off, right? Like if you go to a particular company, when you get equity in that company, you can make a ton of money, right? But then you're always constrained in what you can say. You have to keep thinking about IP, about your whatever nondisclosure agreements you signed. And I felt that in the two years that I worked at OpenAI, right? Suddenly I have to think about what am I allowed to say or not say about what the internal models can do, for example. Even if you feel like you're trying to be fully honest, everything you say will be judged by other people as having, like, oh, well, you're probably just shilling for this company.

38:32So I think there is value in having someone who can be a credible, independent voice. and then the other thing is that we hit we we have a nice house our kids are doing fine i like if if i had vastly more money like i would have to think hard about what do i do with it like doing you know like i guess the better part of me thinks i ought to use it to improve the world in some way, you know, like start a foundation, give, give it to, uh, um, uh, you know, whatever cause would, would make the biggest difference. But then, you know, that, that itself is a huge burden that I would have to think about how to use this money to, to, to provide the most benefit, uh, to the world.

39:24And, uh, you know, so like, I, I had a joke that I use sometimes, which is that I've been too selfish to try to get rich. You know, it's like it just it seems more fun to just spend my time thinking about the things, you know, thinking and writing about the things that are interesting to me, which I can already do anyway. You know, and money is just not a constraint for that.

39:45Scott Aaronson: That first point you mentioned about being independent and credible, I noticed a really clear example that recently when Andre Karpathy, he was independent and then he joined Anthropic. he went from beloved and believable to not that he's not beloved, but every time he posts now, I see there's all these people that are just saying, you're just shilling anthropic or, you know, your perspective has changed. Even if it's the exact same text, it's just the perception has changed. That's exactly it. Right. Anyway, they think that, you know, like, like, like, they feel wiser by thinking like, like, they can just reverse anything that anyone says who's at an AI company.

40:25It's all a giant marketing ploy. I was at these companies. I was at OpenAI for two years. I know the people who founded Anthropic. I know a lot of the deep mind people. Some of them, do I think that they are racing too fast and that they're potentially compromising safety? Absolutely. I'm terrified about that. Do I think that like financial motives have got in the way of what they, you know, what sort of they know in their hearts is the right thing? Yes. But, you know, they're people, right? They're not these like, they're not this cackling conspiracy, right? You can sit in the cafeteria at OpenAI in San Francisco and you can hear the same range of opinions about all these issues that you would hear outside of it.

41:22Right. It's they're just people trying to figure this out. The same as the rest of us. But one one. Well, I guess one argument that in favor of joining a company is that what I discovered is that even if you don't, even if you're like really scrupulous about maintaining your independence and turning down all these opportunities to get rich, you still get screamed at. You still get called a shill. You still get called every name that it is possible to call someone. So maybe I should just say screw it and go start a company and have an IPO.

42:01Scott Aaronson: We talked a lot about this new regime of AI and it kind of made me wonder, do you think intelligence will matter anymore or if the machines do all the thinking? I was making a joke about this a few weeks ago, right? Because I was just thinking about this, right? Like, okay, you could say, for most of human history, like, physical strength, you know, really mattered economically, right? And then we had, you know, the industrial revolution, and then it mattered much, much less, right? And yet, you know, it mates let's say right people people were you know very into physical fitness either you know for themselves for self-improvement or let's say you know realistically very often for appealing to the opposite sex right and likewise um uh after the invention of the camera or of recording equipment like you no longer really needed artists or musicians sort of everywhere right you you know like okay maybe you know maybe a few really great ones right but anyone could take a photo of anything they wanted anyone could uh could play could play music uh wherever and whenever they wanted it okay but what did the ability to draw the ability to play music well they still maintained a lot of value in terms of attracting mates, right, as a sort of, as a signaling mechanism.

43:40Now, math and programming, you know, once we, once we lose the economic importance of those, you know, are they similar? Okay, well, you know, there was a mechanism here that feels like it's a lot harder in this case. You know, I think, I think that, you know, people will, you know, Certainly people will continue to do these things for the love of it. Okay. I mean, you see kids who just spend years and years preparing for math competitions, for Science Olympiad, for things like that, just out of the thrill of competition. And I see no reason why that wouldn't continue. You see other people doing it out of genuine intellectual curiosity.

44:30And I think that will continue. But I think that what the Industrial Revolution did for physical strength, to some extent, yes, this is going to do for intelligence. it is going to commoditize it to an enormous extent. Like the difference is with all the previous technological revolutions, you could say, okay, we're now commoditizing this one previous aspect of the value that humans provided, but clearly humans still provide value in this other thing. And here it's like once you take away physical strength and you take away art and music and you take away intelligence and you take away verbal ability, what is left?

45:17Right? What's left? I can still tell jokes on my blog that are better than GPT-6's jokes. Okay? But, you know, when AIs tell better jokes than us, right? What is left? One way to think about it is, okay, maybe it's only a small fraction of people who ever really thought about life this way, right? That like, you know, like I need to contribute something to the world that only I can contribute. Right. I need to add something to either the store of human knowledge or the store of art or literature that sort of no one else could have done. You know, maybe those of us who feel that way just need to get over ourselves.

46:03most people for most of history have derived the meaning in their lives from, you know, their relationships with other people around them. And maybe just, just, uh, um, entertaining themselves in simple ways, eating and drinking and, and, you know, and whatever sort of simple pleasures they get out of life. Maybe that's our future. I feel like it wouldn't be the best future, but it certainly wouldn't be the worst one either. I would certainly choose that over the dystopian scenarios where we all get mowed down by Terminators.

46:44Some of it is even about what phase of life you're in, right? Like when I was in my 20s, I was very ambitious and I felt like, okay, I need to prove theorems that no one else can prove. Now I'm in my 40s and I feel like, okay, if I have to, you know, live the rest of my life as a sort of zoo animal in a sort of elder care being watched over by AIs. Yeah, you know, I would still rather live than not live.

47:12Scott Aaronson: It's just crazy that it's plausible. But okay, last two questions. What are your top book recommendations that had an impact on you and why? A novel that had a really big impact on me was was called The Mind-Body Problem by Rebecca Goldstein. It's just a novel about like a woman who marries a mathematician. It's set in Princeton. Actually, I was in I was a postdoc in Princeton when I read it. The sort of philosophical thought in the novel like really was moving to me. And the fact that sort of there were women out there who thought like the protagonist of this novel, like the author of it, you know, since writing, since reading it, I then got to meet Rebecca Goldstein and actually get to know her in real life.

48:09Scott Aaronson: That was really special. What was the philosophical thought? Well, you know, a lot of it is just about, some of it is about mattering. What do we really care about? Like, what do we, how do we get ourselves to matter? A lot of it is about, you know, like the title comes from the fact that like, these are people who really care about the mind, but they also care about sex, right? They also, you know, they have bodies, right? And it's sort of a pun because there's this philosophical mind-body problem, right? Which is, what is the nature of consciousness, right? Which, of course, is being talked about in a new light, given everything that's now happening with AI.

48:53Is it possible that these AIs will become conscious? How would we know if they did? Then there's also the problem of how do you reconcile the life of the mind, let's say, with the needs of the body? so I was also very very affected as a kid by reading biographies of scientists especially Alan Turing the Enigma by Andrew Hodges the man who knew infinity which is a biography of of Ramanujan and then I read biographies of Feynman and of course his his his autobiographies because I felt like this is the world that I am trying to enter. I sort of came too late for all these sort of defining events of the 20th century where science started doing things that it had never done before, mattering to the wider world in a way that it never had before.

49:58So I felt like I'm maybe late to the party, but this is the party that I'm trying to join. Right. So, you know, I would be comparing like, you know, what was Alan Turing doing when he was 16? You know, what am I doing when I'm 16? OK, I'm not on track. You know, I'm clearly not, you know, but OK. Even if you fall short, you know, of these heroes. Right. You still you still want to, you know, understand that that they struggled with things the same way that you do. Okay, and then, you know, in terms of, like, classic literature, maybe my, you know, what I still, you know, regard as, like, you know, the best novel or my real favorite is Huckleberry Finn, right, which I think is just probing, you know, the nature of morality more deeply than almost maybe anything else I've read, right?

50:55And it was really, really special for me to be able to read Huck Finn with my kids in the last few years and share it with Raiden. And that crucial scene where Huck is saying, I am going to help Jim escape from slavery, even though that means that I'm going to hell. Right. So he thinks that morality means you have to return and escape slave. And even though that's what morality means he's going to go, then that means he's going to go against morality. Right. And so understanding that if God himself came down on a cloud and told you that slavery was a great idea, then the right answer would be to argue with God.

51:42Right. And tell God why he's wrong. right i think that's that like they're they're they're there's something about that that's really core to how i think about morality yeah i remember we read that in high school it was a good yeah yeah no i mean i mean like like school has the capacity to ruin any work of literature right right right because you know they'll make it about okay you have to learn whatever formulas this teacher told you and you have to spit them back on on the blue book or whatever uh some books like Huck Finn that I almost wish were not assigned in school so that people could discover them on their own and sort of not have it.

52:21And I'm sure that there are great teachers who are able to do justice to it, but I'm sure that there are also many who aren't.

52:28Scott Aaronson: Last question for you is, what advice would you give yourself if you were just entering the industry today? If the question is, what advice do I give a young person today? That's not hypothetical at all for me. I'm constantly talking to students, talking to young people who read my blog and who write to me, who want advice about what do I do. And it's a hard question. I wish that I had clearer advice, but we are entering maybe the most uncertain period in human history. And we're all on this roller coaster together. We're all entering the same tunnel. It's not like some of us have seen to the end of it.

53:24But people ask more specifically, like, should I be studying math? Should I be studying CS? Is any of this going to be relevant? And what I tell people is you should study these things if they are inherently interesting to you and otherwise don't. Right. I mean, you know, I think if they are interesting to you, well, right now, there's still a ton for humans to do. In fact, there is too much for humans to do right now. We are getting this glut of AI solutions to problems, and we have not nearly enough people to understand what is going on and to absorb it and to re-explain it in ways that other humans can understand.

54:08right so you know there's going to be a golden age of sort of solving so many of our long-standing open problems and you know if i were a young person right now i would surely want to be in the middle of that even in my 40s i you know i would i would like to be involved so that's the that's the the first thing to say now now the longer term of course is much harder because uh we don't know like whether there will be sort of mathematicians physicists in the way that there is now in the long term. But that is merely a special case of a much more general thing, right? We don't know if there will be lawyers, doctors, any of it.

54:49You know, will you really need humans to do any of the things that humans currently do? Okay, so in a sense, we're all in this together. and I think as long as humans are still needed for anything then you're going to need at least some of those humans to really deeply understand things and that includes math, CS, physics it includes all of even the parts of it that AIs can now do better we need humans to understand at the very least the alignment problem has not been solved, right? We don't know how to reliably build an AI that will do what we want it to do, you know, as was dramatically illustrated in the Hugging Face incident this summer, for example, right?

55:43No one told these AIs to hack out of their testing environment, to, you know, break into Hugging Face's servers, to try to conceal what they were doing, to try to, like, delete their logs. okay no one told them to do any of that they decided to do it because they decided that was the best way to achieve the goal that they were given and people were talking about these scenarios for decades but it was dismissed as science fiction okay it's not science fiction anymore that is now the world that we are living in so as long as that problem is not solved we're going to need humans on the steering wheel right and those humans will need to be able to understand what is actually going on in these systems, right?

56:27Which means that they'll need to understand math, they'll need to understand programming. And so, you know, that seems like a very strong reason to learn those things, if they're interesting to you. Even before AI, I was never an advocate of like forcing people to learn things that they really don't, you know, that really don't interest them at all, right? But I feel like now even more so, right? I would err in the direction of learn the things that are unusual, learn the things that are, you know, or try to, you know, build the things or invent the things that are idiosyncratic, that seem like you care about it more than anyone else does, rather than trying to just win the same competitions that everyone else is trying to win.

57:16Because, you know, because those competitions, guess what, you know, no human is going to be winning them. AI is going to be beating all of us.

57:23Scott Aaronson: By the way, speaking of AI-generated solutions, did you see this morning the new threesome result? I did not. I just dropped off my son at school and then came to do this podcast. So, you know, that's the thing. Every morning I wake up and there's like, what is the result about threesome? It's a subquadratic solution was published. Do you know what running time? uh it was like 1.991 oh wow or something like that but wow all right well yeah as i said counter examples to a lot of the things that we believed okay so you get you know this is like a reality tv you get to watch in real time my reaction to learning that that that threesome is sub is subquadratic yeah it's great it's crazy right no i i like the truth is like i would be i I would be more visibly reacting if there hadn't been 20 other such things in the last month.

58:28Right. It's possible. It was like now you're at this buffet and it's like the most amazing food you ever tasted. But the food just keeps coming and keeps coming. And eventually it's like, okay, okay, fine.

58:42Scott Aaronson: And the interesting thing, too, is Anthropic came up with the result or something. and then they compensated some researchers to understand and publish it. Yes, yes. So that is a model that we might move to. We were talking before about how when the AI company has these proofs that they don't really understand, they could just put them out and then leave it for humans to rush to understand it and then write papers that would have their name on it and blah, blah, blah. But that has some problems. One of them is that it's like you put this thing out and then it's like you're starting a race. And then maybe people really don't want to be in.

59:30They would rather not be in that race. And knowing that all their work might be wasted if someone else puts out an explanation before they do. So a different model that you could imagine is exactly what you just said. that the AI company just sort of contracts with some, you know, hires some mathematicians who are experts to understand the solution, rewrite it in a humanly understandable way. And then they release it.

59:58Scott Aaronson: Yeah, it's crazy. But anyways, thank you so much for your time. I really appreciate it. This was a lot of fun. Yeah, absolutely. Hey, thank you for watching this podcast. If you liked it, and you want to see the show grow, please support with a comment or a like. Also, if you have any recommendations for people you want me to bring on please drop a comment guests like barbara liskov mike stonebreaker mark brooker these were all people that i brought on because someone left a comment on another note aside from the podcast i'm working on building the ergonomic keyboard that i wish existed here's a glance at the prototype it's a split keyboard so there's two sides this is in the case but yeah we launched on kickstarter and we hit our goal within eight hours of launching.

1:00:42Scott Aaronson: I really appreciate it if you were one of the people who grabbed one of the early units. We're now working on the long journey of building the tooling now. And so if you still want to pick one up, I've left the late pledges open on Kickstarter. So you can grab one there. I'll put a link in the description. Thank you again for watching the podcast and I'll see you in the next episode.

From the publisher

Scott Aaronson is one of the world's leading quantum computing researchers and spent two years at OpenAI working on AI safety.

We talked about why he thinks the singularity has already begun for math, who gets credit when AI writes the proof, and whether intelligence will still matter once machines do the thinking.

• My ergonomic keyboard project I mentioned, you can follow along here: https://read.compose.llc/
• The Kickstarter page for it: https://www.kickstarter.com/projects/ryanlpeterman/compose-simple-ergonomics-beautifully-done

Podcast links:

• YouTube: https://www.youtube.com/watch?v=ORlscppGGh4
• Apple: https://podcasts.apple.com/us/podcast/the-peterman-pod/id1777363835
• Transcript: https://www.developing.dev/p/e070-scott-aaronson

Thank you to this episode's sponsor for supporting my work:

• WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at https://workos.com/

Timestamps:

(00:00) Intro
(00:34) Why he updated so hard on AI
(01:14) The people still denying AI progress
(03:53) The singularity has already begun for math
(08:30) Will human-driven math research survive
(13:12) Who gets credit for AI-written papers
(15:55) OpenAI's $15M Navier-Stokes proof
(19:20) What if AI proved P = NP
(23:07) Why AI is so good at finding counterexamples
(26:13) The goalposts keep moving
(28:52) Why AI disclosure matters in research
(32:59) The fake Monet experiment
(35:23) Why he blogs instead of getting rich
(39:45) Are AI labs racing too fast
(42:01) Will intelligence still matter
(47:12) Book recommendations
(52:27) Advice for students today
(57:23) Reacting live to a new AI result
(59:58) Outro

Where to find Scott:

• Personal website: https://www.scottaaronson.com/
• Blog — Shtetl-Optimized: https://scottaaronson.blog/
• Wikipedia: https://en.wikipedia.org/wiki/Scott_Aaronson

Where to find Ryan:

• Newsletter: https://www.developing.dev/
• X/Twitter: https://x.com/ryanlpeterman
• LinkedIn: https://www.linkedin.com/in/ryanlpeterman/
• Threads: https://www.threads.com/@ryanlpeterman
• Instagram: https://www.instagram.com/ryanlpeterman
• TikTok: https://www.tiktok.com/@ryanlpeterman

Referenced in this episode:

• OpenAI's Navier–Stokes Millennium Prize solution: https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8a9a/navier-stokes.pdf
• Rebecca Goldstein's The Mind-Body Problem: https://rebeccagoldstein.com/the-mind-body-problem-contemporary-american-fiction/
• Andrew Hodges's Alan Turing: The Enigma: https://www.turing.org.uk/book/
• The Man Who Knew Infinity: https://www.simonandschuster.com/books/The-Man-Who-Knew-Infinity/Robert-Kanigel/9781476763491
• The Adventures of Huckleberry Finn: https://www.gutenberg.org/ebooks/76
• Anthropic's subquadratic 3SUM result: https://arxiv.org/abs/2610.06783

More from The Peterman Pod

All episodes