1. Podcasts
  2. The Data Exchange with Ben Lorica
  3. When AI Writes the Code, Who Proves It Works?
The Data Exchange with Ben Lorica

29 Oct 2026 · 46m · 18 chapters

Episode artwork

When AI Writes the Code, Who Proves It Works?

In short

How AI-generated code and proofs shift the bottleneck from writing to verification, and how Lean enables machine-checked formal verification (and auto-formalization) for software, math, and agent safety.

Guest

Leo de Moura, VP and Distinguished Scientist in AWS Automated Reasoning Group; creator of Lean (lean-lang.org) and chief architect of Lean Focus Research Organization.

Key claims

Frontier models make it easy to generate Lean code/proofs, reducing the cost of formal verification; Lean’s trust comes from a small kernel that machine-checks proofs. Formal verification complements testing by proving properties for all inputs, not just sampled cases. AI-generated Lean models must be validated for faithfulness to the original code; multiple independently implemented Lean proof checkers/kernels reduce trust risk.

Notable examples

Kim Morrison porting ZLIP to Lean with a “compress then decompress returns original data” property; Syncrypt (Microsoft) using Aeneas+Lean to verify crypto libraries; Cloud Code developer Boris using Claude+Lean to find and fix real bugs; Navier-Stokes formalization mismatch discussed; VersoBlueprint aligns natural language and Lean (“virtual blueprints”).

Written by AI. May contain mistakes.

Chapters

0:39

The Challenge of Verification in AI

Discussion on the ease of code generation and the challenge of verification.

“So I guess the overarching thesis of this conversation is that we're in an era where generation has become easy, not necessarily cheap, as people who use thought code will attest, but easy.”
1:25

Understanding Formal Verification

Exploration of what formal verification adds compared to traditional testing.

“code reviews, QA, and testing, what does formal verification add?”
3:38

Lean's Role in Formal Verification

Detailing how Lean can be used for formal verification and its capabilities.

“Now, with formal verification, you can start with the properties.”
5:50

The Impact of AI on Verification

Discussion on how AI can aid in the auto-formalization process and model verification.

“And this has a name, at least in mathematics, they call it auto-formalization.”
6:40

The Navier-Stokes Equation and Formal Proofs

Examining the formal proof of the Navier-Stokes equation and its implications.

“I mean, now AI, for example, the mathematical library built by humans for Lean is called Mathlib.”
10:24

Improving Tooling for Verification

Discussion on tools like VersoBlueprint and the dynamic nature of Lean proofs.

“before OpenAI even started directing their models, we had a curated by human statement of the Navier-Stokes problem.”
12:22

The Importance of Formal Verification

Explaining the necessity of formal verification beyond traditional testing methods.

“I mean, you're making, but you find the proof, but Lean keeps saying, yes, it's true of proof.”
14:00

The Role of Verification in Software Engineering

Learn how verification complements coding in software engineering and its evolution.

“It increases your confidence, your understanding, and then you prove now you're proving for any input.”
18:06

The Shift Towards Formal Specifications

Discover the changing focus in software engineering towards formal requirements and specifications.

“This part, I think we're addressing well.”
23:10

Verification in the Age of AI Agents

Explore the implications of verification for AI agents and the need for reliable frameworks.

“But doesn't that mean you have to almost revamp the entire CS curriculum?”
Show all 18 chapters
28:00

Formal Verification and AI Agents

Discusses the importance of formal verification for AI agents and how it can prevent vulnerabilities.

“Good specifications for them, proofs that the implementation meets these specifications.”
31:30

Challenges in Modeling with AI

Explores the complexities of nailing down requirements and how AI can assist in creating formal models.

“And these agentic systems are complex systems, right?”
32:45

The Impact of AI on Mathematics

Examines how AI is transforming the field of mathematics and the potential implications for mathematicians.

“In Israel, the definitions there, you know they're precise.”
35:12

The Future of Theoretical Physics

Discusses the implications of AI for theoretical physics and the potential for disruption in the field.

“the ability of AI to translate math into Lean, I think it's just going to continue, which begs the question, Leo, why can't I do this in theoretical physics?”
36:59

Mixed Feelings on AI Disruption in Fields

Explores the diverse reactions from the mathematics community regarding AI's impact on their work.

“have you personally heard from your mathematics friends?”
39:42

Formal Verification in Finance and Other Fields

Discusses the application of formal verification in finance and other under-the-radar areas.

“I mean, for example, the James Webb telescope, you have to get, you can write a proposal, you get access to the telescope.”
42:00

Exploring Formal Verification in Finance and Science

Learn about the application of formal verification in finance and other fields.

“right but the algorithms are this that so in math it's you have proofs theoretical physics you have proofs, but algorithms, how does that work?”
44:00

The Challenges of Formalization in Non-Scientific Fields

Discover the complexities of applying formalization in fields like accounting and law.

“Because the MATLAB people toiled in obscurity for years, but once auto-formalization happened, boom.”
A man walking through an airport, asking his phone a question

Ask about this episode

Get the gist, explore an idea or ask about something said.

Claude

On The Data Exchange with Ben Lorica, what are the key points of this episode?

Searching your listening

Transcript

Automatic transcript. May contain errors.

0:03Ben Lorica: All right, so today we have a treat. We have Leo de Moura, VP and Distinguished Scientist in the Automated Reasoning Group at AWS. And more importantly, he is the creator of Lean, which can be found at lean-lang.org. Lean is both a programming language and a tier improver. And he is also the chief architect of the Lean Focus Research Organization. So with that long-winded introduction, Leo, welcome to the podcast. Thank you. Thank you. Very excited to be here. All right. So I guess the overarching thesis of this conversation is that we're in an era where generation has become easy, not necessarily cheap, as people who use thought code will attest, but easy.

0:56Ben Lorica: but easy. And so then one of the key challenges is verification. And so Lean is normally associated as the go-to tool. In fact, it's going to be a tool, I think, that will increasingly be in the lips of many people in AI in the next year around the topic of formal verification. So I guess let's start here, Leah, just the basics. For someone, for a listener who's accustomed to code reviews, QA, and testing, what does formal verification add? Yeah, at least both a programming language and a proof assistance, right? You can, at the same time, write codes, but also state properties the code should have.

1:44You can state open conjectures in math, and you can provide proofs to these claims, right? But the proof is not informal. The proof is written in the Lean language, and more importantly, it's machine checked. Lean is a big piece of code, but to trust a result proved with Lean, you need to trust only the kernel, right? It's where the proof is being checked. There's lots of, like any other programming language, you have all the tooling, LSP server, IntelliSense, code generators, auto-completion, all these features are available in Lean, but this is not important for the proof checking step. And we have more than one checker.

2:33You can, in Lean Proof, you can view proof as data that can be manipulated, that can be transformed. You can export your proofs and check with different checkers. Some of these checkers are verified now. We have two verified checkers. The third is coming soon. But this ability to be able to have machine-checkable proofs, it helps to address the trust bottleneck, not just the trust bottleneck between humans, but also between humans and AI. For example, suppose that you want to synthesize a piece of code. You want to implement a piece of code that has specific properties. In the past, you would implement the software.

3:16Maybe you would add some testing infrastructure to make sure these properties hold. Some property-based testing is very popular these days, right? And you would write the testing framework. Testing framework is very useful for finding bugs, right? But it's not useful for proving that there are no bugs.

3:37Ben Lorica: You may even have a separate team conduct the testing. Yes, yes. Now, with formal verification, you can start with the properties. You can, for example, write the properties down, ask the AI to implement a piece of code that satisfies these properties. You're going to implement the code and write a proof, a lean proof that will be machine-checked. And the AI now is fully capable of doing that. For example, a colleague and friend of mine, Kim Morrison, she did an experiment where she ported ZLIP to Lean. This is a compression library. Now she has a property, a property that says that if you compress data and you decompress, you get the data back.

4:26There is no loss. The data was not corrupted. It's an important property.

4:30Ben Lorica: No matter the level of compression. Exactly. Exactly. Exactly. Great. great that the theorem says exactly that's the use that one of the parameters of the theorem is so leo for our listeners so so she did this she wrote this compression tool using lean and then lean served as kind of the formal verification tool so if i'm a developer and i do the same thing and i say i'm gonna use lean to formally verify my software but i'm gonna write it in rust first. So can Lean still help me? Yes, yes, it can in two different ways. The first way is using Aeneas. Aeneas is a tool that allows you to verify REST code using Lean.

5:18Aeneas is used by, for example, by the Syncrypt team at Microsoft. Syncrypt is the verification, the crypto library at Microsoft. They use Aeneas plus Lean to verify the crypto library. You can start by using Aeneas. Another possibility is to ask your AI, build a model of my codes in Lean. The AI will read your codes.

5:45Ben Lorica: No matter what language. Yes, no matter what language. I mean, if someone had told me this last year, I would say this is science fiction. And this has a name, at least in mathematics, they call it auto-formalization. Yes. So in mathematics, for our listeners, you might have a math paper, and it's a paper. So it's composed of words, mathematical symbols, theorems, lemmas, and so on and so forth. And then auto-formalization, which was, as Leo points out, was not a thing as recently as maybe a year ago, is now a thing. Exactly, exactly. So, Leo, this step, you folks at Lean had nothing to do with.

6:30Ben Lorica: This just happened because the frontier models became capable of doing this. Exactly. Exactly. From the Lean side, what changes for us? Scalability, right? I mean, now AI, for example, the mathematical library built by humans for Lean is called Mathlib. It has 2.5 million lines. Incredible. This is like, for our listeners, imagine every, this is not the case, but imagine almost all mathematical books and papers were translated in the way we described using auto-formalization into Lean, which means then you have access to all of them. Yes. But look at the speeds now. There are lean proofs now that have been published that are 50 million lines.

7:19The math lead took 10 years written by humans to go from 0 to 2.5. And now there are people writing proofs that's AI writing the proof that's 50 million. The Fermat's last theorem proof that was written by anthropic AI is 13 million lines.

7:37Ben Lorica: In the case, as you point out, Leo, in the case of software, there's something equivalent going on for every programming language. Yes. There is now, the frontier models are now capable of translating whatever program you wrote in whatever programming language into Lean. And then now you have the Lean infrastructure for verification. Exactly. But one important thing to keep in mind, when you create a model of your program in Lean, if you use AI, there's always, is this model, the question, the main question is, is this model faithful to your code? Some people now are using differential testing, right?

8:20They run the Lean model in the program and see if they compute the same thing, right, to find discrepancies. And people have been finding bugs.

8:28Ben Lorica: So what you're saying is you're still responsible for this. Yes, exactly. And I see it happening in an incremental way. For example, a few weeks ago, the Cloud Code main developer, Boris, he posted that video where he just asked Claude, oh, check Cloud Code for bugs using Ling. And Claude starts creating the models, found bugs, tried to prove properties, noticed that he could not prove the properties, found bugs, filed the PRs, and Boris merged them. Yeah, these were real bugs. And Boris said, I don't even know link. Right, right. But this is, for bug finding, this works reasonably well. But for the next step, for you to say, yes, there are no bugs, now you have to start looking at the model and saying, yes, this is an accurate model of my codes.

9:22And I want to maintain them in sync. Maybe I will use the AI. I will prove properties. You can start using Aeneas if your code is in Rust. You can try to use Aeneas to prove that it's a faithful model of your code. But we expect in the next few months and years, people will focus way more on the specifications, on the models, on the properties you want the software to have.

9:48Ben Lorica: Yes. In fact, this same phenomenon is happening in math most recently in this proof of the Navier-Stokes equation, right? So maybe the auto-formalization step of translating the actual statement of the problem may not have been as faithful or whatever. But then it turns out that, in fact, they did prove the result. It's just that, you know, you still needed humans to kind of catch this and make the subtle adjustments, right? Yes. No, this is an important point for Navier-Stokes because it's so important results. before OpenAI even started directing their models, we had a curated by human statement of the Navier-Stokes problem.

10:34The statement of the problem was already in lean, has been curated by experts. There was no doubt about whether the statement of the problem was correct, formalized. There's some controversy in the results because OpenAI presented informal proof written in natural language and a formal proof written in Lean. The informal proof and the formal one that mismatches in the proofs.

11:03Ben Lorica: I think it's at the level of, are you assuming fourth or fifth derivative? Yes, yes, yes, correct. Which matters for mathematicians, but you can see how subtle the differences were, right? Let me give you a point. There's a tool that I think is important to tell your audience. There's a tool called VersoBlueprint. VersoBlueprint is a tool for if you want to present a Lean result, mixing natural language with Lean, you can use VersoBlueprint. It's almost like literate programming. You can mix natural language LaTeX with Lean and align the two presentations. We are very confident that if the AI was using virtual blueprints, this mismatch would not have happened.

11:54This is a matter of improving the tooling. Another misconception people have is that they think a lean proof is a static object. It's not. Like we refactor code, you can refactor proofs. You can simplify them. There are groups, for example, for the Fermat's last theorem proof by Anthropic, there are groups that already reduces the proof by half.

12:16Ben Lorica: By the way, this is true for proofs written by humans too. Yes, true. Yeah, yeah, yeah. But here you get the machine checked. Right? I mean, you're making, but you find the proof, but Lean keeps saying, yes, it's true of proof. Right? You can manipulate. You can ask. The AI can help you navigate the proof. You can align the proof of informal presentation using Lean blueprints. It's all possible, right? I mean, this is one thing that I see on social media people are missing. These formal proofs are not static objects. They're objects that you can manipulate, you can simplify, you can probe, you can ask questions, you can interact with them.

12:58Ben Lorica: So, I guess I want to just nail down this one point, which I think should be clear by now. but I just want to make sure the audience who may not be familiar with what we're talking about gets this point, which is this difference between formal verification and plain QA and testing. Why should I bother with formal verification when I already have AI writing all these tests for me? And, you know, I'm using AI to generate a plethora of tests, edge cases. I have maximum coverage, Leo. I don't need verification. Yeah, testing is great for showing the presence of bugs, but not their absence. Even if you try to write, use the AI to generate many test cases, your AI may not be as smart as the AI that will be used to find a vulnerability in your software.

13:58right we should not i think technologies the technologies are not in conflict with each other they complement each other for example one mistake i see many people doing trying to prove something that may be buggy well no matter how many the amount of resources you put how many tokens you burn you'll not be able to prove something that's not true right so testing is always a great way to start. It increases your confidence, your understanding, and then you prove now you're proving for any input. You're not proving like only for the tests I tried it to work, but you're proving that for any possible input your program works, right?

14:42It's for all instead of for some.

14:44Ben Lorica: And honestly, I think software engineering may be behind other engineering disciplines, I don't know, like aerospace or whatever, right? So maybe they've had more rigorous discipline around verification because after all, there's much more serious consequences in these other fields, right? Yes, great points, great points. I think this is just a case now that we have more and more people writing code and code is easier to write. And it just happens that verification has had its image net moment, so to speak. I guess, Leah, the main thing that's happened, because you folks have been working on Lean for a while, and Lean has been rock solid for a while.

15:29Ben Lorica: So the main transition is really this auto-formalization, the capability of the frontier models to write Lean. Yes, and also connects to your previous comments. You made the point, yeah, in some industries, People have used formal verification in the past. But why it was not popular if it is so good? Because it's too time-consuming to prove things by hand. And AI dramatically reduces the cost of verifying codes. And it gives you other benefits. After you have a formal proof, right, you can start asking the AI, I want you to change, optimize my codes. But you have to keep proving that it's correct.

16:14is almost like the proofs are guardrails. The AI can try wild optimizations that you may not even understand exactly how they work, but you have a proof that saying, yes, it still computes the same thing. It's way more efficient, but it still computes the same thing.

16:35Ben Lorica: So based on what we've been discussing, it seems like the software engineering profession will now focus on not writing code, maybe not even verification, because verification, maybe you can also semi-automate based on what we've discussed. It's really requirements, right? So describing what you want. So there's two buckets, I guess, Leo. So there's the requirement gap. So is there a difference between what you want and what you are able to write down? and I guess what some people call the model gap, which is really are the conditions around what you're testing representative of production. So in other words, maybe your testing scenarios are too idealized in some ways.

17:31Ben Lorica: And obviously, you alluded to this earlier, to some extent, you have to trust Leo and Thief, the lean kernel, right? So as Lean becomes more popular and you rely on it, and by relying, I mean you're relying not only that Lean verified your code, but all the libraries that your code depended on, right? So I guess those are three things. So there's the requirement, there's the model, and then there's the Lean kernel. Well, we have multiple. You don't need to trust us that kernel is implemented by other teams. This part, I think we're addressing well. I mean, we even have a kernel arena now where we collect kernels.

18:14Yeah, and they compete against each other to see which one is the fastest, if they get it wrong, if they have a bug, they are penalized. And everybody can see it. It's called arena.linlang.org. You can see all of them. And all of them are open source? I think pretty sure. I mean, I cannot say I don't use all of them. I use the main ones. The ones I use, they're all open source. Do you have to be written in Lean to be? No. No. Some of them are implemented in Rust. There is one implemented in C Sharp. There is another one implemented in Haskell. Believe it or not, there is one implemented in tech.

18:55Tech is this language for typesetting program. Math papers and papers in computer science, math, and so on. But tech is a full-blown programming language. and a colleague of mine, Kim Morrison, implemented a checker, a kernel in tech, just to show that you can implement it in many different programming languages. Oh, there's one in COBOL, for example, implemented one in Fortran.

19:21Ben Lorica: So then the education and training of a software engineer will change, right? Well, I think there will be way more focus on writing formal specifications, writing properties, the requirements is essential. So many times we don't even know exactly what the software is supposed to do. And by the way, Leo, so some of these applications are complex, right? So they interact with databases, distributed systems, and so on. So then, I mean, how do you exactly? You're not going to be able to nail down the requirements. Or maybe that's what distinguishes you. in the past, what do you call it? The superstar programmers were the super coders, right?

20:10Ben Lorica: Maybe in the future it will be the people with the ability to, okay, so these are complex. I'm going to use Kafka. I'm going to use Spark and whatever, right? Yeah, but the models will have imprecisions, right? Sometimes because people don't know exactly what they want. But look at the difference. Before, people would write a doc, before they design a database, they would have documents explaining the design, the architecture, in natural language. Some people would use a word document to write the specification of the data. Usually, there's some sort of architectural diagram, right? Yes, exactly.

20:50Exactly. But then what happens? Now, you have these informal documents. You have no way to check if the actual implementation matches these documents. Some human has to go over the code base and keep maintaining both in sync. It's a pain. I mean, you change the documents, you have to check if the implementation still matches. All of us know that we have comments that are out of sync in our code. But if everything is formal, you are going to have tools that will check whether the documents, your design matches the implementation or not. This is a step forward, right? For sure, you are going to have documents that are imprecise.

21:31Your customer wants X. Your specification is X prime. But you're going to keep polishing, refining. This is the ground truth now.

21:41Ben Lorica: And as the thing interacts with the real world, you go back and refine it. Exactly. And refine it. But now your code is going to be, you're going to have a tool that says the code matches the new model. You refine it. You improve it. it matches. For example, some people say, oh, the code has been verified. There are so many caveats. What does it mean to verify something? The properties you wrote are the ones you really want. I mean, you're going to be now this process where you keep polishing your properties, making them more precise. Some software you will have, you're going to model threats, model, you're going to have threat models for people that try to attack your system.

22:27Ben Lorica: So these descriptions, these requirement docs, I guess these PRDs become what you track in your source control system because basically, in many ways, Leo, as you learn more, you're constantly verifying each version, but it's almost going to be the case that each version, I'm just going to have the AI write it from scratch? Why do I have to save any of the previous code? Yes, yes. So the crown jewels becomes this, which ultimately is still going to be owned and maintained by humans, right? Exactly. Exactly. With the systems of AI. Yeah, yeah, yeah. Yeah, yeah, yeah. And tools like Lean, right?

23:12Yes.

23:12Ben Lorica: But doesn't that mean you have to almost revamp the entire CS curriculum? No. Not completely, Completely, but it's going to adapt. I mean, I've seen many colleges and universities where formal verification is not even a required course. And it's the only one, it's more like a curiosity that people do not use in industry because it was too expensive until now. I think the focus on formal verification will increase, but I think it's still valuable to learn low-level education. There will always be people that learn low-level assembly codes, that learn low-level languages, that learn how to write codes and check that we will be assisting designing systems at a very low level.

24:04This will continue to exist. I mean, but you're right. It will change.

24:08Ben Lorica: But in this world, Leo, you could be a very productive and very much in-demand engineer without actually being proficient in coding. Yes, this is going to happen. Well, but it's some kind of, you're writing, let's see, it's still coding, writing formal models. You can't do it as coding. You're just working a higher level. Yes. Yes. By the way, I think you wrote this phrase, which I love, and maybe you can elaborate by what you mean if you did, in fact, write this phrase, which is the verified stack. Yes. What do you mean by the verified stack? Oh, the verified stack is about the reach-hole software stack.

24:53I mean, there are some pieces of code that is not a big concern if they are buggy or not. I mean, for example, a video game is a piece of software. If it is buggy, it's annoying to the users, but there's no crisis in the world if there is a bug there. The critical software stack is our operating systems, our compilers, the communication layer, TCP, IP. This is what I mean by the software stack, and it would be great if you have models, specifications for all these components. we have experts writing them discussing them and we start verifying property making sure these specifications the code the actual code matches these specifications some direction that I think is really worthwhile investing in the future so

25:47Ben Lorica: which is interesting so Leo imagine you're a CTO of a big Fortune 500 company right and then I come to you as a vendor and I go hey Leo You should trust our software because we are formally verifying. Yes. So what are some of the key questions you need to ask me in order to make sure that I'm not BSing you? Show me the specifications. Show me your specifications, the properties, what you have verified. Someone can formally verify that my property is just true. It's basically saying there's no property. You're not making any claim. I think you're absolutely right that we should be aware of claims like, my code is formally verified without explaining exactly which properties have been verified.

26:37What's the specification you verified against?

26:41Ben Lorica: What proof checker kernel did you use? Exactly. It's also important. Which tool? Another one that should be important, the trusted code base. There are tools that have a large trusted code base. In Lean and other proof assistants like Rock, Isabel, they just focus on the trusted code base should be really small, right? Yeah, yeah, yeah. And obviously, to the extent that you can convince me that your software has run in a similar setting, in other words, a realistic setting, maybe that adds to the need behind the verification claim, right? So what about we talk about software as if they're just static things, but now we have software called agents, right?

27:32Ben Lorica: And then we now are entering the world of long-running agents. So what does verification mean in the world of agents? It's just a great question. I think the first thing, we want to put guardrails on these agents would be great to have reliable sandboxes. I mean, we have been seeing the last few weeks, many people claiming the agents are escaping the sandboxes. Part of the verified software stack. The sandbox, we want reliable sandboxes, right? Good specifications for them, proofs that the implementation meets these specifications. We also want to be able to write policies about the agent's behavior.

28:12And we want to make sure the code that enforces these policies have been formally verified. Imagine you spend all the time writing policies about what the agents can do or cannot do. And then there's a bug in the code that enforces the policies, right?

28:32Ben Lorica: But what if another agent is doing the formal verification? So there's layers of trust, right? Yes, yes. One possibility people consider is, can an agent exploit a flaw in the link kernel or a flaw in the runtime where the link kernel is running? These are valid questions. And this is why we have multiple kernels. Because even the agent would have to find a flaw. It would try to trick you in believing it proves something. It does not. And this certificate, this proof, would have to be accepted by multiple kernels that have been implemented and verified. Independently. Yes, exactly. This is serious.

29:19I mean, these agents are extremely good at finding vulnerabilities in software.

29:24Ben Lorica: So have you interacted with people who are using the things we just discussed, verification, specifically for agents? Yes, AWS, we have AgentCore, Dogwoods, they are all using Lean now. Outside, I gave the example of the author. The author of all cloud codes used Lean to find bugs in cloud codes. We'll see more and more projects like that. Because one main problem with formal verification and tools like Lean was this steep learning curve. You would have to learn Lean, write these models. It's too much effort to start. But with AI, it helps you to reduce the startup costs. You can see value almost immediately, like Boris, his video he posted, where he just asked Claude, try to verify Claude code using Lean, and he got value immediately.

30:20And now it's worse. I almost feel like whenever you get immediate value, you start investing more time learning the technology. they're going to say, oh, maybe I should write my formal properties about my codes that they want them to hold. Oh, maybe I should learn how to read lean codes. I don't want to write it, but I want to be able to read, to be able to tell whether the model, the AI agent created a model that's accurate, right? And I want to discuss with the agents whether the model is accurate or not.

30:52Ben Lorica: So Boris, for long-running agents, right? So potentially agents that are exploring, doing research, exploring, coordinating among themselves, maybe for days, right? So we discussed that one of the challenges, obviously, is just nailing down the requirements, right? I mean, how can you nail down the requirements for something like that, where it's like a human being exploring all sorts of directions that imagine a human being who has infinite resources, can travel all over the world and can do anything. Yes, yes. It's a big challenge, but the AI can help you start. It can write the first model and it's good to say, oh, that's how we formalize.

31:34And these agentic systems are complex systems, right? There are multiple pieces interacting with each other. There are protocols there, communication protocols between these different pieces. And you want to make sure these protocols are correct and accurate. The AI agents will help you to get started with the model, the formal model. It's almost like it's going to give you the first draft and you can keep iterating from there, right?

32:02Ben Lorica: So it seems like ground zero for everything we've discussed right now are coding and math. And math, just over the last few weeks, you have fields medalists crying. I mean, the whole field has just basically been disrupted in a big way. Yes, exactly. My PhD is in math, so I used to be in that field. I wasn't very good. Well, maybe now I could be with infinite computing resources. But yeah, so in that case, it seems like the combination of Lean and Matlib especially makes math even more of a sweet spot than coding. Am I wrong? No, no. What MathLib gives you, MathLib was constructed by humans, by people that have a strong math background.

32:52In Israel, the definitions there, you know they're precise.

32:56Ben Lorica: But they must have started years ago. So they had the foresight that, hey, we're laboring here in the coal mines in obscurity. No one appreciates what we're doing, but once we scale. Yes, when they started, I mean, the first, I think nobody expected AI would get this good. But in their minds, one win for the math community was when they realized that Lean helps them to work in big numbers. For example, in 2024, Terence Stahl had a project where he broke a project in smaller pieces. He was like an architect, right? Yeah, yeah, yeah. And then there's no way for him to verify everything, personally verify everything, right?

33:42There, instead of having AI agents, you had many humans filling the holes for him. Many people that he never met. Giving him the proofs, he doesn't have to check the proofs correctly. It's almost like humans filling holes in his big design. And people said, wow, this is great. I remember him telling me, I never imagined that he would be working for amateurs. He only works as a professional mathematician.

34:08Ben Lorica: This goes back to what you said very early on, which is trust, right? Because in many ways, math, like most of science, is built on trust, right? So I'm a mathematician. I'm proving a theorem. I'm citing many, many other theorems from the past, maybe some from 50 years ago, which I personally don't know how to prove these theorems. I just trust that they're correct, right? Exactly. And this is what they've done, right? With Matlib. Exactly. But at the time, I remember Terry giving talks and saying, well, one day AI agents will be in the loop. They will be filling these holes. But I think none of us imagined it would come so quickly.

34:52Instantly, right?

34:53Ben Lorica: Yeah, yeah, yeah. A few months ago, this was, nobody would be. Six months ago, if I had told any of the Lean developers, the Lean mathlib maintainers, oh, we're going to have AI that will be discapable. They would not be. But now, because of auto-formalization, the ability of AI to translate math into Lean, I think it's just going to continue, which begs the question, Leo, why can't I do this in theoretical physics? Well, there's a library called Physlib for Lean, where they're formalizing physics. Well, now I believe they probably switch it to AI. I think everybody's moving to AI. I mean, even the Lean developers, we are now implementing Lean using AI.

35:41I mean, it was funny because it started like, oh, the AI is really good at finding bugs and isolating the bugs for us. Oh, AI is really good at patching the libraries. Oh, AI is writing code. Oh, wow, it's rising cold better than we do. I mean...

35:59Ben Lorica: So what's the timescale for physics to get disrupted as much as math? Theoretical physics. They had their first win. I mean, a few months ago, they found a bug in an important physics paper by trying to formalize the results. The challenge here is that in physics, we are creating models of the real world, right? Our models are always imprecise. They are models, I mean, of the real world. Well, they start out as theoretical, right? For example, string theory has never been proven experimentally. Yes. So you could still be in a field in theoretical physics where the experiments haven't happened yet, but still make progress and publish papers.

36:45Ben Lorica: And those people can get disrupted soon, right? Yes, yes. They can write now very accurate mathematical models, right? I mean, using Lean. Yes, this is totally possible. And so since Navier Stokes and since OpenAI published a bunch of more results, have you personally heard from your mathematics friends? Hey, Lidia, what did you do to our field? No, there's a lot of mixed feelings, right? That some of them are super excited, right? We're going to be able to attack problems beyond our imagination. That people concerned about what's going to happen. I mean, students, what's going to happen with me?

37:27I mean, like software development, we will have developers. The job will be different, but we will have the format. It's the same thing. Yeah, I mean, major disruption. Some people are angry. Some people are excited. Some people are scared. but I hope we will get this group it's going to be a good change I mean

37:49Ben Lorica: so the question in my mind Leo I've been thinking about this is in the past set aside software engineering in math right you kind of knew who the best mathematicians are right so you just start talking to them and then they know much more than you right so they're quicker they know much more than you they can prove harder theorems than you it's almost like raw IQ was there's nothing you can do I mean, that's why I left the field, right? Because basically I can't compete with these people, you know? So then, but now does that change who wins fields medals in the future, right? So maybe it's the person that's smart, but not off the chart smart, but has access to more compute.

38:34Ben Lorica: I don't know. So have you thought about, in software, I would imagine it's the same, right? Oh, yeah, same. Same. If I have access to more compute than you, maybe I can get more. I'll definitely get more done than you. Yes, exactly. Exactly. Right? Yes, I do. But it's all true what you said. I mean, for me, I do one of my superpowers was to tolerate pain. I mean, developing software is painful. Yeah, yeah, yeah, yeah. Proving theorems is painful. Yes. hey, I removed this pain, but yeah, I lost my superpower. It's not as useful as it was in the past, but we will be able to move faster, right? We will be able to achieve more.

39:22I think this is good. I mean, you're right that a person with a different skill set may be more valuable in the future or may be able to leverage, not say be more useful, but leverage the technology better. We will have issues about access to tokens, right? But we see that in science all the time. I mean, for example, the James Webb telescope, you have to get, you can write a proposal, you get access to the telescope. It goes in a queue. I believe in the future, the same thing will be for math. I mean, you have a massive project that requires an insane amount of money, but you're going to write a proposal.

40:04Maybe it will be funded. We'll get access to this model that's extremely powerful.

40:10Ben Lorica: Unless there's a breakthrough and the models get more efficient and, you know, kind of like in chess, right? So in chess, I guess I haven't kept up with this. Maybe you'd know. Is there a difference, Leo, between a chess program I can run on my laptop and a chess program that runs on the cluster? Does that give me an edge? I don't know. I'm not a big chess player, but my perception, the ones that's running a computer are already extremely good. I mean, and I hope you're right, Sammy. We will have models that we run on our computers, but I believe here these stakes are so high that there will always be the supercomputer.

40:48It's like the James Webb telescope. We can buy a telescope. It's really good. And have it at home. But the James Webb is way better. But it's way more expensive.

40:58Ben Lorica: Which means if looking ahead to the future, if you attend one of these academic conferences like in math, the person who used to be popular was the smartest guy, the field medalist. Now it's the guy with the cluster. Well, I mean, I think there's always that. Everyone wants to talk to the guy with the cluster. Well, maybe this guy is not even the guy that has the ideas. I mean, there's a committee probably that manages the James Webb telescope. But scientists submit proposals all the time. I mean, and they get access. I mean, I think the same thing will happen in the future. So we talked about, we started at coding and then obviously math.

41:38Ben Lorica: We speculated on physics. so what other areas are that are you're seeing that maybe is below the radar where this equivalent of matlib or something like that is happening i'm seeing many people interested in finance finance has many algorithms i mean that people interested in this area right but the algorithms are this that so in math it's you have proofs theoretical physics you have proofs, but algorithms, how does that work? There's a lot of math in finance. Yeah, yeah, I know. Yeah, yeah, yeah. I'm seeing people excited about applying formal verification to this area, using more precise models.

42:22Ben Lorica: But that, Leo, the challenge there is they don't work in the open, right? So they're secretive. Yes, yes, yes. So how do you come up with MATLAB? Yeah, I've seen people that are open to the trading algorithms. Probably they want to keep all secrets, right? But there is a mass background for finance. People want to formalize and make it public, right? But you're right, there will be people keeping their secret sauce. So finance, what else? I see people trying to build libraries for chemistry, biology. I think any field where... And what's the relative progress in each of the fields you named? Like, how far along are they?

43:09They are all, I think the finance is the one that's maybe, basically outputs math labs by far, super far ahead because math is precise. Then we have the physics library and we have the computer science library that are trying to catch up to math lab. Then we have these new initiatives in finance, chemistry. But if you get the papers in each area, you see how precise, how rigorous these papers are. The math in these papers will give us a hint on how hard it will be to create a formal library.

43:46Ben Lorica: Yeah, because there's theoretical and computational chemistry, right? Yes. But then, don't you think, Leo, because of auto-formalization, any community that decides to go in this direction will get there faster, right? Because the MATLAB people toiled in obscurity for years, but once auto-formalization happened, boom. Exactly. Yes. So are there non-scientific fields where you think this might happen? So for example, let me draw out a couple of examples for you, but I don't think they will work, right? Accounting. I have generally accepted accounting principles. I have the tax code. The tax code and the GAAG They are my theorems in many ways.

44:28Ben Lorica: Yes. But then here's the problem. I talked to some accountants about this. The problem is, here's some funds. Do I put that here or here or here? Right? So it's subjective. Yes, yes, yes. I saw people trying to do formal law. Oh, so yeah, yeah. I talked to someone about this too. But then the problem is with law, the reason we're suing each other is we disagree. Yes, exactly. But the parts of law, I had once an intern that he was a lawyer. He's a lawyer now. I mean, for example, civil procedures are very, very... Formulated. Yeah. Almost like a program. Oh, really? Yeah, yeah. There are parts of law that can be formalized, right?

45:14I see.

45:15Ben Lorica: Yeah, yeah, yeah. And with that, thank you, Leo. Thank you. Thank you very much.

45:25Ben Lorica: You can follow the work of Leo De Mura online at lean-lang.org. Please join the thousands of people who subscribe to our newsletter, which you can find at gradientflow.substack.com. And we are listener and reader supported, so if you can, please become a paid subscriber. Thanks for joining us. If you like the show, please subscribe and rate us through Apple Podcasts or Overcast or TuneIn.com or Spotify and never miss an episode. The Daily Exchange Podcast is a property of Gradient Flow. And I'll be back next week and we will do this all over again.

From the publisher

Ben Lorica speaks with Leo de Moura, creator of Lean, VP and Distinguished Scientist at AWS, and Chief Architect of the Lean Focused Research Organization, about how AI is making formal verification more accessible and potentially transforming software engineering.

Subscribe to the Gradient Flow Newsletter 📩  https://gradientflow.substack.com/

Subscribe: Apple · Spotify · Overcast · Pocket Casts · AntennaPod · Podcast Addict · Amazon ·  RSS.

Detailed show notes and transcript, can be found on The Data Exchange web site.

More from The Data Exchange with Ben Lorica

All episodes