265 - Jacob Barandes: A New Foundation for Quantum Mechanics

30:44But did you happen to study with him when you were at Columbia? Because I think he's been there since the late 90s. Oh, we met very early on. When I first started at Columbia, there was an opportunity for students interested in various subjects to sit down and get to meet experts in those subjects. And I met Achille Vartzi at one of these small group meetings. And I remember we were sitting in a common area. And he was telling us about his current interest at the time in the other kind of holes, not W-H-O-L-E-S, but just H-O-L-E-S, holes. And I found the conversation extremely interesting, but also very confusing because it was pretty early on for me and I didn't really understand what the big deal was about big questions in philosophy and metaphysics yet.

31:33Obviously, this has changed with time. I wish I'd asked better questions back then, but I really didn't know enough subject yet. And I remember he came with a sheet of paper and he held it up and he asked, what is this? And he held up the sheet of paper and we could all see, all the students could see that he had cut the shape of a horse out of the paper. It was paper and there was a horse cut out of it. And someone in the group, I don't know if it was me, maybe it was, it's hard to remember, but some of the group answered, it's a horse. And he said, no, oh, it's not a horse. There was a horse in here.

32:09It's been removed. What do we mean by this? And it was just an interesting set of comments. For those people maybe who are watching who remember the 2000 presidential election, the famous Bush v. Gore presidential election, some people may remember that one of the controversies came down to these ballots, these physical paper ballots down in Florida, and whether there were proper holes punched in them or not. And these sort of not fully perforated holes were called hanging chads. And there was this real question. Suddenly it became a very important question whether these counted as holes or did not count as holes.

32:52And I could be misremembering, but I do believe I once heard that Achille Vartzi was called in as an expert to comment on whether these hanging chads counted as holes. Not only that, but if you go to YouTube, there's an absolutely excellent little video where Akile was interviewed. It's from some news channel about holes for this purpose, because Akile and one of his close friends wrote this book called Holes around that time. And it's still an enduring classic of philosophy. and he's also, in addition to the leading meriologist, I mean, one of the world's leading experts on holes, and just in short, I think he has a realist theory about holes in which holes are sort of derivative entities.

33:45They're ontological parasites, so the hole in a donut is real, but it depends on its existence on the donut in which it is a hole. but it's another super fascinating great book yeah I read some of Achille Achille Vartzi's work on muriology of course because as you point out he's like one of if not the leading experts on the subject yeah so I feel it I can't not mention one of the things that I learned from this meeting that we all had with him at the beginning of college he told a philosophy joke. That's one of my favorite jokes. And you probably heard this joke at some point. I don't know if he ever told this to you or if he remembers it.

34:35But it was about, the joke goes like this. A UFO visits a philosophy conference and, you know, the bright light shines down and all the philosophers look up and they hear a voice from the aliens saying earth has been randomly selected to receive the answer to any single question uh that earthlings have but they only get to ask one question and one answer because that's all you get and then they're going to go on and then some other planet some other civilization will get the next question and they say they'll back in one Earth week. And so the philosophers, of course, begin debating among themselves.

35:20Well, we only get to ask one question. What is the most important question we could possibly ask? What if we ask, what's the meaning of life? And it turns out the answer is not interesting, or it's a meaningless question, and we lose our chance. Should we ask about what the nature of being is, or the nature of laws, or what it means to live a good life, or what are the ultimate laws of the universe? What's the unified theory of the universe? Or what are the best mathematical axioms. They're debating and debating and debating. And finally, they realize they're never going to come to an answer. And even if they came to a consensus, they wouldn't know that it was in fact the best question.

35:54And they were worried that what if a century from now we realize there was a better question we should have asked. So they hatch a plan. They decide if they're going to ask, what is the two element set that contains as its first element the best possible question we could ask and contains as its second element the answer to that question and they feel very proud of themselves they come up with this question two days before the deadline they decide they're just gonna you know have fun and party with the way philosophers like to party i guess and then you know on cue the alien you know spaceship returns and the light comes down the chair of the conference is ready the light comes down they hear the voice the aliens ask okay well, you know, now you can ask your question.

36:43What is it? And the chair of the conference takes a deep breath and says, here is our question. What is the two elements set consisting of as its first element, the best possible question we could ask, and in its second element, the answer to that best possible question? And immediately they hear from the aliens, excellent what an excellent question here it is in the first set is what is the two element set containing the best possible question is the first element and the answer that question of the second element and the answer in the second element is the question what is the best possible question that you could ask and what is its answer and with that the aliens take off and fly away.

37:34That's very good. I hadn't heard that one before, and as you started saying it, I was thinking, you know, this is going to be a great intro segment to the episode, but it is just slightly too long to be the great intro segment. So people will just have to wait until the 40-minute mark to hear the joke. Very good. But no, I'm so glad I asked about Akilah. I didn't connect with Akilah Vartzi beyond that, unfortunately. The truth is, once I got to college and fell in love with math and physics, I just filled my schedule with as many math and physics courses as I possibly could. I had no time to take anything else except one biology course I took near the beginning of college when I was still considering studying the brain.

38:19I just didn't connect enough with biology, and I felt like I had a lot of ground to make up, and so I filled my schedule with math and physics classes. I ended up not taking a number of classes that, in retrospect, I regret not having taken. Well, OK, the last thing that I'm going to ask, maybe this will be a briefer question, but you mentioned scruta and myriads. And I'm just wondering, do you have a background in ancient Greek or you're just interested in etymology as as maybe a human thing or as part of scrutinizing the history? I took a lot of Latin, and in part because I'm really interested in the etymology of, etymology is a general subject.

39:04I think etymology is interesting just because I like facts, but also because knowing the history of something can sometimes give you new ways to think about it. And I'll give you a good example. I mentioned matrix mechanics. Heisenberg's 1925 proposal for how to provide a comprehensive formulation of quantum mechanics. Do you know where the word matrix comes from? For me, it comes from the 1999 film by the same name. Yeah, yes, right. So when people think of matrix, they think of an array of some kind, usually, an array of numbers. But what makes an array of numbers a matrix, it's more than just that it's an array of numbers.

39:51There are also rules for how you're allowed to manipulate them. Matrices can be added together and multiplied. And it's only once you impose those rules that an array becomes a matrix. But the question about where a matrix actually comes from is interesting. And this is an interesting piece of etymology. So I'm not going to put you on the spot. But if you look at the word matrix, you'll see it begins with matri. and matri, matri, like matriarchical, matrilineal, maternal. These words have to do with motherhood. And this is not an accident. When James Joseph Sylvester, the 19th century mathematician, was, I mean, the idea of the use of arrays of numbers for various purposes is thousands of years old.

40:41You find it in Western mathematics and in Eastern mathematics. Chinese mathematicians use similar ideas. but he was trying to use these arrays in a more formal way. In particular, he was studying systems of linear equations and how to solve them. And when you're trying to approach a system of linear equations, the first thing you want to do is decide whether it can be solved at all and whether the solutions are unique. And for this, you take arrays of numbers that show up in these equations and compute quantities, determinants out of them. And he decided to arrange the numbers in these systems of equations into an array and then use this array to extract determinants.

41:21And what he said in his early work was, this is around 1850, 1851, was that he was extracting determinants from these arrays in a manner similar to extracting offspring from a uterus. And the word matrix is the Latin word for uterus, and this is where it comes from. And this is a case in which I think the etymology is actually quite informative about the origin of this concept of matrices. Eventually, Cayley started working on them, and you see a lot of really rapid development very quickly after James Joseph Sylvester introduces it. I don't want to give any spoilers for those who haven't seen the film The Matrix, but I do wonder, in retrospect, thinking about what The Matrix is, whether the Wachowskis, in deciding to make the film, thought at all about the etymology of the word.

42:16And for those people who have seen the film, you'll understand what I mean. So I've always been interested in etymology. In my Latin courses, we did at one point spend some time learning the basics of Greek, but we didn't spend a lot of time on Greek. So I guess it's partly my connection to Latin. It's my interest in etymology. I think history can be incredibly informative. History is not just interesting, But history, well, we'll talk a little bit why I think history is so important, why I'm also interested in history of physics, not just the philosophy of physics. So the historical component is also there.

42:53So, yeah, there are a lot of reasons I'm interested in etymology. Yeah, it's not just a connection to Greek. Okay, sounds good. Well, then, should I ask why you find the history of physics itself so interesting since you just alluded to it? Yeah, I'll tell you why. So when I go to seminar talks, so I mainly work in philosophy, but I do sometimes attend seminar talks in physics, especially in mathematical or theoretical physics, in part because the contemporary work going on right now in physics is great raw material for philosophical analysis. It's a source of ideas. And sometimes when I go to seminars, I see people reinventing the wheel.

43:37I see people struggling to recreate something that has been done before. And if they just knew it had been done before and had studied the history, they wouldn't be spending their time recreating something. They could be building off of what has come before. When you think about what has made humans so successful as creatures on this planet, an individual human being can be a smart person, but it's taken humanity a really, really, really long time, many generations, to get to our present stage of scientific and technological understanding. But we've made it, and we've made it because we've benefited from our ability to scaffold knowledge.

44:18One human learns something and then teaches the next human who teaches the next human. And this incremental passing along of knowledge historically has brought us to where we are. When we don't pay attention to history, it's not just that we're likely to make mistakes that people recognized earlier we shouldn't make, but it also means we're not benefiting from that scaffolding and that makes us less productive and less effective. So I would like to see less reinventing of the wheel. I think that's key. I think another reason why history is important is

44:55sometimes we forget where good ideas come from. Now, I'll give you a good example, right? This is not going to come as a shock to you or anyone listening, but the history of the relationship between philosophy and physics has not always been really great in the past century. There's been a kind of an attitude, and I don't want to say that it falls only on one side. I do think there's some mutuality to this attitude, but an attitude that the subjects don't need each other very much. having come up in physics having done a lot of my training in physics this was something i heard a lot including from some luminaries in physics uh you know i'm not going to name names i'm sure people probably can guess who i'm talking about but very important physicists who've made statements that physics doesn't need philosophy philosophy is dead it's a waste of time when was the last time philosophy did anything useful for physics physicists i've had on the show yeah and i don't want to be mean, but this is just wrong.

45:55It's just misguided. The people who gave us the most important theories in physics we have, from Newtonian mechanics to Maxwell's theory of electromagnetism to thermodynamics and statistical mechanics, quantum mechanics and relativity, were deeply steeped in philosophical thinking. Many of them had thorough philosophical training. And when you read their papers, their papers are written like people who think philosophically. And I don't need to quote people, but I can. Einstein spoke voluminously about this in his obituary to Ernst Mach. Ernst Mach, who himself was both a philosopher and a physicist.

46:37This obituary was from 1916. Einstein spoke at great length about the importance of philosophy and in particular epistemology to doing science well. And there's a very famous letter in 1944 that Einstein wrote to Robert Thornton, who was a philosopher of science, who was complaining about scientists not always taking philosophy seriously. And Einstein wrote back saying, I completely agree with you. And that a deep understanding of philosophy helps scientists avoid succumbing to the prevailing conventional wisdom of their time, gives them the ability to stand above and not get locked into preconceptions and tunnel vision, the tunnel vision, as we would call it now.

47:22He didn't use the word tunnel vision, but tunnel vision that can be ubiquitous in science. what good philosophy can do in any field is seed the soil for later discovery philosophers are usually not interested in the end product the applications of a particular area of inquiry not usually but sometimes but not usually usually they're interested in laying groundworks in softening and seeding soils so that you get a good harvest and you see this throughout the sciences. I mean, computer science is built on logic, right, on symbolic logic, and physics is built on, I mean, physics, it's easy, right?

48:05I mean, on questions about what we can know about the world and about laws of nature and about how objects are made of other objects and about what it means for things to exist physically and about the nature of time. And people who formulated physics thought about all these questions and it informed how they thought about how they did physics. Nowadays, I think some of the strongest and most robust alliances between philosophers and scientists are between epistemologists and statisticians. I think there is an argument that a lot of statistics is, and I mean this in the best possible way, applied epistemology, using the tools of epistemology to try to learn as best we can, as correctly as we can about the world around us in practical, concrete ways.

48:49And I talk to a lot of statisticians these days, and I love talking to statisticians because at least the ones I speak to, they think about philosophical questions and they like talking to philosophers because they're trying to understand very basic questions. We all know that correlation does not imply causation. Well, then what does imply causation and what exactly is causation and how do we find it? Statisticians are very worried about the nature of probability. What is a probability, right? What are the different kinds of probability? How do they work? How do they interact with each other?

49:21How can we learn from probabilistic things and make probabilistic judgments? So I find that the interplay between philosophers and statisticians in particular is robust and ongoing and has informed some of the best work that goes on in statistics. So with all, you know, with immense respect to everybody, I don't want to call anybody out. There's just a really robust relationship between these subjects. I think an attitude that prevailed for a long time in physics in particular. Over the last roughly century, really post-World War II, and also with the center of gravity of physics, no pun intended, shifting somewhat to America, which has a stronger pragmatic attitude toward things.

50:08I think the introduction of large amounts of research funding and the need for technological development in physics, I think pushed people away from thinking that taking the time to think philosophically about things was well advised and that the focus should be on practical development, on producing new technologies, energy technologies and everything. And also, fundamental physics at the time, after World War II, people were finding new particles all over the place. There was so much work to be done to build new models and to calculate things and to compare them with experiment. I just think there was an attitude that like they didn't need philosophy anymore.

50:48They had a huge harvest, right? There were corn stalks as high as the eye could see. There was wheat and there were strawberries. You know, there was plenty to eat. Why do we need to seed the soil again? We have all of the harvest we could possibly need. I think it's easy when you're living within an embarrassment of riches like that to forget about the need to go back and reseed the soil. And if the planter comes along and says, hey, I know you're really busy reaping all of this fantastic harvest, but you should take some time to let me reseed the soil. You could imagine people saying, no, no, no, we have so much, we have to reap everything.

51:27We have no time for this. I think things have changed. At least in fundamental physics, there's no question that we're not seeing the rate of foundational breakthrough discoveries confirmed by experiment that we were 50 years ago. The standard model of particle physics, our best, most accurate theory of the fundamental interactions of elementary particles and fields today, was mostly finished theoretically almost half a century ago. And of course, we've gone on to confirm more predictions of the standard model and to find ingredients we hadn't found before. The top quark was found in the 1990s.

52:05The Higgs boson wasn't officially announced to discover the Higgs boson until 2012. So obviously there was more experiment to be done to confirm the standard model, but we haven't seen any concrete smoking gun we're sure of new ingredients beyond the standard model that we are, we're sure are there. And if you ask, when was the last time a theoretical prediction was made? When was the most recent theoretical prediction of really substantial significance? When was the last one that was made that was then ultimately verified experimentally? So I don't mean the Higgs. Yeah, the Higgs was verified in 2012.

52:48That wasn't that long ago. But of course, the Higgs boson, a crucial ingredient in the standard model related to imbuing many of the particles in the standard model with their intrinsic inertial masses, that idea was proposed many, many decades ago. If you ask physicists today, when was the most recent prediction made, theoretical prediction made, that then went on to be confirmed experimentally, and again, a prediction of real significance, like predicting the Higgs boson, something of that magnitude or greater, it's been a long time. There haven't been a lot of predictions like that in the last few decades that have gone on to be confirmed experimentally.

53:26That's an important proviso. And so when you talk to especially people who are in younger generations in physics, experimentalists and theorists, and I think they're much more open-minded about talking to philosophers. I think there's a greater recognition that foundational work now needs to be done. There needs to be more seeding of soil. So I would say in the last 15, 20 years, there's been a real growth in connections between the fields. I think it's a wonderful thing. And I'm trying my best in my own research work and in my teaching and talking with people and talking with you to try to build bridges between these disciplines.

54:06bridges between physics and philosophy, but also between philosophy and mathematics, philosophy and statistics, philosophy and other areas of science. I think some of the most important ideas come from those interactions, and I think that philosophy plays a central role in a lot of that. I'm hesitant to continue down this line right now, because I would like to get more into the really basic problem of the difference between the classical and the quantum world. But before we go back to that, I would just like to ask, because the standard model of particle physics is so successful. I mean, the number that everybody quotes is, I mean, it's been confirmed to something like 14 decimal places.

54:51So if it is so successful, why is it incomplete? And why do we want to be making more predictions about it? What is it that it's not accounting for if it hasn't failed yet? It's a lovely question. It is really successful. When someone comes along and says, I have a new theory of physics that's better than the standard model, I will sometimes point them to the particle data groups booklet, which you can just get online or maybe still order a physical copy of, which is just a book filled with theoretical, with like numbers, actual numbers, you know, that the standard model can account for. uh it's breathtaking how successful the theory is um it is very successful in a limited domain if you want to know the gyromagnetic ratio for the electron or you want to know decay rates of certain elementary particles or certain scattering cross-section or branching ratios the standard model is what you're going to use and it makes fantastic predictions about those things It is at present not able to account for a lot of other stuff that we have sometimes indirect, sometimes somewhat more direct evidence of.

56:10the visible matter that we're made out of, the kinds of particles in the menu of particles that are described by the standard model, electrons and quarks, neutrinos, and various force carriers, W and Z bosons and photons and so forth, the Higgs boson and all that, these appear to make up, as best we can tell based on our present observations and our present understanding of cosmology, something like 1 20th of all of the mass energy content of the universe, of the observable universe today. A much larger component appears to be other stuff that resembles matter, so stuff that appears to have mass and gravitates and clumps gravitationally, but as best we can tell doesn't interact in any way we've been able to characterize with the other particles in a non-gravitational way.

57:14And that stuff is called dark matter. And dark matter appears to be a much bigger component of the total content of mass energy in our universe. Separately from dark, and I say dark matter is called dark and people think it's like dark, like it's black or something like that. But that's not quite right. It's invisible. It's dark only because in outer space, invisible things look dark. there's dark matter in principle, if dark matter, if the dark matter idea is correct, there's dark matter all around us. I have a can on my desk that says one gallon dark matter. And people say, oh, is that a funny joke?

57:48And I'm like, it's not a joke. If dark matter really is everywhere, there is in fact a gallon of dark matter in there because there's gallon dark matter everywhere. Now, it may not be the same gallon because the dark matter can move through things. It doesn't interact with regular material. So it may not always be the same gallon. And a gallon is a measure of volume, not mass. So by definition, if there's a can that has a volume of one gallon and dark matter is everywhere, then there is a gallon of dark matter in there. separately from dark matter is dark energy. And dark energy is a name that's given to a different thing.

58:20We think a different thing. And it's confusing the two terms, both have darkened them and a lot of people sometimes confuse them. Dark energy is something else. Dark energy is whatever, it's a name, it's a question mark. It's a name for whatever appears to be driving our universe to expand at an accelerating rate, faster and faster with time. And as best we can tell, there are several things that could contribute to dark energy. None of them are properly understood through the standard model any more than dark matter is understood at the standard model. What could contribute to dark energy? There's a certain kind of self interaction of the gravitational field with itself.

59:02An extra interaction of gravity literally interacting with itself. This is called the cosmological constant term in general relativity. That may contribute. We don't know how strong that self-interaction is. It's characterized by a certain constant, we think. We don't know how strong that constant is. There may also be contributions from energetic fluctuations in all the quantum fields of the standard model and maybe whatever quantum fields go beyond it. That's called vacuum energy. and then there may also be background values of various fields there are certain kinds of fields they're called scalar fields the higgs field is an example that can settle into just sort of like a background value where they kind of switched on everywhere in space in a way that that is consistent with the other symmetries of space time that that we have good reason to think are there And somehow these three ingredients and maybe more ingredients we haven't thought of conspire, they add together or cancel, they contribute in various ways to produce this accelerating gravitating effect.

1:00:07We don't understand that either. And that's not something that the standard model allows us to understand. I mentioned gravity. Standard model does not include gravity in a fully quantum theoretic way. Now, you can augment the standard model with gravity in some ways. There are beautiful, effective versions of quantum gravity that work in certain regimes. But we don't have a complete fine-grained, we call it ultraviolet complete. The word ultraviolet is a term of art that just means at super short distance scales, super high energies. there's a certain sort of fundamentality we think of when we use the word ultraviolet in this way.

1:00:43There's no ultraviolet complete theory of quantum gravity that connects with the standard model. The standard model has not helped us understand what's going on in, you know, there's certain mysteries surrounding black holes, certain mysteries about the very early universe. So although the standard model makes highly accurate predictions within its domain of validity, there do appear to be some things outside of it that we don't understand. and I would add something else. The standard model is built on quantum theory and quantum theory has problems. And we can talk about those problems. The standard model doesn't solve the problems of quantum theory.

1:01:17So I think these are all ways in which, despite the immense and very impressive empirical, I wouldn't say adequacy, adequacy is hardly sufficient, not just empirical adequacy, adequacy the level of agreeing with our empirical predictions and our experiments but but like um enormous success i mean empirical success as a standard model i think one has to to be mindful that one has to be um precise and exactly what one means by that it is remarkable to me the difference in views on this topic that i've encountered because so many philosophers like you view the foundations of quantum mechanics as a problem to be worked on.

1:02:08And so many others that I've spoken to don't think it's a problem at all. And it's just so successful. And that's that. And we don't really need to talk about it anymore. Something that Sean Carroll said to me that I found interesting and stuck with me is that One reason he's very interested in the foundations of quantum mechanics is he thinks that it will be very helpful or necessary re-evaluating them, or at least thinking about them, to make progress in quantum gravity. Whereas many of the people I've spoken to who work on quantum gravity, I mean, string theorists, are not interested in foundations at all.

1:02:51They just want to move forward with string theory. And I want to just ask if you think that these foundational questions then are extremely important, not just for quantum gravity, but for the two other problems you mentioned, which are accounting for dark energy and dark matter. And we don't need to get into why. I'm just curious if that's sure. The short answer is I don't know. OK. The short answer is I don't know. What I do know is that a theory of quantum gravity should include, if not quantum theory in it, some successor theory to quantum theory. And that means if we want to ultimately obtain a theory of quantum gravity, we have to understand quantum theory sufficiently well.

1:03:37And I would argue that we don't understand it yet sufficiently well. So whatever successor theory ultimately replaces quantum theory, if it's not quantum theory itself, will presumably be a part of quantum gravity. And so I would argue we don't understand quantum theory sufficiently well to be at the stage where we'd understand how it should fit into a theory of quantum gravity. um so let me just make an analogy here for a second let's take the measurement problem which is the most famous problem in quantum foundations and let's take the black hole information problem so i'm gonna i'm just gonna lay these two problems out because it's a very interesting sociological question why in the history of physics the measurement problem has often been treated as not worthy of a great deal of attention um by physicists at least and why the black hole information problem, at least by some physicists, has been considered an extremely pressing problem that we need to deal with.

1:04:32So they're both problems involving something that could ultimately be a part of a theory of quantum gravity. I mean, one could argue, well, we need to solve the black hole information problem because gravity is ultimately going to be a part of quantum gravity. Well, you know, quantum will also be a part of quantum gravity, or at least some version of quantum theory will be a part of quantum gravity as well they both uh in some sense represent tensions with a certain kind of dynamical principle what's called unitary evolution of form of dynamical time evolution both of them are in tension with this idea both of them have led to wild speculations over the years um neither of them is at least at present particularly amenable to experimental investigation.

1:05:23And one argument that had been made by a lot of people, the measurement problem is, well, the measurement problem doesn't really become a problem until you have big systems, a system with 10 to the 23 particles in it or more. And at that level, it's just not experimentally feasible that we could really probe the measurement problem. Well, let me tell you, however hard the measurement problem is to study experimentally, the black hole information problem is at least, as best we can tell at this point, vastly harder, right? So to experiment, to study the black hole information problem experimentally, what you'd have to do is set up something like, I guess, 10 ,000 identically prepared black holes.

1:06:02All of them far separated in space so that none of them are eating anything. Because when black holes don't eat anything, when there's nothing to matter for them to eat, then according to our leading pictures of what happens when you mix black holes and quantum mechanics, they slowly radiate away their mass. this is called Hawking radiation, very, very, very slowly for a stellar size black hole, a black hole that you get from a star collapsing. It would take many times the current age of the universe for one of these black holes to evaporate fully. You'd have to wait that amount of time and do it for all 10 ,000 or whatever of these black holes.

1:06:36You would have to enclose each black hole in a perfect detector, like surround it completely with a perfect detector and capture every single outgoing particle that came out without had any loss at all, even tiny losses you'd have to avoid. And then you'd have to subject all the data from these experiments to a level of analysis that various people have argued is computationally impossible. I mean, and then you'd be looking for basically subtle differences between various attitudes toward what happens when black holes evaporate. That's how you'd go about testing a model like very you know so there's certainly no argument that black hole information is a more practical question or experimentally accessible question the measurement problem um both arguably have led to a lot of really great spin-off ideas and we could talk about some of those ideas as well especially the measurement problem which has led to a lot of very important ideas with practical use.

1:07:36And I guess finally, the measurement problem at this point has netted a Nobel Prize. The 2022 Nobel Prize in Physics went to Aspe, Clauser, and Zeilinger for their work on tests of the Bell inequality, which are closely connected to the measurement problem. There are as yet no Nobel Prizes for the black hole information problem. Maybe one day there will be. But I think it's just an interesting sociological question why these two problems have been treated so differently in the history of science, in the history of physics. I don't have a good answer for that. But I do think it's something people should think about.

1:08:09I've asked people, and I should just say, I have made it kind of my mission to talk to physicists, especially physicists who don't think that the measurement problem and problems of quantum theory or, more generally, the interaction of philosophy and physics is a good use of people's time. If you know any physicists whom you think might need convincing, send them my way. And this is a general call to everybody. I'm always happy to talk to physicists. And I think I make a really good case. This is something that I spend a lot of my time with. And to be clear, it's not because I want to prove anyone is wrong.

1:08:43It's because I want to increase the interplay between these fields. I think fostering that kind of interdisciplinary dialogue would enrich both fields. It would make both fields better. And that's part of what I see my role is doing. Well, I have really appreciated taking this time to get, I don't know, an introductory sense of your general views about philosophy and mathematics and physics and etymology even. But for our listeners, and also just for you, where I'd like to go from here is, one, to get a grounding in just the tension between the classical and the quantum world. Then maybe talk about these very puzzling quantum phenomena, like the measurement problem.

1:09:35and then from there to discuss the leading theories or interpretations of quantum mechanics and foundations and then to finally hear about the indivisibility approach or account that you've been working on. So with that being said and a little bit more of a roadmap now in front of us, what is the difference between the classical and the quantum world? And I guess that this might be a place where your attention to history might be handy. Yeah. One way to think about a physical. So there's a whole, you know, there are many ways to think about physical theories. Einstein wrote an article something like a century ago describing a dichotomy between principle theories and constructive theories.

1:10:30And there are a lot of people who have a lot of ideas about how to think about theories. Here's how I like to think about physical theories. And arguably, this is more along what Einstein would have called a constructive theory. Constructive theories are physical theories where we put the ingredients in. We put in specific ingredients and give them things to do. Principle theories are a little bit more general. That's where you begin with sort of overarching principles and then ask what kinds of constraints is that put on the kinds of theories you can build. A principle theory might be saying something like nothing that transmits a signal can travel faster than a certain speed, the speed of light in an inertial reference frame.

1:11:07And then any theories you construct would have to then satisfy that condition. So you might call special relativity of which that's one of the principles. The other is the Gowlain principle, the original principle of relativity, suitably generalized to accommodating the constancy of the speed of light. You can think of that as a principle theory, and then all the different possible physical models you could imagine have to satisfy that. A constructive theory is where we really – some people, some of the principle theories are like top-down. You start with the principles and work down. Constructive theory is more bottom-up.

1:11:37We begin by laying out what are the basic ingredients, and we discuss how they work. Here is how I would characterize four minimal ingredients of any constructive physical theory, a physical theory of the kind that I'm interested in. One is something like effectively elementary moving parts. What are the basic moving parts, at least on your given theory? They don't have to be fundamental elementary in some profound metaphysical sense, but just as far as your theory is concerned, what does your theory take to be for its purposes the effectively elementary moving parts? Sometimes we call these the degrees of freedom of your model.

1:12:14And we can think of all the different ways that those moving parts can be arranged. We call them configurations. We call them arrangements. They're just, think of like, I've got a bunch of particles, and I can think of all their different arrangements, snapshots of what they could look like. Or I throw 12 rolling dice, and all the different ways they could land when they roll. Those are the different configurations. or if I'm thinking of a field, like the electric field, all the different patterns of intensities and directionalities in that field, the elementary moving parts are the things that can change, the basic moving parts.

1:12:50They could be the individual particles. They could be the dice. They could be the little values of the field at every point. And then all the possible ways they can be arranged. Those are the configurations, and we call the set of all those possibilities the configuration space. So we start with some kind of set of elementary moving parts. Then to make it physics, we have some effective rules, the effective rules or laws for those moving parts. This is the dynamics. If you think of the moving parts, sometimes people use the word kinematics for the moving parts, the stuff and how it can be arranged.

1:13:22This is the stuff that follows or obeys or is systematized by the second ingredient, the effective laws, the dynamics, the rules for how those ingredients work. Your model should have rules. Rules that describe how the ingredients should move, Newton's second law, force equals mass times acceleration is an example of a dynamical rule like that. Maxwell equations that describe the behavior of electric magnetic fields, those are rules like that. The third ingredient is a kind of epistemology. How do we assign certainties to the ingredients of the model? How do we say how certain we are that our model has this arrangement or that arrangement?

1:14:01And I mean epistemology here broadly to include questions of uncertainty, subjective uncertainty, and maybe also some notion of probabilism in the laws. Maybe the laws are probabilistic and we can only characterize the behavior of our system in some notion of probability. We can talk about probability, which is a deep and really interesting topic with lots of facets later if you'd like. But for now, just take a pedestrian view, some broad notion of epistemology. That's ingredient three. So ingredient one was the elementary moving parts, the kinematics. Ingredient two is the laws. That's the dynamics.

1:14:35Ingredient three is the epistemology. And then ingredient four is the hermeneutics. Like we take this stuff and we have to interpret it. Hermeneutics is a bad interpretation. What do the things mean? I mean, we have a bunch of ingredients, maybe they're represented symbolically, linguistically. We use mathematical variables for them. We use equations. But how do we translate the mathematical structure of this model into things we're saying are part of the real world? I think the hermeneutical part in particular is often not sufficiently appreciated. I've never heard the word hermeneutics even used in this context.

1:15:13As opposed to the analysis of texts, yes. Right. Yeah. But I think it is like reading a text. Certainly if you think of mathematics as a kind of language, then this is a kind of hermeneutics in a pretty straightforward sense. And when you're not mindful enough about the hermeneutics, you get problems. I call this, broadly speaking, the connection problem or connection problems. You prove some theorem about your model and you believe you've proved some general fact about nature. But if something has gone wrong with a hermeneutical connection between your mathematical structures of your model and physical reality, then even if your proof is a deductively sound proof, it may tell you much less than you were hoping about nature.

1:15:53Because there could be a problem with the connection, the hermeneutical connection between the mathematical structure of your theory and the world. So these things all deserve a lot of attention. We've talked a lot about Mariology, the part-whole relation, from the Greek word meros, which means part. um akila would actually tell you it is the part hood relation the part whole relation is mario topology sure sure sure yeah i i stand corrected thank you very much yes um but i think one should also uh uh you know when i i just sort of like blithely laid out all those ingredients but there's further um uh there's further work to be done you have to make sure the ingredients make sense.

1:16:38And I don't mean make sense in the sense that they're intuitive. A lot of physics is not intuitive. Black holes are not intuitive. Superconductors are not intuitive. Gyroscopes are not intuitive. A rotational motion is where human intuition goes to die. Special relativity is super weird and not intuitive. When I say makes sense, I mean self-consistency. You have to make sure that your model is self-consistent in various ways. One form of self-consistency is just basic mathematical self-consistency. There should be ambiguities or contradictions in the assumptions or premises or axioms of your model.

1:17:09But also your model should make sense in a myriological sense that I think is extremely important and I want to come back to. I'll come back to it shortly because it's actually relevant to the history. Okay, so those are the ingredients I think one needs to think seriously about when thinking of a model. And I really can't think of any models that don't have those ingredients. Every physical theory I know of, I mean, principle theories, yes, don't always specify all those ingredients. If I just say special relativity is the laws, the basic laws of physics should be the same in every inertial frame of motion.

1:17:40And in every inertial frame of motion, there is this particular speed at which signals at fastest can travel. That doesn't fit all those requirements. But that doesn't also pick out a single physical theory. When you're talking about a single physical theory, maybe in the constructive sense, I really think you need those four aspects at a minimum. I think those are necessary, if maybe not necessarily sufficient. Okay, now let's talk about classical physics. So classical physics is pretty clear about those ingredients. Let's take Newton's theory, Newton's theory of classical physics. In Newton's theory, the elementary moving parts are bodies in space.

1:18:17And to be somewhat more precise, you could take the elementary moving parts to be point-like myriological atoms. I say myriological atoms because I'm not saying these are literally the kinds of atoms we talked about today, but in In myriology, the smallest parts in the parthood relationship, like if you've imagined a tower of things belonging to things, if there's a bottom of this tower, those are the myriological atoms. And in Newton's laws, the myriological atoms are point-like masses. They're point-like entities that have a location, and they have an inertial mass, a property called inertial mass.

1:18:54And you can put them together to make bigger things. and bigger composites are the myriological sums of their myriological atoms. And I say is, but you have to be careful because you can certainly ask questions. Is the composite nothing more than the myriological union of its myriological atoms? Those are subtle and important questions that maybe we don't need to talk about right now unless you want to talk about them. But there's some sense in which we can talk about things being made of other things. the arrangements of all these mirror logical atoms these basic particles these newtonian particles that exhausts the elementary moving parts it exhausts the kinematics of the theory and i'll make a hermeneutical statement right now which is that newton's theory says that the ontology of nature is exhausted by these mirror logical atoms once you know where all the atoms are then you kind of have the ontology you have what physically exists Just for my clarification, why is that a hermeneutical statement?

1:19:58Oh, I apologize. Let me step back. You're completely right. If I describe those particles in mathematical terms, if I say that I've got a bunch of points in three-dimensional mathematical Euclidean space that I'm labeling with numbers, then there's a hermeneutical statement you have to make to get from that mathematical structure to the statement that, oh, I'm describing bodies in space. I collapsed those two things, which is a thing I really don't like doing, and I should have done it. But I collapsed those things. On the one hand, we have the mathematics of Newtonian mechanics, and then we have to take the elementary degrees of freedom, these point-like mathematical entities in Newtonian mechanics, and then give them a hermeneutical lead as mirror logical atoms, particles in space that can make bigger things.

1:20:47That's the hermeneutical leap, and it's important to be very clear on that. So thank you. That's a good point. Okay, and then we can put them together to make bigger things with bigger masses. The second ingredient, the dynamics, the nomology, I should have introduced that word, nomology is the laws of the theory from nomos, which is laws. The dynamical laws, that's Newton's laws of motion. Newton's laws of motion, object at rest stays at rest unless acted upon with an external force or moves at constant rectilinear velocity unless acted upon by an external force. And then that's Newton's first law, Newton's second law.

1:21:23When an object is acted upon by an external force, it experiences an acceleration that's directly proportional in the same direction as the net external force, an inverse proportional to the inertial mass of the object. And those are the first two Newton's laws. I'll come to the third law in a moment. Those laws give us dynamical rules. They tell us how the moving parts are supposed to move. And that's really important, and you need it. Without it, you don't have the rules for how the system is supposed to behave. Now, with these two ingredients, you end up with the kind of story that you'll hear from, it's usually attributed to Laplace.

1:22:00This Laplace, this idea that an intelligence that knew where all the particles of the universe were and knew all of the laws of physics and could calculate with no limitation to this. Now we would call it Laplace's demon, this this superior intellect. You know, nothing in the future or the past would be hidden from it. It could it could know the future and know the past. Tim Maudlin pointed out to me that Laplace was not the first to say this. And in fact, the way that Laplace described it was not exactly correct because Laplace didn't mention velocities, which you also need as part of your characterization of the state of the system.

1:22:37You can't make predictions only with positions. You need velocities as well. That also turns out to be very important, which we'll come back to in a moment. Apparently, Boscovich, who predated Laplace, had stated basically the same idea and had included the necessary additional condition that you need to know where all the particles are and you have to know how they're moving at an instant in time in order to then use the laws to make the predictions. Okay. I'm going to come back to velocities in a moment because it turns out that's actually very important. Now let's go to Newton's third law. What is Newton's third law doing in the theory?

1:23:09Newton's third law says that, well, for every action, there is an equal and opposite reaction. That's how it's usually phrased. but somewhat more formally, it says that if one body exerts a force of a particular kind on another body, then the other body exerts a force of the same kind, gravitational to gravitational, electric to electric, whatever the kind of force is, the same kind, back on the first body at the same time with the same magnitude, the same strength, but in the opposite direction. Now, this is usually phrased as two bodies, but you could think of this as a myriological self-consistency condition.

1:23:49What do I mean? Suppose I consider a finite-sized body, and I want to apply Newton's laws to it. And in practice, this is what we do. I mean, most of the bodies we actually encounter and work with around us are finite-sized bodies. They're rocks and boulders. They're cars and bridges and people and planets. how do we apply newton's laws to these things well to apply newton's laws we need to figure out the net force on this body how do we get the net force well an implicit assumption of newton's laws is that you get the net force in a body by vector adding together all of the forces on the constituent pieces of it fine um and then what do you do once you have these net forces If you have an extended body, you want to talk about its motion, but the motion of what part of it?

1:24:43What if the body is not rigid? What if it's kind of like a deformable body whose shape can change? Now the different parts of it can move differently. What if it's rotating? Now the different parts of it are moving at different velocities. what we need is some statement about how to go from the pieces or parts of a body to the to the whole body to the composite body and you have to be careful in how you do that if i apply newton's laws if i think of a body as made up of three bodies three smaller bodies stuck together i could try to apply newton's laws to the entire composite body as one thing or I could imagine applying Newton's laws to the three smaller bodies that I'm taking to compose the bigger body in a muriological sense.

1:25:30If I apply Newton's laws to the three bodies that make up the bigger body, do I get the same predictions as if I apply Newton's laws to the whole composite body? Now, you might say, well, of course you have to. I mean, it shouldn't matter whether in your mind you think of it as one body or think of it as three bodies. It couldn't possibly make a difference. Well, that's right. It shouldn't make a difference. But Newton's theory is a mathematical theory, and the theory better deliver on that requirement. It better be the case that if I apply the mathematics of Newton's laws to the whole body thinking of it as one body or to the three individual smaller bodies that make it up or to the mere logical atoms, if I think of it as made of lots of points, wherever I apply Newton's laws, I should get consistent results.

1:26:14And it turns out you don't if you don't have Newton's third law. Well, I keep talking about net external force. Why do I keep saying that? Because a crucial consequence of Newton's third law is that internal forces cancel. They cancel perfectly. So that when you want to know the behavior of the composite body, you don't have to worry about all the internal forces. You can go back to just asking about the external force. We can ask about the external forces of the pieces, or we can ask about the external forces on the composite body, and we get consistent predictions precisely because the internal forces between the pieces cancel.

1:26:45And they cancel in pairs precisely because of Newton's third law. So I've had a number of conversations with my colleague at Boston University, Miguel Onisorga, who works in philosophy of physics and history of physics and has been very interested in the history of Newton and Newton's work and Newton's Principia. And he told me recently that Newton was very concerned about these questions of myriological self-consistency, that many of the things that today we just take as useful calculational mechanisms, Newton introduced them because he was very concerned about whether his theory made consistent predictions when you went up or down a myriological hierarchy.

1:27:23hierarchy that is you went down to sub components of a system and applied newton's laws to those or you went up the mirror logical hierarchy to composites and applied newton's laws to those and he was worried that the theory would make consistent predictions as you went up and down they do and that's a remarkably non-trivial condition you have to go and check it and make sure that in fact it works if it didn't work you would know there was something wrong with newton's theory even before you picked up uh an experiment right so that's what i mean when i say that beyond just specifying these like four basic requirements of a constructive theory, you also need to check for self-consistency.

1:28:00And one of the forms of self-consistency is myriological self-consistency. Classical physics has a beautiful myriological structure. Again, take Newton's theory. I can think of a body as a whole and study how its center of mass moves. Its center of mass moves just like if all the mass were concentrated there for some questions. subject to the next general force. You have to take into account rotations and torque and stuff. But there's like a beautiful way to go up and down the hierarchy. If you want to talk about a subcomponent of a system, you can talk about the forces there. If you want to talk about the composite system, there's a rule for vector adding the forces together.

1:28:34In other words, we can talk about muriology at the level of the physical objects, the moving parts. We can talk about a big body made of smaller bodies. We can go up and down that hierarchy. We can think of a big body as made of smaller bodies or smaller bodies making big bodies. We can also talk myriologically about laws too. I can talk about the laws of the subcomponents and I can talk about the laws of the whole thing. I can go up and down. I can vector add my forces together and get the laws of the whole thing. And I can also decompose the forces on the whole thing and understand them as being made of vectors down below.

1:29:08And of course, if you want to predict how a body will behave, you may have to figure out the forces at every instant in time. But once you have the forces at one instant in time, you can then get the next instant in time. This is all super non-trivial. And I think not enough attention has been paid to this very impressive myriological structure in Newton's laws. So what do we have here? We have a classical theory. There are mathematical descriptions of things, elementary moving parts, with a hermeneutical read as corresponding to configurations of particles or bodies in space. That's the kinematics and the ontology.

1:29:44Secondly, we have the dynamics, the nomology, the laws that describe how those things are supposed to behave at the level of the model. I did mention probability, but of course we can characterize our uncertainty about where particles are using the language of probability. If we've got chaotic systems where we have systems that are so complicated that we can't predict exactly how they'll evolve, we may use probabilities to characterize where they'll go, even if we think that Newton's laws are, in most circumstances, deterministic. And then finally, we have a hermeneutical read of all of this. And you see a very similar structure in other pre-quantum theories, classical theories.

1:30:21So take Maxwell's theory of electromagnetism. In this case, the myriological atoms are not particles. They are values of fields at points in space, physical space. Those are the myriological atoms. Mathematically, we describe these with vector fields. That's the mathematics. And we have this hermeneutical read that these mathematical vector fields, the assignment of arrows to every point in a mathematical three-dimensional space, These correspond to something going on in the world, field intensities or something happening out in the world. We've got laws, the laws of the Maxwell's equations for classical electromagnetism.

1:30:53We can talk about probabilities if we want, if we are uncertain about the arrangements of fields or how they'll evolve. And then, of course, we have the hermeneutical read of everything. These systems have configurations that change with time according to rules, configurations that change with time in space according to rules. And I think broadly speaking, this is the classical world picture. The story gets a little more intricate when you get to general relativity, which was written down in essentially its final form with Einstein's field equations in November of 1915. Although arguably maybe David Hilbert got a little ahead of Einstein with his work, but roughly around that time.

1:31:34and general relativity is a little bit more subtle because space and time are blended together in such an intimate way that it's a little difficult to say exactly what the configurations are in this theory of course we have a mathematical representation of things we represent various tensor fields on space time that's the mathematics we have laws the einstein field equation and various laws that are connected to it or derivable from it arguably we don't have a good way to talk about probability and general relativity, which is something I'll come back to. That's a hole in the theory. And there are a lot of hermeneutical questions.

1:32:10There are raging debates right now in philosophy, the metaphysics of space-time, about what is the right hermeneutical read of what's going on in general relativity. What do these tensor fuels mean? Is space-time something we should take seriously as an ontological property of, an ontological ingredient to reality. I would say that the probability and hermeneutical parts of general relativity are deeply confusing, and we don't have a consensus as to what they mean. And I will say one other area where I think there's been a lot of beautiful work between philosophers and physicists and mathematicians has been in general relativity.

1:32:51Here at Harvard, we have the Black Hole Initiative, which is the first, and I think maybe still the only, institute devoted to the study of black holes. And from the beginning, it was intended to be interdisciplinary with physics, mathematics, astronomy, history, and philosophy all integrated together. And we have seminars, a weekly seminar that is primarily devoted to philosophical questions and general relativity. And people really talk to each other. And I think it's really great. So I think that's a case where you really begin to see some problems with this classical paradigm. So, but broadly speaking, that's how I would characterize the classical case.

1:33:30Now, let me say a couple of quick important things about this. We see an important role for hermeneutics. We see an important role for probability. The laws of these classical theories are by and large, don't involve probabilities in them at a fundamental level. Newton's laws are not inherently probabilistic. Maxwell equation, the laws of Maxwell theory are not inherently probabilistic. The laws of general relativity are not inherently probabilistic. There are no probabilities embedded into the statement of laws themselves. We use probabilities because we simply can't characterize everything or we're epistemically limited in how well we know the initial conditions or something like that.

1:34:10The other thing to say, and this is coming back to velocity, the laws of these theories are generally Markov-type laws, or at least semi-Markov. I mean something very specific by this. They're all laws of the form. You give me the snapshot now, and then the laws will tell me what comes next. I don't need to know anything about the past, provided I have a sufficiently fine-grained, precise characterization of what's happening right now. And this is Laplace's characterization or Boscovich's characterization of classical physics. If a super intelligent demon knows with sufficient accuracy, with sufficient precision, where every single particle is and what it's doing at one snapshot at a time, then you don't need to know the past.

1:35:03In fact, you can use the laws to predict not only the future, but also retrodict the complete past. So Markovian laws are deterministic laws? They need not be deterministic. In this case, they are. You get a precise, unique characterization about what will come and what has come. But you could imagine probabilistic or stochastic Markov laws where the best you can do is make a prediction about the probability things will happen. But once you know everything about the present state, the present snapshot, knowing more about the past will tell you nothing. You don't need to know anything about the past.

1:35:43Knowing about the past will not make your probabilistic predictions any better. So these are Markov laws. I say the word stochastic. This comes from the Greek word stokos, which originally comes from the stick that would be in the ground and that archers would aim at. Like it was a target for arrows, for like shooting arrows, right? Stochastic has now come to mean aiming or guessing or in modern parlance, a kind of dynamical probability, chanciness in how things behave. We say that a thing that behaves in a probabilistic or chancy way, here I'm using probability in its aleatory sense, meaning in a sense of like objective chanciness, as opposed to subjective.

1:36:29Like I don't know what's going to happen. That's subjective probability. That's credence. We call that credence my belief. Here I mean more in the objective chancy side. We call those stochastic phenomena or stochastic processes. And stochastic processes can be Markov also, right? It could be that once you know as much as you can, once you know sufficiently much about the present snapshot, then you can make predictions either deterministically, if you have deterministic laws, or probabilistically, if you have stochastic laws. And the past is irrelevant to those predictions. The past only makes a difference insofar as that difference is mediated through the present.

1:37:10Once you know the present, the past has done its job. Now, I said semi-Markovian a few times. This is because Newton's laws are not strictly Markov. I mentioned this discrepancy between Laplace's depiction of Newton's laws. You just need to know where all the particles are and the laws of physics, and you can make the predictions. And Boscovich's earlier but more correct statement that you need the positions and velocities, it is curious that you need more than just the positions of all the particles. You need to know also their velocities. If you need to know their positions and their velocities, what you're basically saying is you need to know all their positions right now, and you need to know all their positions kind of an infinitesimal moment earlier.

1:37:49Because knowing where things are right now and their momentary velocities is the same thing as knowing where they are right now and where they were just a moment earlier. They're equivalent information. And Newton's laws say that you can only predict the future definitively if you have not just where they are right now, but where they were an infinitesimal moment earlier, or equivalently where they are now their velocities. And this is more than a Markov description. It's semi-Markovian. You need a little bit of additional information about the near past in order to predict what's to come. We usually put this aside because we just think of the state of the system, the state of the particles as their positions and their velocities right now.

1:38:28But the idea of an instantaneous velocity is a little bit weird. What does it mean to have an instantaneous velocity? In calculus, We talk about this, but it's a little metaphysically weird. Really, what we're saying is you need two pieces of information, where things are now, where they are just a moment ago. And with this, you can then predict the future. Now, of course, we do use velocities, which makes it feel like it's a Markov description. There's an alternative way to formulate Newtonian mechanics called the Hamiltonian formulation, which we really do. We work with this generalization of configurations called points and phase space.

1:39:01And then we can really write down Newton's laws in a way that look really Markovian. When Newton's laws are written down in their original form before making that change of formulation to the Hamiltonian formulation, they are what we call second-order equations of motion, second-order differential equations, second-order in time, which is just a statement that they're not quite Markovian. You need a little bit of past information. When you reformulate them in the Hamiltonian formulation, they look first-order in time. They're first-order differential equations in time, and this makes it look like the theory is Markov.

1:39:31Ultimately, none of this really matters. The point is that our laws of physics up until now are mostly Markov or semi-Markov. This is a reigning paradigm, that laws of nature, to be good laws of nature, dynamical rules should be Markov or at least semi-Markov. But why? Why should they be? The only argument I can think of, a couple of arguments, One is because it's worked so well so far, pre-quantum theories, classical physics has done great with Markov-type laws or semi-Markov-type laws at least. And the second is that once you open the door to generic non-Markovianity, it just feels like it's just too big now.

1:40:18The number of possible kinds of laws you could write down is so voluminous. and a very non-Markovian law seems like something that would be impossible to characterize. You'd have to supply so much information in order to specify laws once you open the door to non-Markovianity that it just felt like maybe at this point theories like this would not be predictive. You would need to put so much information in the beginning to specify what you mean by the laws that the input-output ratio, the input in constructing the theory would be so big compared with the output of what predictions you could get that it would be maybe a waste of time.

1:40:51I'm not exactly sure why people didn't consider non-Markovian laws very much. It's possible. It was just we couldn't imagine them. I mean, so much of the history of science was built out of Markovian laws that I think people took for granted that that's what a law was. And the theory of stochastic processes, which is really where you begin to talk about words like Markov and non-Markov, this really didn't show up until the last half century. So even like being broad-minded enough to think about the idea of laws that could be different in this way, I think just wasn't available and certainly wasn't available at the time when quantum theory was being developed in the first half of the 20th century.

1:41:28So I just think that people just lacked the terminology and the concepts to be able to think more broadly about laws. So what does this mean for quantum theory? People couldn't make quantum theory work in the early days based on the kinds of heuristic formulas they were writing down. So from the period of about 1900 to 1923, 24, 25, this is now called the period of the old quantum theory. And this was a time when people proposed all kinds of heuristic equations and formulas. Max Planck proposed that you could only energetically excite black body chambers in thermal contact with electromagnetic radiation in quantized amounts.

1:42:04This was his quantum hypothesis. He won the Nobel Prize in physics 1918 for this. and then Einstein proposed that light could also be thought of as coming in quanta, now called photons. That was 1905 as part of the photoelectric effect. He won the Nobel Prize in 1921 for that and not for his work on relativity. And then Niels Bohr comes along with his model of the hydrogen atom that we still often see in maybe high school classrooms. Electrons go in these circular orbits, but they can only jump from one orbit to the other. When they jump from one orbit to the other, the difference in energy is emitted or absorbed as a photon, and this explains why we get certain lines of light when we look at light through a diffraction grating.

1:42:47He won the Nobel Prize for that in 1922. Then you have the Bohr-Sommerfeld quantization rules in 1915-16, which generalized the kind of reasoning that Bohr was doing. And you just have all these heuristic formulas people are trying out, and they don't quite fit together. People couldn't find the kind of constructive picture I was telling you about. They couldn't figure out what the elementary moving parts were, combined with the elementary laws for those moving parts, to generate empirical predictions that agreed with what we were seeing. The rules of the old quantum theory did not rise to the level of a full constructive theory.

1:43:25In 1924, Louis de Broglie introduces his wave-particle duality. he says that just as Einstein has associated waves of light with particles de Broglie starts associating particles of matter with their own kinds of waves and in de Broglie's picture the waves have a wavelength, they're given by formulas that you can look up in the book and his original idea was that like every particle had a little wave riding along with it or was riding along its wave or some people might argue that it was only the waves not the particles but the idea was you had all these little wavelets flying around in physical space physical 3D space, little wavelets flying around and doing things.

1:44:03And with these waves, he could account in a more fundamental way for some of the heuristic rules that had come before it. This became his dissertation thesis. He ended up winning the Nobel Prize in 1929 for that work in his dissertation. So yeah, that's a good dissertation when you win a Nobel Prize for it. But then you start to see people begin to question whether any of this is going to work. It just seems like it's been decades at this point. People can't fit all of this together into a constructive model. And people begin to question whether we should be trying to start from a world picture at all, an ontology, things in physical space moving around.

1:44:48People like Wolfgang Pali, Niels Bohr began to question these ideas. And then in Heisenberg's Matrix Mechanics paper, his 1925 paper, spring of 1925, he just says at the beginning of the paper that he's abandoning the world pictures. No more electrons moving in semi-classical orbits. We can't directly see them anyway. We should phrase scientific theories solely in terms of numerical quantities that in principle we can observe. And this is a philosophy of science move. It is a deliberate move. It's a change in philosophical orientation. And he introduces a mathematical apparatus. This mathematical apparatus consists of these matrices.

1:45:27They were infinite by infinite matrices. And with these, he was able to make empirical predictions about energy levels in various systems that worked. he independently reinvented the rules of matrices which he later in his career said he hadn't actually learned up until that point so he both came up with the first complete characterization of quantum theory in some almost modern sense while also reinventing rediscovering matrices so another good example of why history is so important he could have saved himself a lot of work yeah yeah yeah it was max born a better story this way that's right it was max born who had trained in pure mathematics had been a lecturer assistant of david hilbert who identified the things that Heisenberg was using as matrices.

1:46:09And him and Pasquale Jorgen and Heisenberg, they formulated matrix mechanics in 1925. But this theory still didn't have elementary moving parts. It wasn't clear what the elementary moving parts were. There was no dynamics in it. This theory did talk about, I guess, atoms. I guess to some extent there was something vaguely like moving parts, but not a clear hermeneutics of what they were. And when these moving parts changed, there were no rules for why or when they should change. Atoms would transition from one energy level to another, and Heisenberg's theory could predict the radiation you'd see, what frequency of the radiation would see that would come out, but it couldn't tell you when they would transition or why or what the rules were.

1:46:50It was missing dynamics. It was still incomplete in this way. In 1926, Schrodinger comes along, inspired by a bunch of things, DuBois' wave-particle duality and also earlier work done decades before by Hamilton, William Rowan Hamilton, and Carl Gustav Jacob Jacobi, very delightfully redundantly named Jacob Jacobi, Carl Gustav Jacob Jacobi. Back in the 19th century, they'd formulated a different way to do classical physics using these abstract ingredients that lived in abstract high-dimensional spaces called Hamilton-Jacobi functions. And Schrodinger realized that you could take their work and inspired by de Broglie's work, construct something more like a constructive model.

1:47:33This is a model in which there are, in fact, configurations of something. There are elementary moving parts. The elementary moving parts were made up of his wave function, Schrodinger's wave function. This became the thing, the object of the theory that could be arranged in different ways. It could have different patterns in it. finally there was a thing, elementary moving parts. There was a kinematics now. And he also had a dynamics as well. He had an equation that described how this wave function should change with time. And this equation is now called the Schrodinger equation. Now it's considered to be part of a larger idea unitary time evolution.

1:48:18Now he didn't have anything like probability yet. and the hermeneutics was very unclear. What was the interpretation of this mathematical wave function? Schrodinger knew from the beginning that unlike de Broglie's waves, which lived in physical space, Schrodinger's wave function did not live in physical space. It was an abstract mathematical entity in a high dimensional space, the space of possibilities, the space of all the configurations your system might, if you were thinking classically, have had. If you've got, if you're trying to describe the quantum mechanics of three particles that live in 3D space, you need nine numbers to characterize where those three particles are in 3D space.

1:48:58So their configuration space is a nine-dimensional space, and that configuration space is the space where Schrodinger's wave functions lived. So you see the beginnings of a constructive theory, but you're still missing some crucial ingredients. You have something like snapshots of something, the wave function. You've got dynamical laws, the Schrodinger equation. you don't yet have a notion of how probability is supposed to work. And you also don't have a clear hermeneutics yet. You don't know how to read these things. And Schrodinger was very concerned with these questions. He thought about epistemology, he wrote about it.

1:49:32And in those initial papers, he tried to find some hermeneutical lead of the wave function, that somehow you're supposed to project it down into 3D space, and it was supposed to be related somehow to distributions of charge density, but he wasn't sure. That said, for a couple of years at least, 1926 to 1928 at least, Schrodinger maintained that maybe this was what the ontology of nature was. That maybe we should just read somehow this wave function in this high dimensional space in an ontological way. that maybe reality really was configuration space and the snapshot of reality in some ontological sense really was a giant wave function of everything in this configuration space.

1:50:11He tried. Today, this view is called wave function realism. Yeah, I saw you giving some talk. I don't think you didn't use that term back then. Is the wave function real lately? Yeah, exactly. Maybe all that exists really is this wave function in some sense, but he wasn't quite sure. What we do know is that Heisenberg didn't like the idea and we know that Einstein also didn't like the idea. And Einstein wrote a number of letters complaining about this idea that reality should be a giant wave in a high dimensional Q space. Q space because the letter Q, symbol Q, is often used to label the coordinates of these abstract spaces.

1:50:45Einstein wrote a number of letters to Lorenz, to Ehrenfest, saying Schrodinger's work definitely seems like it could be on the right track. But what do we make of the statement that reality should be a giant wave in a configuration space? That doesn't make any sense. and then famously in the letter, December 4th, 1926, letter to Max Born. This famous letter where Einstein says, you know, quantum mechanics is certainly imposing, but I don't think it's the real Jacob. He says the real Jacob. He means like the real McCoy, the real thing. I like the fact that he says the real Jacob. This was apparently in reference to a magazine that was popular back in his time called The Real Jacob.

1:51:24You know, I don't think it brings us any closer to the old one, to God. I, for one, don't think that he throws dice. This is his famous dice letter. But the very next sentence in the letter is Einstein then complaining about Schrodinger's waves in abstract, n-dimensional, high-n-dimensional space. He said he didn't understand what that could mean. Interestingly, that letter was mistranslated in the canonical translations. If you look up the Max Born letters and find that letter, you'll see the n is missing. And it says Einstein is complaining about waves in three dimensional space, which seems like a very weird thing that for Einstein to complain about.

1:52:00But the original German, I know this because of a paper by Don Howard, who's done a lot of, he's an incredible historian of physics. He, he retranslated the letter and that N is missing. Einstein was complaining about three N dimensional space for N particles. Each particle is three coordinates. It's a three N dimensional space where the waves would have to live. And Einstein didn't like this idea. By 1928. Big error. Hmm? Big error. Big is a super big error. Yeah. It just misses the point of what Einstein's criticism was. By 1928, Schrodinger had recanted this view. In his fourth lecture on wave mechanics, 1928, in section 15, the interpretation of the generalized psi function, he says this was his view for a while, that nature really was a giant wave in some abstract space, even though it was a little murky, their hermeneutics was murky.

1:52:51and he says something really interesting. He says, you know, maybe one way to think of it, if you were to think in those terms, is to the extent that the mechanical picture, the picture of particles could still be useful at all, maybe it means that somehow the particles are living out all the things they could possibly do in some arrangements stronger than in others. So he has this proto-many-worlds, Hugh Everett-style view that you already see in this lecture, that maybe one views the wave function as all there is, and this is somehow telling us that your mechanical system, your particles, if there really are particles out there, are living out every possibility, but in some possible ways more strongly than others.

1:53:36But then he says he recanted that view because in 1926, Max Born came along and said, I'm going to give you the epistemology now. The wave function is really a vehicle for calculating measurement probabilities. You do a certain operation to the wave function. You take how intense it is. You do a thing called mod squaring. It outcomes a probability for if you were to measure the configuration of the system, this is the probability with which you get the answer. So Max Forn introduced probabilities into the theory. Now we have a notion of being able to talk about probability and whether this is subjective probability or aleatory, objective, chancy probability is now still something people argue about today.

1:54:13But he injected something like probability into the theory, and this diminished in Schrodinger's eyes the idea the wave function was itself ontological. If it had this probabilistic meaning, then how could it also be ontological? Schrodinger eventually stepped away from his view, at least in this lecture, that the wave function was a fundamental physical object. Now, there is a crucial thing to say about how Schrodinger did all of this to begin with. The Schrodinger equation is a Markov law. it's a Markov law. It's expressible as a differential equation that is first order in time. You tell me the wave function at one time, and then the Schrodinger equation will tell me how it will evolve.

1:54:53In some ways, it's even more Markov than Newton's laws because you only need to know the snapshot of the wave function. You don't need to know it's past. Now, there's some interesting discussions about the wave function being complex valued. If we think of it as being two real valued things because a complex number is really a real number and another real number. In Schrodinger's equation, the two real parts of the wave function are interconnected, and you can actually rewrite the Schrodinger equation in some sense as a second order equation for just one real part. So there's some ongoing question about how first order Schrodinger's equation really is, but at the very least, it is in some sense Markov, or at least semi-Markov.

1:55:34I don't think that was an accident. Schrodinger's search for laws was within the paradigm of the laws that had been understood up to that point. Laws should be Markov. They couldn't find Markov laws for quantum mechanics. They couldn't find a world picture consisting of element or moving parts with a clear hermeneutical read in terms of ontology in space, physical space, with a set of laws that would produce the correct empirical predictions that would give you an empirically adequate description. So Schrodinger introduced new Eleanor moving parts, his wave function, so that he could provide a Markov-type law for them.

1:56:12He was looking for Markov laws, and eventually he found them. My view is that although this was practically speaking useful, I mean today we use Schrodinger wave functions all the time, we use them to calculate things, we use the Schrodinger equation, practically they're quite useful, that this wasn't the only thing we could have done. if we'd been open-minded back then to laws that could be non-Markovian could we have found relatively simple non-Markovian laws married to a relatively simple world picture of elementary moving parts of a more familiar kind particles or fields or whatever it is you want to model could we have kept a nice clear world picture in physical space if we had simply allowed ourselves a more general kind of law, laws that didn't have to be Markov?

1:57:05Arguably, we can. And what I've argued is that one can do this and one gets a non-Markovian picture, a theory with non-Markovian laws. From this standpoint, now we can understand, I think, better what the wave function is. So when people first arrive at quantum mechanics, they'll say, what is the wave function? Well, certainly to begin talking about the interpretation of quantum theory, first you have to talk about the wave function, and then you have to come to some statement. Do you believe the wave function is just knowledge? Is it just a mathematical ingredient? Is it a physical object? What is it?

1:57:36From the perspective of thinking of quantum theory as ultimately not a Markov theory, what the wave function is doing is serving as what's called a latent variable in what's called a hidden Markov model. So among people who do, who study stochastic processes today, people who work with the statistical behavior of systems. When they need to model a system that's not Markov, a system that's non-Markovian, for some models, you can augment your model with a new extra variable that's not observable, that's not physical. It's make-believe. You just add it to your model. And if you add it to your model, you make your snapshots, your configurations, your moving parts look more complicated.

1:58:25But the benefit is that you can make your laws simpler. You can make them Markov. If you can take a model with non-Markovian laws and by introducing an unobservable, unphysical, latent extra variable to make your model effectively Markov, we then call your model a hidden Markov model. From the point of view of thinking of quantum theory, as ultimately fundamentally not a Markovian theory, the wave function, Schrodinger's wave function, is a latent variable. And what Schrodinger had constructed was a hidden Markov model. And then the goal is not to understand or interpret the wave function or treat it as ontology or give it a hermeneutical read.

1:59:10The wave function really is just a pure piece of mathematics. It's a mathematical appurtenance, not unlike the Hamilton-Jacobe function that Hamilton and Jacoby introduced to find another way to characterize classical mechanics, or like a Lagrangian, or like a Hamiltonian, or like many of the mathematical ingredients that we include to give us a more effective, easier way to handle, to do calculations in a physical theory. In fact, Schrodinger's wave function was just built out of the Hamilton-Jacobe function. Sorry. It seems like a very parsimonious ontological picture for somebody who is wary of the wave function.

1:59:47Right. Right. It just sidesteps the question. It says that we were asking the wrong question. Now, Schrodinger, eventually his work was unified with the Heisenberg matrix mechanics work by Paul Dirac, who ultimately wrote a textbook in 1930, Principles of Quantum Mechanics, and then John von Neumann, mathematician 1932, mathematical foundations of quantum mechanics. And they basically put all this stuff together. I mean, Schrodinger won a Nobel Prize also. Heisenberg won a Nobel Prize. Dirac won a Nobel Prize. John von Neumann never won a Nobel Prize. That's interesting. But basically, Dirac and von Neumann constructed what we now call the Dirac-von Neumann axioms, where they axiomatized the theory.

2:00:29And now Schrodinger's waves and Heisenberg's mixture mechanics are just different facets of a deeper theory. this deeper theory doesn't treat wave functions as fundamental it's not entirely clear how the Dirac-Vindon-Land theory which now we would call orthodox or textbook quantum theory it's not quite clear how this can be viewed as a constructive model the wave function is abstracted and now is regarded as merely a way of looking at a more basic object an element of what's called a Hilbert space, a much more abstract kind of high-dimensional space. And the Schrodinger equation becomes a statement of what's called unitary time evolution.

2:01:16And we introduce a whole formalism of observable quantities and all their arithmetic rules. There's a notion of how we get probabilities out using the Born rule. And then there's this notion of collapse. So this is where quantum theory is after 1930, 1932. Dirac and von Neumann, we have these axioms. In very quick description, the axioms say that to every quantum system, there is some kind of abstract space of quantum states. These abstract states are mathematically defined. There are different ways to formulate them. In some special cases, they're vectors, more generally, they're operators. In even more abstract approaches to quantum theory, the so-called algebraic approaches, they're elements of what's called a dual space to a C-strologer, but the details don't matter.

2:02:07There's some kind of a space where quantum states of some kind live. The spaces and the states are mathematical. You could kind of think of those as being the elementary moving parts if you want, sort of, although we'll get to that in a moment because there's some real problems there. And then the next statement is a notion of like, if you've got a composite system that's made of subsystems, there's a kind of meriology there. We're supposed to take the spaces of states of the subsystems and they combine in a certain way called a tensor product to get the space of states of the full system. Then we have a rule for how quantum states change with time.

2:02:50They change with time according to a time-indexed family of rules. The simplest case are unitary rules. The Schrodinger equation turns out to be a special case of this. And you'd call those, I guess, the abstract mathematical parts of the axioms, the Drachlan axioms. When you phrase them in terms of Hilbert spaces, you call them the Hilbert space axioms. That's the Hilbert space side of the theory. And I'm going to keep using that, even though you can phrase those axioms using different mathematical spaces. I'm going to call that the Hilbert space side of the theory. Mathematical states and mathematical spaces and mathematical rules for how they change with time.

2:03:31On the other side of the direct phenomenon axioms, you could call those the stochastic measurement axioms. They talk about how we connect up that stuff with the world. So if the Hilbert space axioms have a kind of element or moving parts, states, and dynamical rules for them, the other side of the axioms give us our probabilities and our, to some extent, our hermeneutics, our interpretation. The quantum system, each quantum system comes with a collection of abstract symbols. We call them observables. And these symbols are associated with the system. They have a hermeneutical read. Each symbol corresponds or represents some observable quality or attribute or property or feature of the system, position or momentum or energy.

2:04:22Every one of them has a symbol. and these symbols can be manipulated according to certain mathematical rules. There's a way to take these symbols and from them extract a spectrum of what are the possible results you could get if you measured that thing. You measure a particular observable energy or momentum or angular momentum and then this symbol will mathematically tell you what are the possible things you could get on any such measurement. The probability with which you will get a particular the result is given by a particular formula that combines the quantum state and a particular piece of the symbol representing the observable.

2:04:59It combines them and gives you a probability to get that measurement outcome. We can average over measurements and get what's called a measurement average or expectation value. So we can talk about averages of things at the level of averaging over measurements. That will also be important. And then there's a second dynamical rule, the collapse rule that says that if a measurement is done by an external observer, external to the system, then the state of the system is projected or collapsed to lock in the result of that measurement so that if the same external observer or a different external observer goes in and measures the same observable feature shortly thereafter before the system has had time to do anything, then that same or different external observer measuring the same observable will get the same results with near certainty.

2:05:56And this seems like a necessary requirement. If I measure something, a good theory would tell me if I measure the same thing right away or someone else comes along and measures the same thing, they should get the same answer. Otherwise, what are we doing? The problem is that this collapse axiom is a different kind of time evolution, a different dynamical law. We now have two different dynamical laws. We have the dynamical law from the Hilbert space side, where there were no, we didn't talk about external observers or measurements, and there were no probabilities, and there was no definite outcome, just the quantum state changing in some way.

2:06:32And on the other hand, we have the collapse evolution on the other side of the theory, on the stochastic measurements of the theory. An external observer comes in, does a measurement, and then the state of the system changes in some abrupt way, in particular by singling out some outcome. A particular outcome is singled out and with a probability. Whether that probability is subjective or objective is not immediately clear, but probabilities enter at this stage. It's important to emphasize that the two kinds of evolution we're talking about here, the Hilbert space side evolution and the stochastic measurement side evolution are not merely quantitatively different.

2:07:22They're not part of some continuous spectrum of possibilities. They are categorically different. And I think this is where there's a lot of confusion about what's going on. They're not connected in some obviously smooth way, they seem to be just categorically different. One of them, the stochastic measurement one, singles out a unique specific outcome with an associated notion of probability. And the other, the Hilbert space kind of dynamics, doesn't do either of those things. And so people will say, well, maybe we just have to think of the collapse rule as being emergent in some sense. But this is not how emergence works.

2:08:04Emergence is not enough to explain how you can go from not having probabilities and not having a single outcome to suddenly having these things. This is why the categorical difference between them, I think, is so crucial. Now, you might say, okay, well, so what? So a theory has two different kinds of time evolution, and they're categorically different. Why is that so bad? The problem is the theory doesn't supply us with a definition of what counts as an external observer or what counts as a measurement. If the theory doesn't supply us with this definition, how are we supposed to know which form of time evolution we're supposed to use?

2:08:42It's as simple as that. This is not, this shouldn't be controversial and this shouldn't be confusing. We have two categorically different kinds of time evolution. And what distinguishes them is undefined by the theory. This is just clearly a problem. It's objectively a problem. This is the measurement problem. So I don't think the measurement problem is some interpretational question. It's not something that, you know, you don't have to believe in. It's just abjectly a problem of these axioms. These axioms have a problem in them. And the idea that a physical theory should have a problem in it is not a new idea.

2:09:20I mean, Newtonian mechanics has weird behavior in some situations, singular behavior. Electromagnetism has the famous problem of the self-energy of the electric charge. General relativity famously has singularities in it. All of our theories have places where they break down. The standard model we think breaks down. So why is it so terrifying to anyone to think that in some circumstances, quantum theory should be incomplete or break down somewhere? We just should face up to it and deal with it. These axioms are incomplete. They're either inconsistent or ambiguous. The theory doesn't tell us what a measurement external observer is, which you need to determine which evolution you're going to use.

2:09:56It's as simple as that. Now, why have people ignored this problem? I don't know. I don't fully understand. Well, not why people have ignored it, but why you think it's so important here is that if we wanted to – It's one problem if a, I don't know, if the direction is lower or higher. Like if plate tectonics has some place where it breaks down, it's okay as long as it solves our problems with, I don't know, earthquakes, I don't know. But we don't want our theory, our fundamental theories of reality to have big gaps in them. Is that sort of...

2:11:11It could mean that the theory needs to be replaced with a better theory, which will not have that breakdown. It could be just a signpost that tells us we need to do more work, more work needed here. It could mean that maybe nature is not fully describable by laws. Maybe something actually is like, I mean, why do we think that nature can ultimately be described by some self-consistent theory in the end? We've been very lucky so far that we keep getting better and better theories in the sense that they make more and more precise predictions. What right do we have to expect that nature will ultimately be describable as some kind of complete theory?

2:11:49Maybe we can't. So, and there's only two possible ways to read this, but certainly it's not new for a theory that at least at some time was purported to be a complete fundamental theory, could have had a place where it breaks down. Why this particular breakdown was regarded as a silly thing for people to think about, I don't fully understand. There are intricacies in the history. This is where the history matters. There are intricacies in the history that I think people have to have to study and understand better. A lot of people have looked into this history. And some of this is just historical contingency.

2:12:19contingency. There were certain philosophical views that were very prevalent at the time quantum theory was being developed. You know, this was the heyday of positivism. And, you know, so there's all there was all kinds of probably historical and sociological reasons why very big, important people declared that this was a non-problem. We shouldn't worry about it. And then they taught their students this and they taught their students this. And meanwhile, there were so many other important things to use quantum theory to do. There was a lot of money and there were grants to apply for and then spend on building new technologies and building smartphones and building new forms of energy and all kinds of things like that.

2:12:59And I just think that it was regarded as maybe a waste of time, also maybe a dead end, that people had thought about this and they hadn't made much progress. And who really needs it? It's impractical. I don't fully understand why. Honestly, I don't fully understand why. I would like to understand better. I think if people understood this history better, maybe we'd realize what a mistake this was. I think it was a mistake. The study of the measurement problem produced a huge number of spinoffs. The original EPR thought experiment, Einstein-Podolsky-Rosen thought experiment, Schrodinger's work on entanglement from the same year, 1935, quantum steering, which Schrodinger, he introduced that term in 1936, inspired by the EPR experiment.

2:13:46And, you know, I mean, when you ask someone about the measurement problem, they will often say, doesn't decoherence solve this problem? Well, decoherence has a really interesting history. Maybe you should explain what decoherence is for our listeners who aren't. Yeah. So the history will help explain what it is. So there's some early work that you see from Mott in the 1920s that leads to this. But basically, decoherence in its reasonably modern form was introduced by David Bohm. And he's someone we'll talk about in a little bit because of his work on interpreting quantum theory. In 1951, he was at Princeton, and he wrote a book, a textbook on quantum theory.

2:14:26It's called just quantum theory. I have it somewhere on my shelf. And it's a traditional textbook on quantum theory. But Bohm tried to do something that by the early 1950s was considered not a really good idea. he tried to account for how measurements work he tried to give a self-contained theory an accounting of how measurement could work consistent with the axioms of quantum theory to explain away the measurement problem basically to resolve it and he did this starting in a chapter chapter 22 in the book chapter 22 um uh the measurement process quantum theory of the measurement process. And he thought he'd accomplished it.

2:15:11And he thought he'd accomplished it because he introduced this thing that we now call decoherence. The title of the section, section 22.8 of that chapter, is called The Destruction of Interference in the Process of Measurements. Let me tell you what this is. If you take a quantum system with a quantum state represented in some sense, although, again, the interpretation is not clear, but it has some notion of a quantum state in some notion of a space, a Hilbert space of some kind. And it's some blend or superposition of things you might have classically thought were specific ways the system could be at some superposition of them.

2:15:50You can get strange effects. You can show that that if you were to try to think of this quantum state as representing a classical epistemic mixture of the possibilities, the system really is in this classical configuration or that classical configuration or that classical configuration with some appropriate probabilities. And you let the system evolve, you might expect certain phenomena to happen. You don't see those phenomena. You see weird new phenomena that we call interference. The classic example is the double slit experiment. You send particles one at a time through two slits. Classically, if you're thinking of it as either having gone through one hole with some probability or having gone through the other hole with some other probability, then you'd expect a certain pattern of landing sites over many, many repetitions of the experiment.

2:16:43When you actually do this experiment with electrons, it wasn't actually done with electrons until relatively late, historically speaking, but it was finally done. You see a pattern of landing sites that looks very strange. And we call that pattern interference. interference. It's almost as though if we were to give a very flat-footed hermeneutical interpretation of the classical configurations that have been blended together, it's almost like they're talking to each other. They're in some sense interfering and reacting with each other. When you look at the final pattern of landing sites, there are places where there are no landing sites, which is very strange.

2:17:25What David Bohm showed, is that if you bring a measuring device in and treat it like another quantum system, assign it its own quantum state, and let it interact according to unitary dynamics, the Hilbert space dynamics, so not using the measurement dynamics, using the unitary Hilbert space dynamics, what will generically happen is the quantum states will become intermingled. they will become, borrowing from Schrodinger, entangled. They will go from having their own separate definitions to being inextricably linked or tangled up with each other so that you cannot easily assign them a simple quantum state alone.

2:18:12You can. I mean, you can assign them through what's called a partial trace, but the original very simple versions of their quantum states get kind of mixed together. And what Bohm was able to show is that after this happens, the if some other system were to come along like a human were to come along at this point after the measuring device and the system was measuring after they've gotten tangled up if a human comes along and tries to look for observable manifestations of interference the human will no longer see them the interference effects will be suppressed they will apparently not be there anymore.

2:18:52Where we had interference, we will now have decoherence, the physical destruction of interference in the process of measurement. And so, you know, Bohm looked at that and went, oh my goodness, this looks like the interferences are gone. This very characteristic quantum exotic thing that thinking of the different possible configurations that then blend together as a position, they all seem to be talking to each other, interfering with each other. When you bring in some kind of measuring device and it interacts with your system and then a human observer comes along and and and now looks at the behavior of the original system the human will no longer see evidence of interference it's like the inference has gone away and so the human could if the human wanted to pretend that the blend is just a flat-footed classical probabilistic mixture now the human could just pretend that really the system is in one of its configurations and not in a blend of them.

2:19:49And Bohm thought in this way he had accounted for the measurement problem. He then spoke to Einstein and showed Einstein his work. Einstein was, this is the early 50s, Einstein was still alive. And Bohm presented this argument to Einstein and Einstein after some lengthy discussion convinced him that it didn't work. And it was the next year that Bohm introduced his hidden variables, interpretation of quantum theory, 1952, in a pair of papers. And the only acknowledgement in the acknowledgement section that he made was to Dr. Einstein. Now, arguably, Bohm understood decoherence. After all, he first formulated decoherence, right?

2:20:29So I think we could grant that he knew what he was talking about, and he understood that decoherence alone wasn't able to solve this problem. And the reason is very clear. Decoherence is still Hilbert space evolution. There's nothing about decoherence that generates a probability anywhere or that singles out one definite specific outcome. Decoherence cannot do these things. You still have to then take whatever it is, that tangled up quantum state that no longer shows a lot of interference, and you still have to do the second kind of stochastic measurement evolution to single out one state, collapse it down to one of them with some probability.

2:21:09It doesn't get you over the categorical gap. Other people worked in decoherence. uh uh data ze worked on decoherence in the in the 70s uh and then later people also worked on it and a kind of attitude grew up that oh decoherence solves all the problems we don't need to worry about this anymore and it was basically a it was false hope it gave people the erroneous assumption that this problem had been solved and it wasn't but the reason i bring up bohm is because decoherence is clearly important it's not the full story but it's certainly part of the story and decoherence went from being a philosophical curiosity to something that we now worry about today people building trying to build quantum computers are worried about decoherence all the time the word decoherence shows up in quantum computing papers all over the place to build a working quantum computer you have to shield these delicate qubits from decoherence and so people are talking about how to build fault tolerance uh robust qubits that you can put through many logic gates that don't decohere, that have a sufficiently long decoherence timescale, they should be paying Bohm royalties.

2:22:19They can't pay him royalties anymore. Philosophers of physics will take the royalties. Please give us the royalties. But this is a clear case. We're trying to approach a question, a measurement problem, spun off an incredibly important result. And we see many more of these results. Quantum advantage in quantum computing was inspired by David Deutsch in 1985, trying to argue that Everett's many-worlds interpretation was right. He thought he could convince everybody that there were many worlds because he thought you could use them, use the parallel universes for computation to do certain computations more efficiently.

2:22:53And he thought if you could really build a quantum computer that could take advantage of those other worlds, it would convince everybody those worlds were there. Now, arguably, that doesn't quite work. In the modern Everett approach, which we'll talk about, you don't get well-defined worlds until decoherence has separated them so they're no longer interfering with each other. And once they've decohered and they've become separate worlds, they're no longer useful for computation. So to the extent that the extra worlds are there, they're not there for computation. And so this picture turns out not to work so well.

2:23:25But nonetheless, thinking about these philosophical questions, these questions, the foundations of quantum theory, these questions about the consistency of quantum theory, They spun off a lot of important results. And I'm not mentioning the no cloning theorem, the no signaling theorem, GHC states, Cauchy-Specker theorem, Bell's inequality, which went from a philosophical question or a question of the foundations of quantum theory to a practical tool that's used for all kinds of practical purposes. And I'm scratching the surface. I mean, in terms of input to output, funding going in and important groundbreaking results coming out, I think there are a few fields that have been as productive as foundations and philosophy of physics, especially given what happened to many of the people who tried to work on it.

2:24:12A lot of these people who worked on these foundational questions saw major career problems. They were warned away from working on these problems. They left physics for various reasons. And I think that's a shame. And I think we ought to revisit that. so that's some history i think the history is really important i think if people forget about this history then then they don't realize how important these problems are to solve and why they weren't solved once you understand what happened with bohm it becomes i think clear why decoherence is not enough and why this is still an outstanding problem so for the the remainder of our time i'd really like to get to the indivisibility approach and to that end and also because while I've greatly appreciated the mathematics and physical rigor of the discussion thus far, I want to make sure that our listeners who aren't as familiar with these topics also get a good picture of what's going on.

2:25:09So I'm wondering if there are any more intuitive ways possible, possibly using thought experiments or referencing actual experiments like the double slit, twin slit experiment, you could point out some of the other problems with foundations and quantum mechanics and why various approaches to them have not been successful and then how that leads into the invisibility approach. Great. Yeah. So we've talked about the measurement problem. There are, in my view, at least three other problems. One of them is another kind of category problem, even if we solved the measurement problem, even if we had some clear objective definition of what an external observer is and a measurement is, we could know when we're doing a measurement and not doing measurement, we would still have the problem that the Diraclan axioms only talk about the only physical processes they talk about are measurements.

2:26:08That seems like a very narrow category of phenomena. What about the much broader category of phenomena, non-measurement phenomena that we think are happening all around us. You know, in the early universe, there's no, there are no observers that we think. We don't think there are any observers or external observers or measurements going on in the early universe. Are we to say that nothing was happening in the early universe so that quantum theory can't account for it? I talk a lot with early universe cosmologists now, and they're very open about this problem because they often use quantum theory.

2:26:36They'll just compute probabilities about things happening in the early universe, and I'll ask them, what do you mean by probabilities? There's no one doing any measurements. And they'll say, well, you know, we put brackets around things. We take an average. And I'm like, but the average you're getting is a measurement average, not just an average of stuff happening. Like, if you imagine classically, little particles flying around, you could ask, on average, what are they doing? Or if you look at people, you know, deciding what concert to buy tickets for, you could compute averages. On average, what do people like?

2:27:09This is a very different kind of average. This is a measurement average. And how do we do that without measures? And the answer I usually get is we don't know. This is like a real question. What justifies our use of averaging or probabilities in the early universe when there are observers? This is a serious problem. Merely saying that we want one kind of average, and so we're going to use a measurement average instead, is to make a category mistake about what kind of average we're taking. So I call this the category problem, the category problem as opposed to the measurement problem, which in some sense is also a kind of category problem.

2:27:40How do we get from the narrow category of measurements in particular to the broader category of phenomena that we think are happening? How do we go from averages of measurements to averages of stuff really happening? The third problem is what is the right hermeneutics of the elementary moving parts? Which mathematical features are the elementary moving parts and what do they correspond to in the world? What are the physical things in the world they're supposed to correspond to? I mentioned quantum states. Could quantum states be the thing? I mean, Schrodinger thought his wave functions were the thing, and those eventually became the quantum states.

2:28:18But he gave up that idea because they had a probabilistic meaning. Some people have gone back to arguing the quantum state is the physical thing. This is the Everett interpretation of quantum theory being a good example. Everett originally said, no, what the universe really is is some giant quantum state, the universal wave function or the cosmic state vector. That's a hermeneutical move, but this runs into some problems. One problem is that you can change the mathematics around in radically different ways. And then what happens to the hermeneutics? Quantum mechanics turns out to have a number of, broadly speaking, you could call them gauge choices, which is just a fancy way of saying a complete change in how we represent the mathematics that doesn't change any of the predictions of the theory, but makes the link between the mathematical structure and the ontology, the hermeneutical link, it makes it look suspect.

2:29:12We have a state doing a particular thing, moving in a particular pattern in its abstract Hilbert space. With a mathematical transformation, I can make it look like it's doing a completely different thing. And yet none of the predictions change. This is called a Foley-Wauthausen gauge transformation that was first introduced in 1950. I won't go into technical details, but it means that it's hard to make a hermeneutical link between the object and something physically out there in the world because we can radically transform what that mathematical moving part is doing in radical ways. Even the entire Hilbert space as a whole, which is where the quantum state is supposed to live, can be switched with different inequivalent Hilbert spaces.

2:29:52mathematically an equivalent, but empirically essentially equivalent. And so the whole Hilbert space appears to be like a giant choice up to us. This makes it very hard to think that quantum states living in Hilbert spaces or suitable generalizations could be the thing that has the hermeneutical link to something physical in the world. I call this the physical object problem. Some people call it the ontology problem. And the last problem, which is connected to all this muriology stuff we're talking about, I went to great effort to explain why pre-quantum theories had a nice myriology, a myriology the level of the objects.

2:30:27We can go from composite objects to parts or back again. We can go from the laws that apply to the parts to the laws of the whole and it's all consistent. textbook quantum theory doesn't have a well-defined muriological structure of that kind, the kind that Newton was worried about, and we should be worried about it too. If I give you a quantum system, it will almost invariably, in all cases we know of, it will be a subsystem of something else, and it will be a composite system containing subsystems. You could make an argument that maybe elementary particles are not like that, but we don't know that all that many particles are truly elementary, and really we think of them in terms of fields.

2:31:06But the quantum systems we generally deal with are likely to be subsystems or composite systems. They live somewhere in a tower, a myriological tower or hierarchy. Think of a stack of Russian dolls, right? There's a stack of Russian dolls. There are bigger dolls, the composite dolls, and then there are the smaller ones. And every quantum system we encounter is somewhere in that myriological hierarchy. hierarchy. We know how to go from a quantum state of a composite system down to the quantum states of subsystems. There's a rule for that called the partial trace rule. There's no rule for going up the hierarchy.

2:31:43That rule doesn't generically exist. There's no general rule. There are some contingent cases where you can do it, but generally there's no rule for going up the the meriological hierarchy. This is related to entanglement. But that's at the level of the ontology, the stuff, the bodies. But Newton, in his Principia, was worried about whether the laws were meriologically consistent as you go up or down. And in quantum theory, we know the answer. The answer is there's no answer. Where do I insert the Hilbert space rule into the meriological hierarchy. The Dirac-Vindom and Axiom say you're supposed to, if you have a Hilbert space side, you've got a dynamical law.

2:32:25Do you insert it in the middle of the miriological hierarchy? Do you have to insert it down at the miriological atoms? Do you have to insert it at the miriological so-called top system, the all-encompassing one? It's not clear where you insert it. Usually people think you have to insert it at the top of some hierarchy so you're now at what's called a closed system, a system that you could ignore anything else. But we don't know if there are any closed systems. No one knows whether there's some biggest closed system. Our entire observable universe is not closed. It's possible our entire observable universe is inside of some bigger system that's closed, but we have no idea.

2:33:02We can't possibly know from our armchairs. I call this the Miriology problem or the Russian doll problem. Without a top system, it's not clear where we insert the laws. If you try to insert them somewhere in the middle, well, you really can't. a quantum system in the middle of a mirological hierarchy that's not a top or bottom, you know, it actually, there's not like a rigorous way to insert it. And even if you did, there's no general rule for relating the laws at one level of the mirological hierarchy to other levels in a general way. There are special contingent cases where you can, but not as a general rule.

2:33:39So there's just a catastrophic mirological failure in quantum theory. And this is one thing that I'm spending a lot of my time thinking about and talking to people who think about muriology. Now, you might ask, well, then how do people use quantum theory? I mean, if you don't know where to insert the laws, and even if you do, you can't, how do you know you get a consistent picture up or down? It's because they use the measurement axioms. If you collapse a system in the right way at the beginning of an experiment, You can create the special contingent circumstances in which you can go down the myriological hierarchy and give your system laws.

2:34:19You can only do it if you appeal to the measurement axioms, which are exactly the axioms that people seem to have a lot of trouble with. so you kind of use the measurement axioms as this like escape hatch as this trick to creating the special contingent circumstances in which you can go from the laws of a top system down to laws of the system you want to study basically you prepare an experiment you put the system you want to study in a particular quantum state in doing that you're using the collapse axiom and then once you're there, then there's a rule for taking the laws of the full system and applying them to the subsystem and then you're off and running.

2:34:58But it relies on the measurement axioms in this very suspect way. Naturally, without the measurement axioms, you're just never going to find yourself in a circumstance in which you can do this muriological move. So I consider that to be the fourth problem. There's a measurement problem, a category problem, the physical object problem, and the muriology or Russian doll problem. I think these are all really serious. And when we think about interpretations or reformulations of quantum theory, we should ask how they address those problems. And this is where we begin to see some problems for the existing interpretations.

2:35:32Now, one of the interpretations is Bohm's interpretation. Bohm supplemented the wave function. He took the quantum state to be there in some abstract high-dimensional space. And then he included additional ingredients that he called hidden variables. These are the locations of actual particles. And the idea was that he was able to write down laws now. He wrote down a constructive theory. He has moving parts, the wave function, and these particles. He gave them some kind of hermeneutics. He gave them some kind of interpretation. He wrote down laws for them, dynamical laws, deterministic dynamical laws in the original non-relativistic version.

2:36:08And these laws worked great. They gave great predictions. The probabilities didn't come from the laws. They came from uncertainty about the initial conditions. With the right uncertainty distribution over your initial conditions, you get the correct probabilities from quantum theory. And so you have all the ingredients. You've got the element moving parts. You've got the dynamical laws. You've got probabilities. You've got hermeneutics. What's not to like? In a lot of ways, the Bohm theory is really very nice. It's just that it works so nicely really only when you have a system of a fixed number of finitely many non-relativistic particles where you have a lot of simplifications.

2:36:41It has not generalized easily or elegantly beyond that case. If it did, I would be happier with it. But it really hasn't. And that's a serious problem. So let's put the Bohm theory aside. That's Bohmian mechanics, also known as the de Broglie-Bohm theory, because de Broglie himself formulated a very similar kind of a theory. And eventually he and Bohm, you know, met and discussed and it's called the de Broglie-Bohm theory. And pilot wave theory for another name. The wave function serves as a pilot wave that guides the particles around using the dynamical laws. Now let's get to the Everett theory, which is the other.

2:37:17So people ask, what are the big interpretations of quantum theory? They go, well, there's the orthodox interpretation, which comes from the direct final and axioms. There's the Copenhagen interpretation, which is a little bit different, which we don't have time to talk about. And then there's the BOM or pilot wave or hidden variable type approaches. And then there's the Evert approach. I would also think of spontaneous collapse as another one. Yes, yes, 100%. So spontaneous collapse attempts to basically give a definition of when collapses should happen. It's not because humans are involved or measurements.

2:37:50It's just any time you have enough particles involved, then they're entangled. Eventually, you know, the system will collapse. You have to introduce a number of parameters for these models. We could talk about them, but in the interest of time, maybe for now we won't. But that's another way to do it. And just let me add, though, for our listeners, if this sparks a recollection that Everett is also known as many worlds, that's all. Yes, yes. Basically, you have options. If you're confronted with a measurement problem, you can either give me an objective way to say when measurements happen, and then that's like a dynamical collapse theory.

2:38:25or you can take the Hilbert space side of the theory and push it all the way to the other side until it covers everything. Now there's no stochastic measurement axioms at all. And you could call that the unitary option and that's kind of what Everett did. It's a little bit like what Bohm did. Their probabilities are no longer in the laws anymore. There is no collapse postulate anymore. There's no collapse happening at all. and we talked about what Bohm did what Everett did was just take the only physical object to be the universal wave function evolving according to the Schrodinger equation just take the Hilbert space axioms and just basically say no more

2:39:08Everett introduced this idea in 1956, 1957 I don't know if he was aware that Schrodinger had said words that were kind of in this direction back in the 1920s and it took over a decade decade for this to get real purchase in the physics community. 1970 was when Bryce DeWitt basically announced this interpretation of the physics community in the physics journal, Physics Today. Originally, Bryce DeWitt was a skeptic of this idea, and then he was won over after some correspondence with Everett. And there's a beautiful history to be told about Everett coming out of retirement, going back into physics, talking to Bryce DeWitt, talking to David Deutsch.

2:39:53This history, you know, it would be worth a whole conversation with a historian of physics by itself. I can make recommendations to people that I think you'd enjoy talking to about this, but we don't have time to talk about that.

2:40:08So the idea took a while to get purchased, but it had a lot of problems. One problem was if there's just a universal wave function, how do we break it up into realities? How do we, because, you know, we look around and we see a world around us. How is it that this universal wave function should be expressed, represented, broken up into classical looking parts in some sense? The famous branches of the universal wave function. How do the branches show up? What are they, how do we get the right branches? How do we get the universal wave function to be branches that each of which corresponds to a way the world classically could more or less look at macroscopic scale?

2:40:57And this is known as the preferred basis problem. Because you could take a universal wave function and you could actually express it in infinitely many different ways as branches in infinitely different kinds. And it's like, what prefers the branchings that individually look like worlds like ours and those that don't. Another problem was the problem of probability. When we talk about some probabilistic thing happening, we have in mind a set of possibilities, one of which is realized. It's realized with some probability. What does it mean if they're all realized? And if they are all realized, and we do somehow find a way to think about that probabilistically, what fixes the probabilities to be the precise numerical probabilities given by the born rule if i split into two branches you might go okay i guess they're 50 50 now but if the born rule says one of them is 80 percent one of them is 20 what does that mean how do you get that even if you had a notion of probability how do we get the particular probabilities that you get um that the born rule since the beginning.

2:42:08If I'm not misrecollecting, I spoke with David Albert and Sean Carroll about this. And I think Sean, who's even a major proponent of the many worlds theory, recognizes that this might be the biggest problem for the theory, though, of course, he thinks it's surmountable. Yeah. I mean, Hugh Everett himself had a one-page argument that the Everettians, some Everettians today will say didn't work. I mean, it is a logically circular argument. It supposes things that it can't help itself to. In some sense, you kind of have to because as John Norton, the philosopher of science at Pittsburgh likes to say, you can't get a conclusion at the end of the deductive argument that is stronger than the premises.

2:42:55And if you don't put some kind of probabilistic assumptions in your premises, you can't get a probabilistic conclusion out at the end. So at some point, you have to either cheat or you use logically circular reasoning. You beg the question in a sense, right? And so Everett had a one-page argument. It doesn't work. David Deutsch presented an argument based in decision theory in 1999 that has a lot of problems with it. It's got a lot of serious problems with it. It engages in some logically circular reasoning and helps itself to assumptions it shouldn't be able to. David Wallace, who I think is one of the greatest philosophers of science who's ever lived and we're living at the time where he's currently living, which is kind of amazing.

2:43:41He wrote a book called The Emergent Multiverse, which attempts to make the Everett approach work properly. That's a 2012 book. And he has something like an 89-page proof of getting the Born rule probabilities out. It is an extremely technical proof. and it helps itself to a lot of extra assumptions that he takes as reasonable, as justifiable, that don't follow from the Hilbert space axioms. And you need a lot of them to get this to work, and so this leads to what I now call the stone soup problem. I've said this in a couple of other places, but there's this old folk story of these traveling soldiers.

2:44:23They're very hungry. They arrive at a town. The townspeople are kind of afraid of the soldiers. They don't share and eat food with the soldiers. The soldiers say, no, it's okay. We don't need any of your food. We're going to make an old specialty. It's called stone soup. All it needs is water and stones, like three stones. And it's a delicious soup. And they ask only for a cauldron and some water. And they fill it. They throw some stones in. They start boiling the water. And the townspeople are so curious. It's supposed to be such a delicious soup. And the soldiers say, oh, this soup is great.

2:44:52But, you know, what would help it is like a little bit of parsley. Just anyone have some parsley? And the townspeople says, I've got parsley. And they give them some parsley. It's like, okay, it's already great, but you know, some carrots. Carrots would really make it complete. And then a person says, I've got carrots. They throw carrots in and eventually, you know the story. They throw chicken in and they throw stock and they throw all kinds of stuff. And then finally, of course, there's a delicious soup at the end and they feed everybody and they share it with the whole town and they all eat.

2:45:13And the best line in some tellings of the story is one townsperson says, can you believe at all this from just water and stones? And I think this is a problem with the Everett approach, which is the claim is you get this whole picture out of just a few couple of very simple axioms, the space of states and the unitary evolution, the Hilbert space axioms, the water and the stones. But you have to tuck in a lot of extra assumptions and it's hard to justify them. One way that they're justified now, some people justify them, is using this decision theoretic approach, which you find in David Deutsch's work and in David Wallace's work.

2:45:52Basically, what they'll say is, well, look, we have good experience that using the tools and methods of probability, Bayesian analysis, and decision theory have worked really well in the past. They're good, reliable things. We have good experience. They work in the past. And because they've worked so well in the past, we're justified in applying them in making predictions about the future. And they combine this decision-theoretic structure with the Hilbert space axioms and claim to obtain the Bourne rule as the probabilities that someone using decision theory, a so-called rational agent using decision theory, should use.

2:46:32A rational agent using decision theory, using the Hilbert space axioms, ought to be rational, to be in accord with the decision theory, should assign probabilities and outcomes according to the Bourne rule. The problem with this argument is that when we make an appeal to past experience as a guide to what to do in the future. We need to justify that somehow. At some level, you can't really, I mean, you get into the questions about the problem of induction and how do we justify making projections into the future based on past experience. This is obviously a very deep and old problem. It goes back at least to David Hume, the 18th century Scottish philosopher.

2:47:12But at just a basic level, if I'm going to make a prediction about the future, my basic moves, if I'm being scientific, are I can appeal to some physical law, some law of some model, and then I can make a deduction. I can say, well, taking this law as a premise, I can deduce that a certain thing will happen. That's one way to do it. Another way is to appeal somewhat more vaguely to a kind of uniformity principle of nature. To say, well, I have experience of a thing working a certain way, I believe there's a kind of uniformity of the patterns of nature. And so appealing to this notion of uniformity, this uniformity principle, I can take my past experience, my past success, combine it with this idea that nature should be uniform across time and then predict what will happen.

2:48:03You have to make one of these two moves. I don't really know of any other examples. And if you're going to appeal to our past success with decision theory, you have to ask, okay, well, in virtue of which of these approaches can we project our past success with decision theory and probability theory and Bayesian analysis? In virtue of which of the two pictures can we project that past success into the future? Well, the Everettians acknowledge that they're not going to get the Born Rule out of a deductive argument from the Hilbert space states and the Unitary Evolution. Those are their basic laws, and those are not enough to pick out what a given branch of their universe is going to do.

2:48:47In fact, their universe is going to have branches where all kinds of things happen, things that are very strange. change. So they don't have the Bourne rule as a basic principle. The other option is to appeal to a kind of uniformity of nature, but the answer is their branches aren't uniform. In an Everettian universe, all kinds of zany things happen on these branches. Some of the branches have for a long time been called maverick branches. These are branches where the coin is heads every time you flip it. Some very unlikely set of, seemingly unlikely set of outcomes, a sequence of outcomes that seems very unlikely at the edge of what you would call the probability distribution.

2:49:27And then you have what I call super maverick branches, which is where you go beyond even the boundaries of what you thought were the game. So you're flipping the coin and the coin just explodes or the coin rearranges atomically into a mouse that smiles at you, right? Things that you wouldn't even have put in your distribution, but that do have some amplitude of occurring. There's some branch where those things happen in the Everett universe. So it's just false that uniformity could be true. Well, it's just it can't be that a uniformity principle in some general sense is true. In a non-Everett universe, a universe where only one set of events takes place, there's only one history that happens in the universe, you can believe that there is some uniformity between the past and the future.

2:50:15That may be correct. But in an Everett universe, it is clearly false. There are branches that violate any uniformity principle you could have. So what do you do? Well, what you need is you need to appeal to some bespoke new uniformity principle. You need to say that although at the level of the whole universe, we can't appeal to a uniformity principle, we can't deductively get predictions about the Bourne rule from the basic Hilbert space laws. we can't appeal to some uniformity principle for the whole universe because the universe has all these branches that do all these dany things we need a new principle that says that when you're on a branch and you have a certain experience of things in the past along your branch then we should be justified in projecting those past experiences into the future of our branch that's an extra principle right and how do we justify that i mean there are infinite number of branches in some of those many of the branches of the same past we've had so far and then they go in different direction so we just have to just declare that no even though there are branches that agreed with our branch up until now and have gone off in some zany direction no we are allowed to project the past experiences of our branches into the future of our branch and you can see once you look at it that this is clearly ridiculous and there's just no way around this problem all of the tricks and clever ways to get around it they just basically run headfirst with this problem and I don't think that this, because I think this is the crystallization of what the problem is.

2:51:48When you talk to people who have misgivings about the Everett approach, they're like trying to pin down, how do we, why can we take the past and project the future? And you really can't. And honestly, if you could, you wouldn't need decision theory. You just say, well, the Bourne rules worked great in the past. So I'm justified in thinking the Bourne rule will work great in the future. And of course, that seems ridiculous, but it's no more or less ridiculous than saying decision theory worked well in the past, and therefore decision theory should work well in the future. So ultimately, I think this can't be solved.

2:52:15I mean, maybe it can be, but I'm not optimistic, and that's why I wanted to work in a different direction. So now let's get to my approach. My approach... Just as an aside, though, this is the first time I've heard of the many worlds approach bearing on the problem of induction, and I appreciate that. Yeah, it's a serious problem. But now let's get to my approach. So we've talked about how one way to solve the measurement problem is just to have an objective reason why collapses happen or don't happen. Another is to take the Hilbert space axioms and stretch them across the whole theory so that now the whole theory is just deterministic, unitary evolution.

2:52:53But of course, another approach is to start from the stochastic side and stretch that across the whole interpretation, stretch it from just measurements to everything. The virtue of this is that we immediately solve the category problem. the fact that we were beforehand only talking about measurements, we've now stretched the stochastic side of the theory to cover everything. Now everything is happening. Measurements, not measurements, it's all happening stochastically. We don't have to account for where probabilities came from by starting from non-probabilistic premises because by assumption, we're taking the part of the theory that had probabilities in its axioms and just extending it across the whole theory.

2:53:31So we deal with the category problem. We deal with the measurement problem. There's no special role for measuring devices. Everything is probabilistic. Everything is stochastic. So we deal with a measurement problem. We deal with the category problem. The physical object problem is addressed in the following way. So in the textbook orthodox Dirac-Vein-Namen theory, there is one ontological commitment. External observers exist and measurement outcomes exist. Without measurement outcomes and external observers, then the whole theory is self-undermining. There are no outcomes. There's no empirical results.

2:54:06There's no theory. There's nothing for the theory to be about. A natural question you could ask is, why only observers, external observers and measuring outcomes, measuring devices? How could it be that the fundamental theory of nature only commits to that as being part of the ontology? I mean, no one thinks that external observers, well, not no one, but maybe a lot of people don't think that external observers are like fundamental to nature. We appear to be made of things. Again, this is a question of muriology. We appear to have smaller parts that we're made out of. you know what is the observer independent like invariant notion of what are the physical objects in the theory and in this stochastic approach you have extended the probabilistic behavior from just measurement outcomes and external observers in particular to just everything so now the physical object problem is well whatever you're saying your element or moving parts are in your model they could be particles if you're dealing with non-religious particles They could be arrangements of fields.

2:55:06They could be strings if you're doing string theory. Whatever it is you're taking to be the moving parts, those are the moving parts. Those are the effectively elementary moving parts of your model, and we give those a hermeneutical read. The physical objects are those moving parts, not wave functions, but just physical objects. There are no fundamental wave functions in this picture that need to be interpreted anymore. So we deal with the physical object problem. And the last problem is the muriolity problem. and to talk about that we have to talk about laws we talked about how newtonian mechanics newton's third law in particular played a very important role in establishing the myriological self-consistency of the theory i should have said that the everett approach also has this same problem in the everett approach there's no clear way to go up or down the myriological hierarchy the everett approach doesn't solve the myriology problem either at least not obviously you could make an appeal to emergence and stuff but to see the fine details would be important, I think.

2:56:04In my approach, you can specify the stochastic laws at any level of the hierarchy you want. You just pick whatever system you're talking about. You write down the stochastic laws. And then there is a way to go up and down the mirror logic hierarchy and talk about, okay, well, here are the stochastic laws at this level, this composite system. Here are the stochastic laws at this lower level. We can now insert the laws at various places and we can talk about how we stitch together the laws in a mirror logically consistent way. So this is just sort of an invitation to the argument that this approach deals with these problems.

2:56:38Now, a natural question is, if this could have been done, why wasn't it done decades ago? If you could just stretch the stochastic side of the theory to cover everything, why wasn't this done? And the answer is it was done. Everett himself in his dissertation, the long version of his dissertation, the 137-page version that wasn't ultimately published, talks about the work of Fritz Bopp, who was working on stochastic approaches of this kind. Everett calls them the stochastic process interpretation. He actually uses that phrase. And what he says is he doesn't object to them. He says quite clearly he only objects to having a hybrid theory with two different kinds of time evolution.

2:57:15Everett says that he thinks it's more elegant to extend the Hilbert space evolution across the whole theory, but he says that Bop is going the other way and that could be a viable approach. He says, in fact, that it would just take prejudice to think nature has to be deterministic. And so he doesn't really render a judgment on that approach. But the approach doesn't work so well. It was picked up by later people, Imre Fenyev in the 50s and most famously Edward Nelson in the 60s through the 80s. It became known as Nelsonian stochastic mechanics. and the problem is that you have to write really really messy stochastic laws to make it work the laws are really messy the whole set of stochastic laws you have to give the system look kind of jerry-rigged to work you start with the predictions of quantum theory and you kind of reverse engineer them and write down very complicated stochastic laws you still have to like solve the schortinger equation and take the wave function and plug it into these complicated stochastic differential equations.

2:58:10And it wasn't clear, I think, at the time how to generalize this from, again, like in Bohm's theory, systems of fixed numbers of finitely many non-relatific particles to the more general case. What all these approaches had in common is they all assumed that the dynamics was Markov. So now I can tell you where we end up. If we extend the stochastic side of quantum theory across the whole theory. But we don't impose that the laws have to be Markov. We allow the laws to be non-Markovian. We can ask, can we get a relatively simple set of laws acting on a relatively transparent world picture? Can we supply those ingredients I said, elementary moving parts that are whatever your degrees of freedom are, give them a hermeneutical read as just arrangements of real things in space, can we write down a set of laws that aren't too complicated?

2:59:06If we give up determinism, allow them to be stochastic, and importantly, allow them to not be Markov. Probabilities are baked into the laws at this point. We could talk about the subtleties about probability at some point. We have ingredients now. Now, the question is, what kind of non-Markovian laws do we need? Do they have to be super complicated? And like I said earlier, one could be worried that you'd need, once you open the door to non-Markovianity, you're opening a Pandora's box and anything could go.

2:59:41In late 2022, I was trying to understand the precise relationship between quantum theory and the classical theory of stochastic processes, which had really come into its own after quantum theory, starting in like the 1960s and later. I wanted to understand how these theories fit together. They have many mathematical resemblances. They both involve vector spaces and vectors. They both characterize time evolution in terms of matrices. They both have observables or random variables in them. You do measurements, their probabilities. They bear a lot of similarities, and I wanted to understand exactly how they fit together.

3:00:18And I was able to make them fit together, and I was quite surprised until I discovered that what I had given up in the process of making them fit together was the Markov assumption. And I thought this was strange because a non-Markovian theory can be really complicated, but the theory I had was not complicated. How would I have found a relatively simple set of laws that were non-Markovian and were also capable of capturing the behavior of quantum systems, all their exotic behavior. Interference effects could be characterized now as a breakdown of the Markov assumption, entanglement, non-commutativity of measurements, all this stuff that we usually associate with quantum mechanics could be understood as a failure of Markovianity.

3:00:58So I immediately started looking at the research literature. I gave these processes a name. The failure, the breakdown of the Markov assumption took a particular form called the indivisibility. I called it indivisibility because in a Markov process, we can pick any time we want, any time, look or specify, specify the snapshot of the system at that time. Maybe if we're doing Newtonian mechanics, we need the time a little bit before that too. But basically a snapshot at one moment of time. And at any moment in time we want, we can do this and then the laws will tell us what comes next, either deterministically or probabilistically.

3:01:34For an indivisible process, you just can't always do this. There are certain times you can ask, you can specify the configuration of the system at any time, but the laws won't tell you how to go from just any time to any other time. If you give me one time where you can specify, where after specifying the snapshot of the system, the laws tell you what comes next, you could ask if I specify the state of the system in between, which I can do, will the laws tell me at that intermediate time what the system will do. And in general, the laws won't tell you that. The laws are too sparse to give you that information, at least for microscopic systems.

3:02:13Once systems get very big and interact with environments, you get a natural process where the evolution gets divided up into lots and lots of very short duration times. And now you can effectively take the state of the system at any time and project it forward. You get a Markov description, a divisible description. But more generally, for a system that is not in contact with large environments, a system that's evolving in a very generic, indivisible way, there are only certain times where specifying the conditions, the laws will tell you what will come next. So I called this breakdown in divisibility, I called it indivisibility, and then I looked in the research literature and I was scooped.

3:02:52There was a review article from 2020 by Simon Mills and Kevin Modi, which was published in 2021. That was a long review article on stochastic processes, classical stochastic processes and quantum stochastic processes or the quantum generalizations of them. And they mentioned this idea that you could have an indivisible stochastic process in a figure, figure six, I think is figure six of their paper. And they don't really explore it in any depth. They don't discuss whether you could, with this generalization, have a stochastic model that's broad enough to encompass quantum theory. They just sort of mentioned it as a possibility.

3:03:31It was inspired in part by work done earlier by Wolf and Chirac in a paper called Dividing Quantum Channels, like 15 years earlier than that. It wasn't about classical stochastic processes. It was a term related to what are called quantum channels. But basically, Mills and Moody were the first to say you could do this with a stochastic process in terms of ordinary probabilities and folding with a classical kind of ontology. But again, they hadn't applied it to see if these were sufficiently broad that you could contain quantum theory within them. And what I had accidentally stumbled upon was that same kind of process, an indivisible process.

3:04:17You have an ordinary ontology, elementary moving parts with a hermeneutical read as just arrangements of things in space. You've got dynamical laws that are stochastic and indivisible. And you get the probabilities out, the epistemology comes out. And that's kind of it. as long as the predictions of this model accord with the predictions that we would get from ordinary quantum theory in the cases where we can test them with observers doing measurements, then the theory makes the same predictions. But arguably with much less mystery, this is a deflationary view on quantum theory. There are no wave functions that play a physical fundamental, there are no physical wave functions out in the ontology anymore.

3:05:02and interference is now not a specifically quantum mechanical thing. Anytime you have some non-Markovian process, whether it's quantum or classical, and you make a Markov approximation to simplify things and get answers that are not exactly right, the discrepancy between your approximate Markov answers and the more correct non-Markov answers, those are all interference. And the interference of quantum theory is just one more example of that. um and uh

3:05:36so you know arguably this is an improvement i mean now there are a lot of questions you can ask how well does this generalize the relativistic case unlike the bohm picture it's not really tied down to particles the relativistic case brings some new challenges the theory of measurement at finite times, local measurements at finite times in quantum field theory. If you imagine doing measurements at multiple distinct finite times in quantum field theory, it turns out to be a really complicated problem, even in textbook quantum theory. So maybe this new approach can shed some light on that as well.

3:06:14So it's, I think, an exciting time. I think we now have a better way to characterize what's wrong with the textbook theory. I think this new approach is metaphysically simpler and more transparent. A lot of the strangenesses of quantum theory are deflated in this picture, but there are still some outstanding questions. And a lot of what I spend my time thinking about are those outstanding questions and talking with my colleagues about them. Excellent. The key word for me is deflationary. I love that. and one thing that i i want to ask a i don't know a maybe a way to put it is a hermeneutical question about this fundamental object problem and i know this is a separate conversation all in itself but do you have a hunch or any thoughts on what the fundamental fundamentalia actually are?

3:07:14Like, do you think that they're strings or point particles or perturbations of fields? So I guess it would be fields. But do you have any thoughts on that? I have some thoughts. I don't think that the fundamentalia are wave functions. In my picture, wave functions are just a piece of mathematics that's useful to calculate things if you want a Markov description. So what else could they be? Well, in these models, it's basically like the old classical days where you would just pick some LNR moving parts, whatever they were appropriate to what you wanted to model and wrote down laws for them. They could be particles on some models, they could be fields on other models.

3:07:54There was no pretense that any one particular choice of LNR moving parts would be the er elementary moving parts, the fundamentalia. When I talked about constructive models, I was clear to say I meant effectively elementary, at least on that given model. So we have a succession of models. Some of them are particle models. Some of them are field models. Some of them are string models, whatever. They all come with effectively elementary moving parts. We can treat any of them as effectively elementary moving parts within this indivisible stochastic approach. Do I have a hunch about what the true fundamentalia are.

3:08:30Again, I said they're not wave functions in my view or abstract quantum states living in Hilbert space. I'm skeptical of the idea that they're fields ultimately. Field theory, so one way that a lot of people think about field theory today is that field theories are effective field theories. That's the word effective again. If you give me a quantum system with certain principles, think of it in terms of principle theory. You can be a quantum system. You say that the quantum system has certain symmetries. It's got certain symmetries. They include things like Lorentz invariance, translation invariance, that the interactions between the ingredients of these models have certain properties.

3:09:12You're working at relatively large distance scales or low energies. There's like a set of heuristic principled arguments that are supposed to lead you to being able to model the system with fields. The fact that fields are so universally versatile is good in some senses. It means that we can start making predictions. We can build a standard model and make highly accurate predictions. At the same time, it also makes one suspicious that this is, in fact, the way nature truly is. If fields work so well and give a good coarse-grained description of so many different kinds of systems, it makes us, maybe at least me, less, it lowers my credence that fields are the fundamentalia.

3:10:02I'll make this even more specific. There are a huge number of fine-grained systems that look totally different, but that at a suitable level of coarse-graining all look like fields. This is great in one level because it means I can describe their coarse-grained behavior you're using a single universal framework, but it also means that if there's so many things that differ at the fine-grained level, but that look like fields of coarse-grained level, it might make me suspicious that fields are really telling me very much about what the fine-grained ontology really is. So I don't think it's fields.

3:10:36Fields are great right now, but we don't know what ultimately it's going to be. And then without fields, I mean, particles have lots of problems. String theory doesn't really say that strings are the fundamentalia, right? In string theory, it's really, there's a whole zoo of other things that are really going on in string theory. Strings are kind of just the tip of the spear, so to speak. They're the accessible, perturbatively accessible degrees of freedom in certain phases of the theory. So I guess I can say I have a hunch about what it's not. But as to what it is, I have no idea. I do think, though, that when people talk about the macroscopic world being emergent, some people will say things like, we don't need to solve the physical ontology problem.

3:11:26There doesn't even need to be a physical ontology. The physical macroscopic world can just be emergent directly somehow from the rules of quantum theory without there being some kind of thing. And I do find problems with that. All the examples of emergence we've seen require a physical substrate. fluidity the fluidity of water is emergent from the gross motions of large numbers of water molecules um cognition in the human brain appears to be the emergent uh um an emergent pattern of a physical substrate individual neurons firing um the emergent intelligent behavior of social insects has a clear physical substrate the individual insects who often don't even know what the whole colony is doing we have no examples of emergence without a physical substrate the patterns of which are what the emergent phenomenon is supposed to be right the emergent phenomenon is supposed to be patterns in some physical substrate and so you need a physical substrate for all the known examples there's some people who say like no the macroscopic world emerges from pure information or something like that.

3:12:39But without a physical substrate, I don't even know what that's supposed to mean. All information I know of has a physical substrate. All information is some pattern of some neurons in the brain or switches in a computer. So I do, although I don't know what the fundamentality is, I think we do need something. We may never know what it is. But just like I think mathematicians kind of have to go into mathematics believing that that paul erdish's transfinite book of all of the proofs from the book right all these perfect proofs you have to kind of believe that's there that you're looking for you know a really beautiful proof i kind of think that we should at least take intellectually seriously the idea that there there is some fundamental physical substrate in nature maybe it will be more confusing than we think it is maybe there'll be some snake-eating-its-tail way to think about it, that it's not one thing, but it's several things linked in some complicated way.

3:13:37Maybe we'll require whole new ways to think about metaphysics in order to think about what these things are, but I think there's something. The best I can say is quantum theory is already so strange. An indivisible stochastic process is actually pretty strange, arguably more deflationary than quantum theory. We don't have to worry about superpositions literally happening. Schrodinger's cat is not literally alive and dead anymore. It's definitely one or the other in the indivisible stochastic theory. So arguably it's deflationary, less exotic, but it's still weird. It may be that whatever the fundamentalia is, is beyond what we can currently understand.

3:14:15When the people trying to develop quantum theory were trying to find laws, they were limited to the kinds of laws they'd seen before. It's amazing to me that von Neumann finished his 1932 book, Mathematically Formalizing Quantum Theory, a year before Kolmogorov axiomatized probability theory. Random variables weren't even part of the general, weren't in general use until like decades later. So there were a lot of things that we just didn't even, couldn't even have thought about yet at the time quantum theory was being developed. And it may be that we need more conceptual revolutions between now and when ultimately we have a better idea of what the fundamentalia could be.

3:14:56I'm not trying to propose a theory of everything. The indivisible stochastic theory is only supposed to be an improvement over quantum theory as we currently understand it that deals seriously with these problems in the theory. Ideally, the kind of mathematical tools that we use can find application to other uses, just like lots of the rest of the history of quantum foundations has spun off useful practical things. but it's not a complete theory of everything I don't know what's fundamentally happening at the myriological bottom level of reality

3:15:32but if you find out please let me know sure great note to end Jacob I have so thoroughly enjoyed this first conversation of what I hope will be many more to come so thank you again so much for joining me on this fine Saturday afternoon. It would be a delight.