AI Governance and the National Security State | Dean W. Ball

20:03Dean W. Ball:I think that's true. I think that's true that we have to do that. But I'm also extremely skeptical of the notion that you can do something like that rationally from the top down. I think that like the new thing that we do, the way the world will look in 100 years or whatever, like that itself has to be grown. And I can't just dream that up and imagine it like, you know, from scratch out of whole cloth. And so, yeah, like whenever I say things like that, I say things like that as a shorthand to describe what I think will be a monumental process of historical change. But the thing is, is like, there's a part of me that's like yelling at me when I say that.

20:43Dean W. Ball:There's a part of my brain that's like, but be careful, don't be arrogant. Don't fall into the conceit that you with your tiny brain and limited perspectives can do this yourself because we can't. No individual can.

20:57Demetri Kofinas:Well, I feel like that kind of spirit is captured in the New Sages Unrivaled piece that you wrote and elsewhere where you talk about having, you're comforted by the fact that America has always winged it. And in fact, your regulatory proposals, which I'm very excited to talk about. And again, I feel somewhat uncomfortable even using that word because it comes with a lot of framing that may not be necessarily helpful, but even that approach of yours is much more decentralized. Again, not a great word to use, but there is a kind of hands-off approach to how you think about this that I think is really actually refreshing.

21:32Demetri Kofinas:And we're going to have a chance to talk about that. So I think we did a good enough job of giving people a sense of who you are and where you're coming from. Let's just go really high level now. What is your definition of AI?

21:45Dean W. Ball:Good question. So I think basically AI has this history as an academic discipline where we were trying to make machines that could mimic human thinking, basically. Thinking machines. And then there's been a lot of twists and turns through that. And so there's a lot of academics who have like really particularized definitions of AI that are sort of like a legacy of this archaeology. But for me, basically, what is AI? It's machines that can think in the flexible ways that human beings can. And it is the set of approaches to statistics, essentially, that have been invented over time to try to make machines think in the flexible way that humans and animals can.

22:35Dean W. Ball:And those approaches, to be clear, are broader than just like, you know, there's obviously like large language models, which are the closest thing we have to like, yeah, that seems like it's kind of thinking in ways that are similar to humans. But there are also applications of the exact same statistical approaches to completely different sets of data that are like totally alien to the human brain. So there are models, you know, that can predict the next nucleotide in a DNA sequence in the same exact way that there are models that can predict the next word in a paragraph of text. And like the next word in the paragraph of text thing feels somewhat analogous to human cognition.

23:20Dean W. Ball:But obviously no human can think natively in nucleotides. But the same approaches work. So it is both the particular thing of we want to make machines that think like humans and that can perform human cognitive labor, and we want to take those same statistical approaches that we use to invent the machines that can think like humans and apply them to different modalities that have nothing to do with human cognition.

23:45Demetri Kofinas:So how do you define human cognition? Or maybe to broaden it out even more than that, how do we define intelligence?

23:54Dean W. Ball:Yes. So what is intelligence? I think that's the ground truth, really. Intelligence, in my view, is the ability to find patterns from the observation of data. And the reason that that's true, when you think about what that means, if I'm observing some, just the world, right? I'm looking at the world and what I eventually, I find a pattern, I infer a pattern in the world. When I drop the ball, it falls to the ground every time. And I can predict. Using that pattern, I can predict what will happen in the future, right? Because that pattern has been very reliable. I can also name that pattern. So like that pattern is a concept and it's a compression.

24:42Dean W. Ball:It's like what I'm doing is I'm finding a faster way to go to understand what's happening around me because it's like, oh, I have a name for that thing. That thing is not totally alien to me. That thing is part of a pattern that I've observed before. And therefore, all intelligence is a form of compression. Language models are a compression of the internet. Well, they're a compression of their training data. Literally, that's what they are. My brain is forming a compression of the real world, a very, very lossy compression, it's worth noting, right? Like, I mean, it's fantastic that, you know, I perceive the world in 3D space and it's quite impressive rendering of reality that my brain is able to deliver.

25:21Dean W. Ball:At the same time, like we are aware scientifically that my brain perceives an extremely narrow range of light, right? There's like this whole universe out there of things that are happening that we have no sensory input to. And so, you know, my brain is forming a really contingent and ultimately quite lossy compression of reality, but intelligence is that active compression. That's what it is. And so intelligence, yes, to put it simply, it is the inference of patterns from the observation of reality.

25:55Demetri Kofinas:So I think that comparison of lossy to lossless compression is actually useful because the way that we compress information as human beings is much more lossy than how large language models do, but we are better at understanding what's important. So in other words, we're better at inferring meaning, I would say, from the patterns, which is essential. It isn't just, just to be clear, you're not saying it's just the observation of patterns or repeatable sequences. It's also using those patterns to infer something deeper about the structure of the data or the structure of the world that allows you to make viable predictions.

26:36Yeah.

26:36Dean W. Ball:It's naming the patterns in such a way, and that would be like concepts. And then it's also using those patterns to make testable predictions about the future.

26:45Demetri Kofinas:And deriving meaning. How important is that? Because it's also unclear exactly. That's the thing that's also confusing. It's certainly a word we could use to describe, but still unclear. And it's not clear whether it will ever be clear, which is, how are these systems actually modeling the world? To what degree are they modeling and deriving something that we would identify as understanding versus simply engaging in a highly superficial sort of idiot savant level intelligence because of just how much data they can process and have access to.

27:19Dean W. Ball:And also what's the difference between those two things, right? I mean, there's a certain extent to which like there's the old meme of large language models being stochastic parrots, right? Just machines that mindlessly just predict patterns.

27:33Demetri Kofinas:Right. Or that other example of where you tell an AI, I need to wash my car. There's an actual example I think a user posted on Reddit. You tell it, I need to wash my car and the car wash is 100 meters away. Should I walk or should I drive? And the LLM responds with, it's a short distance, so you should just walk. What do answers like that reveal to us about the way in which these systems model the world that they can at once be both incredibly intelligent while also occasionally forming opinions or providing answers that just seem utterly idiotic. Granted, we have seen a major, and it's important to know, we've seen a major reduction in instances like these with each subsequent model improvement.

28:18Dean W. Ball:Yes. So, I mean, I think that a big chunk of that is, I think you could make the same criticism of humans, you know, common sense itself is dependent upon the kind of reasoning engine that the observer possesses. And so like that, which is common sense to a dog, dogs probably have some sort of a comment. Like they have, they have their own intuitions about the world. And those intuitions are like, probably like quite right for them, but we wouldn't think a dog has common sense. And like, there are all sorts of things like Humans fall for, there are so many tricks we fall for. A, we can be rhetorically swayed in ways that large language models are harder to, you know.

29:02Dean W. Ball:Like we fall for all sorts of fallacies that a large language model would catch immediately, right? Large language models would be like, that's an ad hominem. Like you're begging the question, right? In ways that like - Right, because they're so good at identifying patterns. Yeah, and also, well, they identify different kinds of patterns than the ones we identify. So they learn ultimately in a very, very, very different ways in the way that humans learn.

29:25Demetri Kofinas:Well, can I ask you something as an example, Dean? So if I read a book, I independently, and in ways that I don't really understand, identify what I think is important in the book. If a large language model reads a book, it seems that what it's doing is it's identified some pattern statistically in language that tells it what areas the author thinks are important, or are important based on some kind of statistical correlation. I spend enough time using these models in thinking through podcast episodes. And again, I'm in no position at this moment to arrive at a conclusion about what's going on, but I can feel something meaningfully different about how I think, versus how a large language model thinks.

30:16Demetri Kofinas:And I'd love for you to speak to that.

30:19Dean W. Ball:Yeah. Yeah. No, I think you're exactly right. You can read a book and you might not remember, word for word, a single sentence in that book, right? There are many books that I love and that inspired me very deeply that I don't think I could quote. I don't think I could literally quote any of them from memory right now. And you'd probably hallucinate if you tried. I'd probably hallucinate if I tried. And yet, I drew things from those books that like high level, high order ideas that inspire me and structure my thinking every single day. And, you know, obviously, the language model, the language model probably does pick up on those ideas, too.

30:56Dean W. Ball:But it's also doing something very different. I think it's worth going to the foundational insight of, hey, why don't we train a neural network to predict large sequences of web text? And it is 2017. There was a researcher named Alec Radford at OpenAI. And most of the AI companies at that time were trying to do, they were teaching computers to play video games. They were doing robotics, board games, things like this. And Radford had this idea of like, there's a data set of 80 million Amazon product reviews. And let me just train a model to predict the next character of those things, of those reviews.

31:37Dean W. Ball:And in doing that, what they found is that they had accidentally, accidentally invented a state-of-the-art sentiment analysis system. Sentiment analysis is like the ability of a machine to tell the underlying emotional sentiment of a chunk of text. And it turned out that as an instrumentality of predicting the next of its fundamental goal, which was to predict the next letter in this database of product reviews, the model learned on its own to analyze the sentiment. Because understanding the sentiment of the text is useful for predicting the next token of that text. So there is like some higher level like understanding going on there.

32:23Dean W. Ball:but I think it's also extremely different. And I think that large language models are subject to all kinds of, you know, again, like we fall for optical illusions. Humans, there are all sorts of optical illusions that humans fall for. And there's all these weird ticks of our cognitions that are ultimately downstream of contingent realities about the way our brains evolved. And I think the exact same thing is true of LLMs. But I hesitate to say that that means that they're not really understanding. I think the more complicated reality is that sometimes they're really understanding and sometimes they're not.

33:00Dean W. Ball:Sometimes they really are just, you know, sort of mindlessly predicting. And other times they're doing genuine understanding. And I think we've gotten much more genuine understanding over time. There are some people who would say, you know, an LLM can never genuinely understand. And my view of that is that that is ultimately like a metaphysical claim about the nature of human cognition, that you're more than welcome to believe. But like, I guess my view is like, at this point, OpenAI has a model that is unreleased, but maybe six weeks ago, made like a really important, like actually important discovery in geometry through connecting different ideas together.

33:41And like, it kind of seems to me like

33:44Dean W. Ball:that if you have an entity that is making like legitimate breakthroughs in mathematics on actually important problems that humans have paid attention to that have befuddled them for decades and a large language model has like arrived at a new insight about that problem. Seems to me that like if your definition of thinking doesn't include the entity that can do that, I feel like it's your definition of thinking that is the thing that is flawed.

34:08Demetri Kofinas:Do you think that reflects the fact that math is a formal system, whereas the world that we inhabit is unbounded?

34:14Dean W. Ball:Well, I mean, I think math is structured in various ways, but I think math is more unbounded than I think we often give it credit for.

34:23Demetri Kofinas:I mean, or maybe it's better to say that it's more amenable to this kind of intelligence, perhaps, than our form of intelligence. Putting aside whether or not we should be humble about what we can know or not know, which I completely agree with you.

34:36Dean W. Ball:Well, yeah. I mean, think, I'm looking out at my window right now and a bird that just landed on a branch. And think about the way that a bird moves about the world in flight versus the way that like a Boeing 737 does. You know, the Boeing 737 cannot like land on a branch. It is incredibly energy inefficient compared to the bird. The Boeing is very useful for lots of things, but in many ways, you know, we have not figured out cognitively. I think there is an elegance to the way that animal cognition works, that we have only dimly captured in neural networks. But the thing is, is like, that should make you quite bullish about AI because like the AI is already objectively pretty useful right now.

35:33Dean W. Ball:And if there's this huge difference, and the difference, by the way, it is, I think it is sample efficiency. I think that is the difference, right? It is the fact that I can read a book once and derive the important high-order insights without memorizing the entire thing. And in that sense, I am more sample efficient, which is also another way of saying, I have formed a better compression of the book. I have found the important patterns in that book, named them, and made them useful to me. I am more intelligent than the LLM is. And I think probably humans are orders of magnitude more sample efficient than the LLM.

36:10Dean W. Ball:But the LLM, by virtue of being a computer, is still quite useful. And it would imply that there are many orders of magnitude we can go to make the LLM closer to human-level sample efficiency.

36:23Demetri Kofinas:So there's a quote that I'm looking to see if I can find as we're talking, but I'm actually going to give up and just try to wing it from memory. And I'm paraphrasing it, but Claude Shannon was talking about information theory. and he said something along the lines of sometimes these messages, these messages that are compressed have meaning, or sometimes the patterns that we find in the data have meaning. In other words, the pattern doesn't require meaning. And so when you're talking about sample efficiency and yes, human beings need a smaller sample size to derive meaning from a corpus of data, I wonder to what degree, Dean, and I don't want to take us too far afield here, because this is more of a philosophical discussion that you and I need to have at 3 a.m.

37:08Demetri Kofinas:in our dorm rooms. But I wonder to what degree something very different is going on here for large language models. And that in order for any of this to make sense, they need human beings as the sort of person in the mirror. That the meaning is something that we derive. It's not something that they are able to derive from that information? Feel free to respond to that. And then I have a series of, I think, more answerable questions.

37:39Dean W. Ball:Yeah, no, I mean, I guess what I would say is this. I would say there are certain commonalities in the types of inferences that LLMs need to make and that humans need to make. Ultimately, the thing about the world is that the world is stochastic. It is incredibly complex, but it's not fully random, right? There's a lot of structure in the world. And there are things about, you know, humans had to identify, like, we find structure in the world and we make predictions and infer patterns based upon that structure. And there are common structures that the LLMs are finding and that the humans are finding.

38:18Dean W. Ball:But the world is also very large and rich. And there are all kinds of structures that humans find that LLMs don't find and all kinds of structures that LLMs find that humans don't find. And I think, by the way, we've barely scratched the surface of what the LLMs understand about the world that we don't understand. Because I think there actually probably is quite a bit that they understand that we don't understand. But I guess what I would say is like, I think what you're getting at is this notion that like agency and goals and like meaning, purpose are things that are much closer to being unique to human cognition.

38:54Dean W. Ball:Ultimately, we don't know the answer. Ultimately, I think one thing that really scares people in the AI safety world is we call AI systems of today, we call them agents. But are they agents in the legal sense, which is to say, my lawyer is my agent, and that means he does exactly. He's like, I'm delegating some aspects of my life to that person, but that person is expected to have an absolute duty to me and is to be like a complete subordinate essentially to me? Or are they a philosophical agent, which is to say something that can act rather than behave, act on its own, have goals of its own that it autonomously acts upon?

39:35Dean W. Ball:And I think that is like a scary thing to think of like, you know, super intelligent machines with goals like that. I think that is like a really big unanswered question. But I think there is some reason to believe that the human agency and finding of purpose, it's not purely an artifact of our intellect or our reason. It is coming from some deeper aspect of our biology and our survival instinct and things like that, that language models are not trained to have.

40:02Demetri Kofinas:So the quote I was trying to remember was actually in the opening of a mathematical theory of communication, which was Claude Shannon's opus on information theory. And the quote was, the fundamental problem of communication is that of reproducing at one point, either exactly or approximately a message selected at another point. Frequently, the messages have meaning. So I just wanted to put that out there for anyone that's interested in obviously reading Claude Shannon's work is I think an important compliment to this conversation. In your article, 2023, or Why I Am Not a Doomer, you wrote that in that year, you went from being a skeptic to a believer, and you would walk around, was it the campus?

40:44Demetri Kofinas:Where were you at this time?

40:46Dean W. Ball:I lived in DC, but I was spending a lot of time at Stanford University because that's where I worked.

40:50Demetri Kofinas:You would ask yourself questions like, what is the nature of the challenge posed by the alignment problem, which is something that you were alluding to earlier, how we should think about risks of misalignment? Is AI something fundamentally new or is it consistent with the pattern of prior emerging technologies, which again is something we briefly touched on earlier? Does AI break the existing constitutional order of the United States or does it merely challenge it? Which I think is a very important question that we'll have a chance to dive into when we talk about regulation. I'm just curious to understand where would you say that you fall on this continuum of doomer versus accelerationist boomer?

41:29Demetri Kofinas:And by boomer, I don't mean baby boomer. I mean like boom, this is the AI boom.

41:34Dean W. Ball:I think fundamentally, I am much more of a techno optimist and a believer in the notion that I have faith in human ingenuity and the human ability to adapt to changing circumstances. At the same time, sometimes I think that the traditional doom world, as typified by a figure like Eliezer Yudkowsky or Max Tegmark, I think that they have a quite simplistic understanding of intelligence and the way the world works, broadly conceived, that causes them to have this very simplistic notion of what could bring about human extinction or doom, as it were. And I guess my view is, again, put yourself in the perspective of 1775 and then try to imagine like World War I happening, right?

42:27Dean W. Ball:And like, imagine explaining to that person, like what a person who only had the knowledge of 1775, the specific war fighting dynamics of World War I. And it's like, man, that would have been so terrifying and unpredictable. And there are, there are things - Well, it kind of was.

42:42Demetri Kofinas:I mean, it was catastrophic for - Oh, it was horrible.

42:47Dean W. Ball:It's one of the worst things that's ever happened.

42:48Demetri Kofinas:And in fact, people's ignorance around exactly how industrialization had changed the nature of warfare is in part why World War I turned out to be as catastrophic as it was. Yes.

43:00Dean W. Ball:And I think that the nature of technological change, when you just think about it through human history, we have constructed a reality, a civilization that is completely alien to the vast majority of human experience in the history of our species. and it's terrifying in some ways, right? In some ways, there is a certain like terror and awe in the, it's awesome in the like 18th century use of that word, and awe-inspiring, right? Not like cool. It is cool. It is cool to be clear, but it is also like there is a certain terror to it. And so I guess when I think about like technological change in macro historical terms, there's a part of me where like sometimes I feel like I'm actually more of a doomer than the doomers because I just realized how alien what we're going to build will be and what sort of unimaginable horrors likely lurk around the corner.

43:55Dean W. Ball:But I also think unbelievable joy, surprising joy to the upside, delight is also just as likely. And in the end, I think that there will be more good than bad. But yeah, I do think that there's tremendous humility that you need to experience if you are doing the enterprise of commenting on AI. Yeah.

44:15Demetri Kofinas:Something I really actually appreciate about your work, Dean, is that you do try to be even-handed, not for the sake of being even-handed, but because you're being intellectually rigorous. And I think that's what happens when you try to do that. In the early days, if you were reading Bostrom's work or some of the stuff that was coming out of the existential risk community. It was very easy to adopt the, if anyone builds it, everyone dies view that Yudkowsky puts forward. But I think as these models get built out and they get better and better, you begin to, I think, at least if you're being intellectually honest, understand that the picture is much more complicated.

44:56Demetri Kofinas:And AI alignment's a great example. I mean, the way that I used to think about the alignment problem, it was like, how do we bridge the explanatory? First of all, well, how do we even come to a clear idea internally about what the values are that we want to impart on these systems? And then two, how do we bridge the explanatory or epistemological gap and convey them in a manner that they will understand? And then so that we don't all end up dead because AI decides that we'd be happier if they lobotomized humans or put sensory neurons in our brains and jacked us up with dopamine or put us in a vat or something.

45:34Demetri Kofinas:But actually, what you realize is that I feel at least that the alignment problem is less of an issue, because so much of what these models are trained to do is to understand, I think in some ways better than us, what it is that we want. Now, I might regret saying that, and I should also say that it doesn't make me less of a doomer. I'm not a doomer, but that's less where my doomishness comes from. But it's just to say that it's way more complicated. I think it's important to have a nuanced view of this. So in the spirit of that, what would you say are some of the things that both the doomers and the boomers, again, this is my term, but it's not the baby boomers.

46:16Demetri Kofinas:I can see how that can be confusing. Get wrong in your view.

46:20Dean W. Ball:Well, I think the fundamental mistake, I think that in the doom conception of the world is a kind of view of intelligence being completely correlated, maybe even identical to power, like ability to make stuff happen in the world. I don't really think that's what intelligence is. And I think that for that reason, though, they tend to think that like that all problems, like a more intelligent entity will be able to solve every single problem better than humans and also have like incredible amounts of agency and ability to like affect change in the world. And I think that's like not quite accurate.

47:07Dean W. Ball:It's in some ways it's accurate, but I think it's a, they're painting with too broad a brush. And there's a, there's a lot of hand waviness in that view. What I would say about the, I mean, when it comes to the boomer side of things, the sort of techno optimist side, there's a lot of different worldviews that are captured there. So like, for example, there's one category of Boomer who really, their optimism about AI and their opposition to government involvement in AI basically comes from a place of actually, ironically enough, kind of pessimism about the technology. They have kind of a deflationary view of the technology.

47:52Dean W. Ball:So there were a lot of people, you know, there was a leaked draft of an executive order that the Trump administration, that the White House put out. It leaked from the White House, but it was real. It leaked in like November, December of last year that referenced the notion of catastrophic risks like, you know, autonomous cyber attacks or bio and stuff like this. And it dismissed those things as purely speculative and hypothetical. And it's like, looking at the state of the technology as of late 2025, if you were willing to even put like write down the words that those kinds of risks are totally speculative and hypothetical, you had a profoundly deflationary view of the technology.

48:37Dean W. Ball:And one thing I've always suspected about the boomers, and we actually see this a little bit starting to emerge now, one of my theories of mind of that community, which again, like I've largely been allied with for most of my career writing about this. But one of my theories of mind about them has been like, I bet you they're actually like, once they fully realize what is actually going on with this technology, they will actually be quite terrified of it and that they might actually be closeted AI pause ban people. You know, like that the notion of super intelligence actually being real. Like very few people have internalized that notion in their bones and then, or at least made an effort to, and then also been like, and we should build it.

49:26Dean W. Ball:I feel quite lonely in that camp. There's a few of us, but there's not many of us. And that a lot of the people who are boomers actually just really, their optimism is rooted in fundamentally a quite deflationary view of the technology. That means that like, oh, well, these problems that the AI safety world points out They're just not problems that we're going to have to deal with, really. They're just dismissing the problems. They're kind of taking the can down the road. I would say that's the fundamental mistake of most boomers, but it is worth noting that there is a spectrum there, and not all of them have that deflationary view.

49:57Demetri Kofinas:So, Dean, I'm going to move us to the second hour. Let's start that part of the conversation fleshing out what superintelligence is in your view, what it looks like, how quickly we get there. And then let's talk about so much of the other stuff that you've written about that I think is really interesting and why you were hired by OpenAI, which is your approach to regulation. Again, not a really good word. Maybe your approach to how to oversee and manage this transition into the era of AI so that we can maintain and protect the things that we value, whether those things be our humanity, our constitution, our bill of rights, whatever.

50:33Demetri Kofinas:And so adjust our socioeconomic and cultural systems and socioeconomic systems to this new reality. And I'm also curious how that fits into the larger geostrategic competition between the United States and China, where this has also shown up in the ongoing dispute between Anthropic and the Department of War, what this means for other countries, their economies, societies, and cultures, whether some are better adapted or more maladapted than others. And also, what does this do to the labor market and the economy? This is something that people are very concerned about, and I would love to talk about all of these things with you in the second hour.

51:12Demetri Kofinas:For anyone new to the program, Hidden Forces is listener supported. We don't accept advertisers or commercial sponsors. The entire show is funded from top to bottom by listeners like you. If you want access to the second hour of today's conversation with Dean, head over to hiddenforces.io slash subscribe and sign up to one of our three content tiers. All subscribers gain access to our premium feed, which you can use to listen to the rest of today's conversation on your mobile device using your favorite podcast app, just like you're listening to this episode right now. Dean, stick around. We're going to move the second hour of our conversation onto the premium feed.

51:49Demetri Kofinas:If you want to listen in on the rest of today's conversation, head over to hiddenforces.io slash subscribe and join our premium feed. If you want to join in on the conversation and become a member of the Hidden Forces Genius Community, You can also do that through our subscriber page. Today's episode was produced by me and edited by Stylianos Nicolaou. For more episodes, you can check out our website at hiddenforces.io. You can follow me on Twitter at Kofinas, and you can email me at info at hiddenforces.io. As always, thanks for listening. We'll see you next time.