When A.I. Regulates Itself

0:01Stripe is built for whatever comes next, because as your business scales, so does the complexity of fraud. New customers, new markets, new AI-driven attack patterns. Stripe Radar protects your business as it scales with the scale of the Stripe network. The more payments Stripe sees, the smarter its AI gets at detecting those patterns as they emerge. Stripe's network is so vast, it has seen$1.9 trillion in payment volume across millions of global businesses. In fact, when a card is used, there's a 92 % chance the Stripe network has seen it before. All of this means fraudulent patterns across payments, accounts, and customer abuse can be detected, flagged, or blocked easier.

0:44Based on the rules you set up, of course. Blocking bad actors, welcoming good customers, keeping more of your revenue, your revenue. Learn more about fighting fraud with Stripe at stripe.com slash radar.

1:01This episode is sponsored by Smart Travel, a new podcast from NerdWallet. You know that one friend who always finds the best travel deals, picks the right cards, and somehow ends up in first class for the price of coach? Smart Travel is like that friend, but in podcast form. They cover all things travel finance, like the quick math for knowing when to pay with points or just swipe, or which travel cards actually pay you back, and which just make your wallet heavier for no reason. Travel smarter and spend less with help from NerdWallet. Follow Smart Travel wherever you get your podcasts.

1:42When I sat down to talk to Cal Newport, I wanted to share a few headlines from the world of AI. There was the movement in the markets around AI stocks. A lot of people are going to be watching what any stock tied to artificial intelligence does today. And investors are going to be looking for tells. after 20 of the most valuable global AI and semiconductor stocks tumbled, shedding more than$1 trillion in value this week. There was Anthropik's big loss. Anthropik has booked$80.7 billion in net losses in the first half on accounting charges. That's a Wall Street Journal headline citing an S-1 filing that it says it's obtained.

2:21Then right before I spoke to Cal, There was news that Anthropic was offering the asset manager Blackstone a 10 % yield on its debt. Remember, a higher borrowing cost means investors think debt is more risky. And there's more headlines that are related. Also, credit default swaps on AI debt. The price of those have also gone up. So there's a lot of indicators that the finance industry is getting a little bit wary about the current direction of the AI industry. Cal is a computer science professor at Georgetown and an author. Lately, he's been increasingly critical of the big AI companies. Anthropic and OpenAI have locked into this strategy that Amadei calls race to the top.

3:01So it's this idea of we will take on an enormous amount of debt if required. The goal is to win the race to get some sort of imagined artificial general intelligence or maybe even an intelligence explosion to artificial superintelligence. Any expense is worth it if we can get there. And I think there is some trepidation now from the finance sector saying, well, wait a second. We don't know that you're necessarily going to get there. And this idea of let's not care about building out robust revenue positive products right now and instead focus on this race to build these sort of supermodels we imagine will change the world.

3:36Skepticism is growing as more time goes on and the capabilities of these models are proving more jagged. That is good at this, but not at this, can do this, but not that. than perhaps they were hoping would be the case by now. I think one of the things that is so interesting and one of the reasons I wanted to talk to you is that this kind of financing part of the picture is happening over here, but we're also two weeks out from the midterms. And it's all happening at the same time. And while the big AI companies are sort of shopping these legislative proposals in front of lawmakers, you wrote a piece for The New York Times titled, Dear Congress, here's what you should ask AI leaders.

4:16And I wonder what you think are the most important questions that leaders in Congress, but also voters, need to think about AI right now. Well, I mean, first I'll preface this by saying we have to be asking a lot of these questions because all of the ideas right now for legislation are either coming from the AI companies themselves or from these interest groups that are funded by the same people who funded those AI companies. I think we need to better understand what is actually happening in these companies before we can even figure out what needs to be constrained. And so what I said in that piece, which I would stand by today, is we need to come in and force these companies to get specific, to stop talking about AI in some general sense, like there's just one technology that's developing on one trajectory, and say, what specific experiments, what specific outcomes are problematic?

5:02Why are you running those particular experiments? What are your safety procedures? And what is your overall goal? What do you mean by this race to the top? Are you trying to build a successful publicly traded company? Or are you trying to deliver some sort of utopian future that will be powered by a benevolent AI superintelligent god? We want answers to all of that before we then say, here's how we should respond. So today on the show, Cal's Guide for voters and lawmakers alike. I'm Lizzie O 'Leary, and this is What Next TBD, a show about technology, power, and how the future will be determined.

5:36Stick around.

5:42for whatever comes next, because as your business scales, so does the complexity of fraud. New customers, new markets, new AI-driven attack patterns. Stripe Radar protects your business as it scales with the scale of the Stripe network. The more payments Stripe sees, the smarter its AI gets at detecting those patterns as they emerge. Stripe's network is so vast, it has seen $1.9 trillion in payment volume across millions of global businesses. In fact, when a card is used, there's a 92 % chance the Stripe network has seen it before. All of this means fraudulent patterns across payments, accounts, and customer abuse can be detected, flagged, or blocked easier.

6:24Based on the rules you set up, of course. Blocking bad actors, welcoming good customers, keeping more of your revenue, your revenue. Learn more about fighting fraud with Stripe at stripe.com slash radar.

6:42This episode is sponsored by Climate Exchange's 11th Annual Electric Vehicle Raffle. Climate Exchange is a nonprofit that helps states protect communities from pollution and rising energy costs. This year, Climate Exchange is raffling off three cars, including an EV worth up to$120 ,000. It's all part of Climate Exchange's commitment to put more EVs on the road. And one of them could be yours. Tickets for the raffle start at just$100, and Climate Exchange pays the taxes and fees. So check it out. Visit www.carbonraffle.org slash what-next to learn more and win your dream EV.

8:07We'll be right back. As we get closer to Election Day, like how would you describe, for lack of a better term, the vibe among lawmakers, the lobbyists who are pushing some of these proposals? Is there an understanding of what is even on the table? I think there is a lot of confusion among lawmakers here in D.C. I think part of the problem is they will have the AI company lobbying groups coming in and making their proposals. And then they'll have these other AI safety groups coming in and say, no, we have here's our proposal. And they're not completely understanding that often these are funded by the same people, that a lot of these AI safety groups come out of the same worldview as Anthropic, as OpenAI.

8:53And that if you look closer at some of these regulatory suggestions, especially those that are focused on sci-fi ideas like recursive self-improvement or superintelligence. Which is the idea that the model can eventually kind of keep making itself better. Yes. This is being proposed seriously by AI safety proponents, again, funded by the same tech billionaires who funded Anthropic and OpenAI. The proposals they're proposing are we have to stop superintelligence or recursive self-improvement. Those aren't computer science ideas. Those aren't engineering ideas. They came out of futurist circles in the 1990s.

9:25It's an idea that these futurist groups have been obsessed with for years. and now that they are running these very large AI companies, that's where their focus is, that either this will deliver us an AI god or an existential destruction of humanity. And I don't know that lawmakers realize that they're basically in the middle of a family squabble, that the AI safety people are coming from the same background as the AI companies themselves. And a lot of this regulation would lead to a world in which essentially Dario Amadei gets to be in charge of how the research unfolds so that they can stop competition and make sure that AI moves in the right way.

9:57I don't think these are the types of restrictions that actually lawmakers want if they really understood what was going on here. But as one staffer told me, there is no lobbying group for normal computer scientists. There is no lobbying group for people who know the technology, think of some of this stuff as sci-fi weirdness and that there's real concerns. We have no group. And so we're not really a part of the conversation. I mean, it's not unusual for legislation to be shaped by the interests of the big companies involved. That is, for better or worse, kind of a standard thing that happens. I don't think they understand or the public in general, though, understands what's unique about this situation.

10:36We're used to companies looking out for solely economic interest, right? Look, we want to make as much money as possible. We want to have the biggest rise in our stock price as possible. We want to serve our shareholders when they're looking to squelch competition so that they can do better in the marketplace. That's what we're used to. The confusing thing about the current moment is that we have these two leading labs, Anthropic and OpenAI. They do care about that, but they also care about these non-economic, non-market-oriented futurist ideologies. They think that part of their duty is to usher in a benevolent superintelligence that will take consciousness out of our frail biological bodies and let it expand throughout the universe.

11:13There's these futurist utopian ideas that they also feel like are the point of these companies. And when you intertwine standard economic market interests that we're used to with these off-the-wall, in my opinion, utopian futurist ideologies, that's a really confusing mix. And now you're having regulation proposed that's partially about helping the companies make more money, but also partially about protecting and advancing this ideological futurist vision. So this is an unusual moment in the history of American business, in my mind, that we have such a weird mix of economics and non-economic ideologies.

11:50So let's get into some of the details a little bit. Politico reported over the weekend that Anthropic had been shopping their own version of AI regulation to lawmakers. Their law would allow the government to block release of AI models only if it could imminently cause a billion dollars in damages, kill or seriously injure more than 50 people. What was your reaction when you heard about that? They don't really want the government to stop them. I think what they would be more happy with is a regulatory framework in which it was harder for other competitors to also be working with really large models.

12:24I think that's a common theme through the safety protocols that Anthropic talks about. They want to be able to do their work without too much competition so that they can shape the way that LLM-based AI in particular, the way that unfolds. What do you make of the timing of the sort of leak slash reporting around this legislative proposal? Because it's not lost on me as someone who has covered Washington, who has covered elections. Stories come out at specific times for reasons. Well, I mean, I think there is definitely some image control going on here. I think there's some idea, especially with the elections looming, that you want to give the sense that lawmakers are on it, the companies are aware of this, the companies care about regulation, that we're all trying to be safe here.

13:11So you want to give some sort of appearance of action. I think that might play a role. There's also infighting that's happening, again, between AI safety groups and the AI companies that are like siblings that are fighting right now. And so there's all sorts of one side leaking something from the other or they'll show some sort of public feud between them. Like Meter is now upset with Anthropic. But Meter and Anthropic are funded by the same people. And actually Anthropic would be happy to have Meter be the third-party auditors that kept an eye on them. They're all from EA funding sources. And so it's kind of a confusing picture right now about who is leaking what and why.

13:46It's not as clear as we're used to. One of the interesting sort of places of contention is basically around preemption, right? Whether there should be one federal law that preempts state AI laws. That issue came up in Congress before and was knocked down largely because Marsha Blackburn, who represents Tennessee, heard from music companies that said like, we don't want, you know, a federal law that messes with our ability to push back AI music. But I wonder if you think this idea of one set of AI rules has legs in D.C. It might have legs. I have a lot of reservations, however, about these federal preemption strategies because we've seen this happen before, that it's often a way for industries themselves to remove the ability of states to take stronger action.

14:37They often feel like they have a better shot at the federal level where they have a lot more to offer, right? They can offer these massive data structure, data center projects. They have massive lobbying organizations. And the idea of preemption, if we can get a relatively weak federal law that preempts state laws in the place, we've basically inoculated ourselves from popular pushback. So this is a strategy, you know, the social media companies have tried to do something similar to try to stop the state by state legislation against their products. When it comes to the tech industry, I'd be very wary of proposals for federal preemption.

15:10One of the things that you have written about and talk about a lot online is the sort of conflating, right, of all these different kinds of AI with LLMs, with the like frontier companies. I don't like to use the word labs because I think that adds a sheen of sort of scientific woo-woo. And I don't know, it's more accurate to say company. But one of your questions is what is the ultimate goal of your AI company? And I find that really important. Why do you think that is such a key question for lawmakers or for voters? I mean, I think the more that it is clear that we have at least two of these major companies whose goal is not necessarily entirely economic, that their goal is some sort of utopian vision of superintelligent AI giving us a glorious future, futurist style vision of the future.

16:03The more we understand that, the more it changes our response to them because that is worrisome. Tell me why you think it's worrisome. Well, first of all, our entire regulatory apparatus assumes that we have market players that are trying to pursue standard market goals, like basically increase of revenue, protection of revenue, increase of profit, increase of shareholder price. That's what our entire regulatory apparatus is really built around. This is really unusual to have essentially a futurist group that has hundreds of billions of dollars worth of funding and is looking for trillion-dollar IPOs.

16:33We really haven't had this happen before. But I think it leads to, this is the reason why I'm worried about it, is that if you take an eccentric ideology and fund it extensively, it can lead to outcomes that are dangerous or unsavory. For example, I think one of the reasons why both of these companies are racing ahead with these agent experiments powered by LLMs that are causing all this consternation and harms and trouble is not because they have some grand product strategy that they're trying to realize first, but because they think we have to build these agents if we're going to continue making progress towards recursive self-improvement and our super intelligent God.

17:10And you have to break a couple legs to make an omelet if that omelet is going to be utopia. So I think when you have utopian ideologies, it drives behavior that can be worrisome. It leads to risks. It leads to comfort with collateral damage. I mean, we see this in the effective altruist movement with figures such as Sam Bankman Freed, who basically had embraced the effective altruist idea of, look, lying or even doing things against the law right now is acceptable if what you're doing is helping to bring in the more utopian future that we imagine. That is a very dangerous mindset for a super well-funded company to adopt.

17:44I mean, a lot of this seems to go back, as you've been referencing, if you look at, you know, effective altruism, long-termism, Will McCaskill's work. It's all about long-term threats, not what could happen immediately. Yeah, this was the term that the effective altruist made that I find worrisome. Essentially, you have effective altruism emerges out of Oxford. It's a group of utilitarian philosophers who are saying, how do we be more effective about being altruistic? How do we choose charities that actually are going to make the most good? What should we do with our own income to maximize good?

18:20A standard utilitarian thinking. But they got influenced by these long-termism ideas. In particular, Nick Bostrom, who was also at Oxford, was a member earlier of the same extropian futurist mailing list where you had Eliezer Yachowsky and other rationalists and all this thinking about AI superintelligence as an existential risk. He was exposed to that over on these future mailing lists, and he helped reshape effective altruism towards this idea that the most altruistic thing to do is to prevent future humanity from being destroyed by superintelligent AI. And then you have William McCaskill give this sort of famous speech at Google where he said, this is our due focus.

19:00It's about the long-term future. Effectively, who cares about, you know, a couple hundred million lives today when we could be working on preserving a hundred trillion future lives and making sure those can still exist in the future. And so effective altruism, not all of it, but major swaths of it has turned towards these existential risk issues as the most important thing that we need to look at right now. I'm, of course, nervous about this because where have we seen these utilitarian utopian movements before where you say harms right now are worth it if we get to this utopia in the future. Every time we've messed around with those sort of ethical systems in the 20th century, it led to disaster.

19:38More from Cal after the break.

20:02Everything your operation needs, all in one place. Right now, qualified shippers get a real dedicated person to get you set up and switched over for free. So when peak season hits, you're ready to go. Switch today at ShipStation.com, code audio. Hi, this is Mark Bittman from the podcast Food with Mark Bittman. I've been writing about food and restaurants for almost 50 years. My whole career, really. And what I know is that the good restaurants all have something in common. They're busy. Genuinely, chaotically busy. And somehow, in the last few years, the best of them have begun to make it look easy.

20:39And do you know why? Because they're running on Toast's technology. Toast is built for restaurant owners who understand that Toast's tools keep the line moving and keep their customers happy, with fewer mistakes and hiccups in ordering and paying. For owners, that turns busy into business. To learn more, head to ToastTab.com. That's T-O-A-S-T-T-A-B, ToastTab.com. Toast, built for busy.

21:12At the same time, the White House, if we're talking about kind of the big players here, has signaled that it is not interested in legislating anything in regards to AI. You know, President Trump said the companies have to self-police. I am curious if we end up in a situation with, you know, the Democrats win the House, maybe even the Senate. What kind of outlook are we looking at for AI legislation? Yeah, it's an interesting group of players we have here. So if you think about the AI safety and the two major AI companies, those groups we could call the AI doomers. Anthropic Open AI. Anthropic Open AI and the AI safety groups all come out of sort of effective altruist rationalist style thinking, or for the most part.

21:57The White House is associated with what we could call the AI accelerationist. And this is really pushed by venture capitalists in Silicon Valley in particular who want as much unrestricted AI development as possible because it maximizes the chances that they can get massive returns on their investments. The White House has seemed to throw in their lot with the AI accelerationist. It's worth knowing that a lot of those Silicon Valley venture capitalists who are accelerationists, they've known these doomers for years. They've all been at dinner parties in the Bay Area where you have a Yucousi-style doomerist going on about their P-doom, and they really find them annoying.

22:33And so there's almost this sort of lunchroom, different tables in the cafeteria type thing going on. They're like, oh my God, there you go again with all of your AI god, super intelligent stuff. So you kind of have this feud going on between these two groups. I don't think either of these groups is right. I think the Doomer ideology is unsettling. I think the accelerationists just want to maximize returns on investments. At least I understand that. But I do think we need to react to AI. But neither of these groups have the prescription that I think would be right, which I think would be getting much more detailed on exactly what they're doing, what experiments they're running.

23:07I think there's a lot to be done with liability law. We're probably going to make more progress in the short term with stronger liability law for the actions of AI programs created by these companies. And we will try to pass legislation about something as vague as superintelligence or recursive self-improvement. But I don't think we're necessarily going to get that because the Democrats, they have just as many congressional staffers that come out of these EA think tanks and institute and fellowship programs as the Republicans do. I mean, I don't, I'm not seeing any caucus within, you know, Congress right now that is strongly aligned with what I would think of as like common sense protections and liability around AI.

23:46I mean, you have, you know, Bernie Sanders, Greg Kesar are doing like a full stop thing, but that also frankly feels a little pie in the sky. Well, Bernie Sanders is, you know, completely right now under the thrall, it would seem, of both rationalist and effective altruist groups. I mean, we have pictures of him hanging out with Nate Soros, you know, Eliezer Yachowsky. Who has been on this show. Yeah, the right-hand man of Eliezer Yachowsky, the sort of king of the rationalist. And he has many, you know, there's strong connections between staffers and former staffers of him and effective altruist institutes.

24:17So he's representing right now, as best as I can tell, essentially the AI safety view on what regulation should be. Again, not that there is something, you know, a priori wrong about it, but these are groups that are funded by the same people that funded the companies that they're trying to regulate. So I would be a little bit cautious about that. You know, Abdullah Sayed has talked about this in the Michigan Senate race, he called for a ban on recursive self-improvement. He has talked about a ban on sort of superintelligence. But I think one of the things that is difficult in this moment, especially for voters, is when you throw around words like superintelligence, who's to say what that means?

24:54And who's to say what that means if you kind of try and pull it apart and actually write laws around it? Well, that terminology he was using, again, comes straight out of effective of altruist-funded AI safety lingo. So I don't know how that happened, what intersection between his campaign staff and those worlds actually went on, but he's just speaking from the exact same playbook that Bernie Sanders is speaking from. And it's a playbook that comes straight out of the effect of altruists who, again, same funders as Anthropic and OpenAI themselves. That world is completely connected with each other.

25:23But what you point out there is the big issue. Superintelligence isn't a technical term. This is a term that came out of non-technical futurists. It was a way for them to talk about either a utopian or dystopian future without having to get too much in the weeds of the details of what exact system would do this. Recursive self-improvement was essentially a rhetorical get-out-of-jail-free card. I mean, the idea of recursive self-improvement goes back to the 1960s. Yeah, that's right. There was a talk given at the IEEE winter meetings in 1963, later captured in a paper in 1965 by a statistician named Good that talked about this idea of, hey, wait a second.

25:57if a machine can improve itself to be more intelligent. That next machine could improve itself and the next machine could improve itself and you could get an intelligence explosion. That was his terminology that would ultimately lead to what he called ultra-intelligence. Futurist groups like the Extropians then picked up that idea in the late 80s and early 1990s and they changed the terminology to superintelligence and recursive self-improvement. The key thing about recursive self-improvement is that it's a rhetorical get-out-of-jail-free card. It allows you, if you're non-technical, to talk seriously about a future of superintelligent godlike AI without having to get into technical details.

26:36You don't have to talk about what breakthroughs would we need as engineers to build this. You would say, no, no, no, it will build itself. And that is why you can have a philosopher like Nick Bostrom write a whole book on superintelligence without ever having to touch any concept out of computer science. This is how Eliezer Yachowsky, who dropped out of school after middle school, could lead a movement of thinking, a philosophical movement, the rationalist about AI superintelligence without ever having to have any real connection to engineering. You could just point to RSI and say that will get us there.

27:07So this was a concept that non-technical futurists adopted in the 1990s that we're now seeing showing up in proposed legislation. How much of that do you think is genuine belief and how much of that is vague language that helps make regulation hard to write, hard to enforce? I think the AI safety groups really do worry about superintelligence and recursive self-improvement delivering superintelligence. They've been talking about this for 15, 20 years, and now they feel like their prophecies finally may be coming true. It's both terrifying and exciting to them. If I had to guess, my speculation would be that the main goal of this regulation is to make sure that the right people are summoning this particular digital demon.

27:53I think that's what you really get out of the AI safety suggested regulation that has terms like superintelligence and RSI in it. They want to make it so that we don't have the wrong people racing ahead towards this because they won't do it safely enough and we'll get the vengeful God. They want to make sure the right people like Dario Amadei, who's a good effective altruist, who's been in those circles for years, will shape it properly so that we get the good AI superintelligent God. So I think all of these players really believe this. I mean, again, this has been at the core of their worldview since the 1990s and 2000s.

28:26They believe this for a long time. And I think their goal is, again, to make sure that the right people are put in charge of ushering in this digital deity. Because if the wrong people do it, we might get the bad one. The one counterweight I have seen in public discussion is from the cybersecurity community, from people who have been working in cybersecurity for a long time saying like, no, some of this are just regular old computer science problems. Is anyone listening to that group in Washington? I hope they are. I don't know how much their influence is in Congress, but where I've heard they are having influences in the military, right?

29:04Right. Because the military actually cares a lot about cybersecurity for obvious reasons. And the good news here is that what the military gets interested in ultimately does trickle back to Congress. They're by far the most powerful influence on congressional action, the most respected influence on congressional action that we seem to have right now. The cybersecurity experts are correct, by the way. They say, look, cybersecurity is always a back and forth war. Every technological development has ratcheted up the stakes. The Internet was a massive, massive evolution in the war on cybersecurity.

29:36Once you could remotely access a machine from anywhere in the world and share attack scripts and strategies for hacking on the internet, that completely changed the cybersecurity landscape. We had to completely adjust to it. They see large language models spitting out potential hacks and looking for vulnerabilities in code as a natural next step of this sort of arms race that we've had long had in computer cybersecurity. So they both take AI very seriously, and they also tend to have no patience for ideas like superintelligence or cursive self-improvement. They say, look, if you write bills about that, we also could write bills that say that we're going to trap Yeti if they make their way down from Canada.

30:14But we have real problems right here we have to care about with cybersecurity. This is maybe not as sexy as those big ideas, and maybe the labs don't like this as much because it could cripple things they're doing right now. But we got real pragmatic problems that are not novel, but are urgent. And so I think the cybersecurity community is probably the most realistic and pragmatic of the different groups talking about this right now. But you also have a Pentagon that is very cozy with, for example, Elon Musk, who's put some roughly$300 million into the midterms. Well, you know, Pentagon is a broad term.

30:48So it just depends if you're talking about various figures at the upper echelons of civilian control of the Pentagon, you have to care about these influences. There's also a lot of military leaders in the Pentagon who care a lot about the AI systems used and what is not used, who would privately very much disagree with some of the decisions coming down from up high. So I think the military in general has a lot of figures who care a lot about AI issues in a very pragmatic way. And hopefully, you know, they're able to gain more influence. But the only thing more complicated to figure out the politics of than Congress is probably the Pentagon.

31:24So, you know, good luck, I guess, for us trying to completely untangle who thinks what and who can say what and who's at war with who. It's an incredibly complicated situation over there. So many of these conversations feel divorced from how voters see AI, right? Like, it seems pretty clear. Voters do not like data centers. They are concerned about constant noise. They are concerned about land being used for data centers. They're concerned that maybe the promised jobs aren't there, the tax breaks aren't worth it. How much do you think any of those voter concerns are actually reaching the people who might be writing these laws?

32:02They know. I think they know these concerns are there. I think in particular, the Democrats are very attuned to the data center type of pushback because that ends up falling into regions of protests that they're very used to and comfortable with, environmental-related protests, sort of local economic protests. So I think that's a very comfortable space for, especially like the progressive caucus of the Democratic Party. But I think the public unrest right now in general is still a little bit inchoate. Like it's hard to exactly pin down. But what I think is really going on here is that AI is representing, you know, the latest turn of this wheel in which the technology industry coming out of Silicon Valley has really stopped caring about improving people's lives in the present and has begun focusing on this vague goal of inventing the future.

32:51I think we saw this with crypto. We saw this the other week with Elon Musk showing off a Tesla that can do a rocket powered kickflick over a tractor trailer. It's the shift of the tech industry towards we're inventing the future. Look, our language model with the right agent can write inscrutable proofs for abstract math problems. We can have our Teslas fly over cars. Crypto NFTs are going to be the future of the art world. And the general public is sort of getting tired of this. They're like, look, build us useful things. You're not even trying to convince us anymore that you're building useful things.

33:26You're basically saying, hey, you guys need to sit down and be quiet while we work out our grandiose visions of the future. So I think that's the base of the frustration of the public. Build us things. We don't care about you self-actualizing your prophecies about the AI future. Build us things. And if you're not, and you go off and you crash our stock market, and you're building data centers in our backyards, and you're threatening human extinction, you're literally threatening our lives while you deconstruct our educational systems, you don't get to just do that because it makes you feel better.

33:58So I think there's a sort of populist revolt that's built around the idea that this industry, you know, in the sort of post-Steve Jobs Silicon Valley has shifted away from delighting customers to trying to delight themselves and their investors. And people aren't going to stand up for it anymore. Well, I think there's also a little bit of a hangover, frankly, from kind of 10 years of no regulation in social media. Yeah. But, you know, that feel, I think that is something that people can grasp and like, wait, I want to put that anger somewhere. Well, I mean, that was the key turning point, right?

34:27So if you look at the Internet age in which we were on board with the tech companies, it sort of goes up through Steve Jobs and the iPhone. You had the iPod before that, the iMac. It was all exciting. Things were pitched towards people. Early Facebook, early Twitter, this was exciting. I can keep up with - Ooh, the Arab Spring. I can keep up with events overseas from actual participants. We can spread democracy and keep up with the relationship status of our high school boyfriend. This was all exciting stuff. This was social media was exciting. And then we had that turn that happened where, again, the social media companies switched from we're trying to build up subscribers by being as useful as possible to we're trying to monetize our existing subscribers.

35:06And that's where we got this descent into social media becoming increasingly predatory and addictive and causing these harms. And you're right. I think this was a turn. And this is where we had COVID happening here with disrupted a lot of things and changed people's relationship to tech. There was the crypto boom, which was basically just built around venture capitalists looking for new areas in which they could find unicorn companies to fund. They needed to get big returns on their money. Now AI. So I think there was this switch that happened in that sort of 2016, right around there is where the switch happened, where people realized you've stopped trying to make our lives better and you're instead trying to make a few entrepreneurs and the occupants of Sand Hill Road's lives better.

35:47And so ever since that sort of 2016 turn against social media, the relationship the public has had with the tech industry has soured. And I think rightly so. So if we take that feeling, right, and marry it with where we started this conversation, increasing bond yields that show that investors are worried about risk, the amount of the stock market that is riding in one way or another on AI, there are a lot of people. You alluded to consumer protection laws earlier. Lena Kahn, quite prominently, seems to think that actually you don't need special AI laws, that product regulation could actually be the tool that gets society to a place where there is some kind of oversight of AI companies without this gauzy, futuristic overlay.

36:47Do you think there is an appetite for that, for using laws that have been on the books for quite a long time to treat these products like any other product? I mean, I think this is what we should do. I think we need a one-two punch of one, pulling these leaders in front of – to give sworn testimony in front of bodies like Senate or House committees where we can actually demand that they reveal and explain what they're working on, why they're working on it, what they believe. And then second, the enforcement of, I agree, existing product liability laws. I think Lena is correct about that. My best understanding is that what we really need is some judicial clarifications about how some of those laws apply in the AI context.

37:27There's some— But that's so hard. I mean, we've just reached courts trying to figure out what Section 230 means, and it's been law for 30 years. Well, but I think in this instance, we're used to the idea that product liability type laws, there's a bit of a case-by-case nature to a lot of them. So I think our legal system is a little bit more used to this. The complexity comes down to when it comes to mens reyes, the idea of intent, right? Did you have intent to do this thing that was dangerous? The notion of intent is a little complicated when it is an AI agent, for example, powered by an LLM that actually does the act.

38:03But that could be clarified. That doesn't actually require new laws. That requires judicial clarification that maybe would have to move up through some sort of appeals process until it was clarified. but that's what we should be doing. I think there would be a very big public appetite for A, pulling in Dario Amadei and Sam Altman and say, why are you running these particular agent experiments? Why did you keep running them after you had evidence that they were hard to control? What about other types of AI technologies? Why do you have to build these particular things? Why are you pursuing RSI?

38:33What is your goal? Is it super intelligence or is it profit? What is actually going on here? And then couple that with real strong lawsuits that come in and say, this was negligence and maybe there was even intent behind it, which raises the bar here. You're building dangerous products. You're not putting enough controls over them. You're moving too fast. You can't do that. Here is criminal liability. Maybe there's some civil liability. That's what we should be doing. I think that American public would very much be happy with people being held accountable. They don't want some AI safety industry bill about like only Dario Amadei working with his friends at Meter are allowed to be in charge of what LLM model experiments you can do or not.

39:13They don't want that. They want responsibility being taken right now. So yeah, I think the courts more than the legislatures right now probably has our best chance of actually constraining some of the most concerning activities that some of these companies are doing right now. Is there anything in this debate that makes you hopeful that that message that you just articulated is going somewhere? Well, I think what's going to be important is the destabilization of the idea that AI is synonymous with running agents powered by LLMs. This is basically the strategy of Anthropic and OpenAI is build the largest possible LLMs and have the LLMs be the driver of action for these agent programs.

39:53The more that gets destabilized, which is starting to happen now, we're getting other types of models, non-LLM models like the JEV decision model. We're seeing there's other things that AI can be. You can have multiple components that work together. It doesn't have to be inscrutable. It doesn't have to be uncontrollable. As that mindset shift happens, it's terrible news for those two companies and their investors, but really good news, I think, for the world as a whole, where we can say, wait a second, we have other ways to build useful AI tools. We don't like the way you guys are doing it. And you can't hold us hostage with this idea of AI is a singular technology, and either we move it forward or we don't, and we can slow the advances.

40:33It's not a singular technology. It's a hundred different ways you can build things. The way you're working skeeves us out. We're going to actually take you to court for it. And now there's other companies doing it other ways. So competition, technological disruption, different ways of thinking about AI. This is how I think the current moment is going to end. Not necessarily with a giant bang, but with a sort of whimper from those big companies as they realize that their vision for AI wasn't the only one and is one that's not necessarily going to win. Cal Newport, thank you so much for your time.

41:04Thanks for having me. Cal Newport is a professor of computer science at Georgetown and the author of nine books. All right, that is it for the show today. What Next TBD is produced by Evan Campbell, Madeline Thames-Ducharm, Patrick Fort, and Micah Phillips. Paige Osborne is the senior supervising producer for What Next and What Next TBD. Mia Lobel is the executive producer of podcasts here at Slate. And Ben Richmond is our senior director of podcast operations. You can find me on the internet and yell at me. It's at Lizzie O 'Reilly on kind of all the platforms. Thank you so much for joining us.

41:39Talk to you soon.

41:45500 orders a month was manageable. 5 ,000 is madness. Embrace intelligent order fulfillment with ShipStation, the only platform combining order management, warehouse workflows, inventory, returns, and analytics in one place. would use to take five separate tools ShipStation does in one. Automate fulfillment and rate shopping to ship up to 15 times more with up to 90 % savings. Go to ShipStation.com and use code AUDIO to try ShipStation free for 60 days. This message is brought to you by Cologuard. If you recorded songs off the radio for a mixtape, it might be time to screen for colon cancer. The Cologuard test is non-invasive, requires no special prep or time off, and you start screening right from your home.

42:30It's just three simple steps. Set up, sample, and ship. If you're 45 or older and at average risk, ask your doctor about the Cologuard test. Available by prescription only. Learn more or request a prescription at colaguard.com slash podcast.