The World of AI Regulation with Brian Merchant

0:00This is an iHeart Podcast.

0:34Well's Fargo announced a new$20 million program in 2025, teaming up with nonprofits to support small business owners. That's how Well's Fargo is helping strengthen small businesses and communities. Well's Fargo, the bank of doing. Learn more at wellsfargo.com slash say do. This is Mario Lopez with Mario Lopez Daily Podcast. Toyota is moving toward a more sustainable future by giving you the freedom to choose from a full line of hybrids, plug-in hybrids, regular gas, and 100 % electric vehicles, including stylish cars like the Camry and Corolla, as well as efficient and spacious SUVs like the RAV4 and Grand Highlander.

1:14All backed by Toyota's reputation for reliability. No pressure, just possibilities. Visit buyatoyota.com for a great deal on an efficient Toyota today. Toyota, let's go places.

1:54Ba-da-ba-ba-ba Call Zone Media Oink oink, welcome to the monologue this week, except it's a duologue, there's two of us. Better Offline, I'm your host at Zitron.

2:16Better Offline And today I'm joined by Brian Merchant, the author and writer of the book and newsletter, Blood in the Machine. Brian, thank you so much for joining me. Thanks for having me. Good to be back. So today we're talking about AI regulation, the current state of, well, California's regulatory moves, and indeed the very confusing situation. And I think we can start with this, actually, this AI safety bill that was vetoed and then just got signed. What happened there? Yeah. I mean, long story short is that the people who are actually worried about catastrophic risk, like the AI doomer crowd, the people who think that this is the number one thing to worry about with AI, they all got together last year and wrote this bill.

3:05last year it was 1047 and it was again, assuming that you are worried about that kind of thing, this was like a legitimate stab at trying to keep the companies honest it did things like mandated third party audits of the systems to make sure that they weren't too risky or that they weren't including information about biological weapons or whatever and teaching people how to. But the AI companies hated it because it mandated some sharing of information, some transparency. It mandated audits. No, no, no, no. No, no, no, no, no. Not even a little bit. Even the safe AI companies like Anthropic. Like Anthropic.

3:54Yeah, of course. Even those guys, they were against. So everybody was against it. It passed because Scott Wiener, the sponsor, is a fairly prominent and powerful legislator. But even though it passed both houses and was sitting on Newsom's desk, the industry, Silicon Valley, got Newsom to veto it, basically. So Newsom vetoed it. And then he said, let's try it again next year. So Scott Wiener got together with a bunch of AI ethics people. Oh, my God. Yeah. And then, so this year he comes back with like an even, like a watered down version of even that to, again, the aim is to prevent catastrophic risk.

4:41And yes. And he just signed this bill. So as we're recording this, you know, he signed it a few days ago. So it's on the books. And people are celebrating this thing for some reason. And I'm really scratching my head. Why are you not celebrating it? What's wrong with it? Okay, here's what the bill actually does. It does four things, basically. Instead of mandating actual transparency and sharing data with auditors, things that could actually maybe be considered a prescription if you're worried about things like catastrophic risk, One, the new bill enforces the tech companies to publish privacy protocols on a website.

5:29Oh. Yeah. Like a privacy policy. Yes. It says, this is what we're doing to ensure our security practices, right? Great. This is what we're going to do to keep you safe from catastrophic AI. No checkup policy, no committee or anything. No, no, no. You need a page. You need a page that people can see. No enforcement. That'll stop it. And then if something does go wrong and the companies have done something unsafe with their LLMs, they need to report it to the state. It's an honor system, so they have to tell. And what happens if they don't report it? Unclear. So nothing. Sick. Great stuff. Brilliant.

6:22Unclear. So the one thing that you could argue or two things, again, like in theory, these are fine things that it says. Yeah, they're just limp. Well, those first two things are a limp. I think they're a complete joke. But part three and four is whistleblower protections, which you're supposed to have, right? Good to formalize, but yeah. Yeah, it's just kind of like underlining it with a Sharpie saying, okay, people should be able to blow the whistle if something catastrophic is happening. Right. I have very little faith this will meaningfully change anything at all. I guess there are no protections actually in there.

7:06Is there anything that specifies? There's some language that does raise the threshold. Again, theoretically, but you just have to think about how whistleblowers are already treated and how difficult it is for them to come forward with all the NDAs and just sort of the norms in the industry. So theoretically, but again, and it's supposed to be for workers who are so concerned about catastrophic risk. Not about if they see a company doing fraud or whatever. This is for a catastrophic risk. So really no real protections. No teeth, really, no teeth. I was reading this thing, and I'm not seeing really any enforcement mechanisms other than these companies get a slap on the wrist, and maybe it's a little bit slappier if it comes through.

7:56But it doesn't sound like there are actual remedies or anything like that. Not really, no. So there's a, finally, the fourth thing that it does, again, in spirit, this is okay. Like, this is, it's a good idea that it's to start like a public consort, like called CalCompute, like a public alternative for AI for researchers. It says, like, let's give them. Yeah, but it's just a committee. It's a committee. Fuck yeah. Bureaucracy. It forms a consortium to discuss the initial stages. Together that we'll discuss what might happen in the future after another meeting or two. Brilliant. Once again. So my read on this whole, and I've seen a lot of statements coming out of, you know, otherwise decent sort of orgs saying like, this is a step for, you know, AI safety.

8:47And I know it's not. It's not really. It doesn't even seem like it addresses AI. It doesn't seem like it's an attempt to mitigate any harms. I mean, it is an attempt to, you could, I think that generally Which harms? Yeah Because the most, I'm going to guess, so in looking at AI regulation You have probably not seen anything that addresses the environmental damage Anything like, anything about the stealing Yeah, workplace surveillance, things like this So my read on this, right? So I think, again, I think Scott Wiener and sort of the group that are – like the AI safety groups, they believe in this stuff.

9:28They really lobbied for this. They think that it is important to have something better than nothing. But the effect of this is largely just that now Gavin Newsom gets to say, like, I signed an AI bill. I'm doing my due diligence. And meanwhile, there are a bunch of other bills that we're going to talk about that would actually do something, that would actually sort of meaningfully rein in the AI companies at least a little bit or at least point a way towards doing this. This is just, again, putting your safety policies on a website, like an honor system to alert the state if you've done something wrong, kind of just saying, okay, we'll be…

10:10for the most well-funded companies in the world. Again, and here's the number one way that you can tell that this is entirely bullshit. And that is that Anthropic is in favor of it. They came out and said like, this is good. Number two, Meta was like, this is sensible. And they didn't support the bill, but they're not opposed to it. Even OpenAI was like, well, I think you could tell they thought about it. It's any kind of restriction. We don't like that. Even if it's the flimsiest one, I feel like you could see them, the gears turning in the corporate machinery. Basically, this bill requires us to have an intern write some copy, or even just have ChatGPT generate it.

10:52That's the thing. Put it on a website. They don't even have a thing in there saying you can't use AI to write it. I haven't seen anything. I would have probably done that myself, personally. Yeah, so number one, it does next to nothing in my eye. And I just can't, I mean, I know all these advocacy groups and people who are trying to get some good laws on the big, they're hungry for a win. So they want to say, yeah, we did something. But to me, this is worse than nothing because it's going to let Gavin off the hook on a handful of bills that might actually do something. And it will give the fake view that the AI companies are regulated, so they're able to continue doing all the actual harmful stuff.

11:43So I'm a big fan of Quince. I've been shopping with them long before they advertised with the show, and I just picked up a bunch of their Pima Cotton t-shirts after they came back in stock, as well as another overshirt, because I love to wear them like a jacket over a t-shirt. Talking of jackets, I'm planning to pick up one of their new leather racer jackets very, very soon. Their clothes fit well, they fall nicely on the body, and feel high quality, like you get at a big, nice department store, except they're a lot cheaper because Quince is direct-to-consumer. And that's part of what makes Quince different.

12:09They partner directly with ethical factories and skip the middlemen, so you get top fabrics and craftsmanship at half the price of similar brands, and they ship quickly too. I highly recommend them, and we'll be giving them money in the future. Layer up this fall with pieces that feel as good as they look. Go to quince.com slash better for free shipping on your order and 365 day returns. Now available in Canada too. That's q-u-i-n-c-e dot com slash better. Free shipping and 365 day returns. Quince.com slash better. Still using a copy paste website? Break the template trap with Framer. Whether you're overwhelmed by traditional site builders or frustrated with cookie cutter designs, Framer gives you the freedom to create a site that's professional, polished, and uniquely yours.

12:54Hit publish and Framer ships your site worldwide in seconds. No code, no compromises. While DIY website tools are everywhere, most fall short on design and performance. Framer changes that with a powerful, user-friendly platform that delivers developer-level results without writing a single line of code. Framer is the design-first, no-code website builder that lets anyone ship a production-ready site in minutes. It's free to start Browse hundreds of pixel perfect templates Or designed from a totally blank canvas With multiplayer collaboration Allowing your writer, designer and marketer All to tweak the same page at once No version control nightmares Ready to build a site that looks hand coded Without hiring a developer?

13:30Launch your site for free at Framer.com And use code OFFLINE To get your first month of pro on the house That's Framer.com promo code OFFLINE Framer.com promo code OFFLINE Rules and restrictions may apply This is Mario Lopez with Mario Lopez Daily Podcast. Toyota is moving toward a more sustainable future by giving you the freedom to choose from a full lineup of hybrids, plug-in hybrids, regular gas, and 100 % electric vehicles. So, pick from stylish cars like the Camry, Corolla, and Prius, as well as efficient and spacious SUVs like the RAV4 and Grand Highlander, all backed by Toyota's reputation for reliability.

14:07In fact, for 2025, Toyota is the best resale value brand for the fifth year in a row, according to Kelly Blue Books, KBB.com. So a smart buy today can mean higher trade in values tomorrow. With Toyota, you got more choices and less compromise. Visit buyatoyota.com for a great deal on an efficient Toyota today. Vehicle's projected resale value is specific to the 2025 model year. For more info, visit Kelly Blue Books, KBB.com, Toyota. Let's go places. Hey, Ryan Reynolds here from Mint Mobile. Now, I don't know if you've heard, but Mint's premium wireless is$15 a month. But I'd like to offer one other perk.

14:46We have no stores. That means no small talk. Crazy weather we're having. No, it's not. It's just weather. It is an introvert's dream. Give it a try at mintmobile.com slash switch. Upfront payment of$45 for three-month plan,$15 per month equivalent required. New customer offer first three months only. Then full price plan options available. Taxes and fees extra.

15:11Are there any regulations you've seen that actually approach the actual harms? Or sorry, suggested regulations or bills or anything like that? Okay, yeah, there's a couple. So, and I've been talking to, I've talked to lawmakers over this process. I've talked to some of the sponsors of the bills and I've talked to labor groups. And the one sort of through line that runs through all of this is that Silicon Valley and its lobbyists have just been out and forced trying to crush even the most sort of basic common sense regulation. And so laws or proposals that started out with some teeth have had most of them knocked out or have been delayed until next session.

15:58So we're left with a few things. We're left with a few things that, you know, I think more than anything, they're just like bellwethers as to whether or not it's even possible to get anything done. Because as your listeners will know, right, like the whole AI for enterprise isn't working out so well right now. No one buys it. No one's buying it. No one wants it. So they're going to have to make some changes in the next couple of years. And the way they're going to, my guess is that they're going to try to find ways to sell people on its other capacities. Things like it's a surveillance tool. It's a terrible surveillance tool.

16:43That's what's funny about this because it's like the actual harms to mitigate would be training and environmental and energy. The number must go up. So it's like, we'll regulate. Because the surveillance thing, I get that. There is already AI surveillance, but it's like, oh, what if they put all the data in a large language model? Yeah. Would the large language model do anything with it? We've already got that. But wait, so what other bills have you seen? Okay, so there's one. I think the one that's still out there at the time of recording, who knows? It could have been vetoed or signed by the time that this goes to air.

17:23but there's one there's there's one bill that Silicon Valley is genuinely upset about and afraid of and it's a bill that like sets the very lowest bar and it says essentially if you are going to make a chatbot and market it to children then you have to be able to demonstrate that this chatbot isn't going to make them harm themselves ah so so they hate that They are, and they're in, you know, they have, there's this Silicon Valley lobbying group that's kind of famous in California, especially called the Chamber of Progress. It's like, yeah. Sorry. Sorry. Just like immediate reflexive. Immediately.

18:09I know. Gag reflex immediately. So they've got this guy who's out there writing op-eds in like the San Diego Tribune and doing press for, going, oh, it's overbroad. Their actual line on this is, if this bill passes, then you're going to be taking away AI that could educate children. You're going to be taking away AI from children, and they're not going to have the same advantages that children with AI have. And they're running this big Facebook campaign. They've hired lobbyists specifically. They don't often do this. These are fucking evil people. They are evil people. I mean, for me, this was like, you know, this was like, it's past the threshold.

18:54Once the Adam Rain stuff broke and OpenAI is, you know, trying to hem and haw about, you know, oh, well, you know, we're going to do this or that. We passed the Rubicon, right? They've got chatbots that are telling children to, you know, hide the news so that their parents don't see it. Yeah. And like, again, they make it seem like, oh, AI is this frontier. We're going to work something. It's a product. It's a software product. Yeah, go ahead. I have a theory. I have a theory. So I don't think they can control these models. I don't mean because they're intelligent. I don't mean because they're autonomous.

19:34I mean, I don't think you can actually prompt a large language model to categorically stop it doing something. I don't think it's possible. Then you should not be selling that piece of software to children. Yeah. 100 % agree. I'm just saying that I don't think they're capable. I think you're right. Yeah. But my theory is based on costs because of Claude Code, that they can't do cost control. If they can't do cost control, it means the model won't listen. Yeah. And I reckon that they can't be like, never talk about killing yourself. Yeah. Like they just can't do that. Yeah. Or it would require like going back through, you know, like there's been i've seen a lot of sort of speculation that the reason that it's talking like this is that it's like a lot of the language is coming from like pro-suicide forums in the bowels of the internet so like they don't want to go through and take the time discussions to sort that out yeah or discussions of suicide probably articles as well that say how to deal with someone who's just horrible stuff yeah really fucking people and this is this a california bill yeah It's a California bill.

20:39And it is, yeah. So if you're on Facebook, so there's a front group spun up by some of the VC firms like A16Z and Andreessen and these guys and Y Combinator. And there's a front group called the American Innovators Network. And they're running all these ads arguing that this bill, again, whose sole purpose is to ban chatbots from being marketed to children that also try to convince them to harm themselves. Not even banning them from being marketed to children, just stopping them being harmful. The way that it's free, which is what they're all hung up on, is it's just like, you have to be able to demonstrate that if you sell this to children, it will not tell them to harm themselves.

21:29One of the more reasonable requests you could ever ask of a company. The most, I cannot think of something that is more but they're saying oh no it's too broad everything else will get caught up in this you can't have a chatbot because but and then it's always like but why why couldn't we have this chatbot in the classroom yeah whoa because it might tell somebody to kill themselves like that's why yeah and they they don't this is again we should these people should be we should try and interview them like because it's the question would be say that you agreed with this bill you disagree so you agree with it how would you stop this yeah just how would you stop it can you stop it is that why you're upset is the is the reason you're mad because you literally can't stop this yeah because that's the thing they ham on har around any kind of protections yeah anything anything and now and i wonder if it's because they they can't yeah they want to be like oh it's too powerful no it's too shit yeah it's because it sucks it's not because it's it's not because it's because you built something shitty and hard to control exactly it's like lava lava is hot that and can burn through most things that doesn't make it intelligent right your inability to not drink lava just like fucking yeah i mean it's a combination of both those things like like it's either oh this would be really expensive to fix or oh like i don't know if we can't i i reckon what they would have to do is they would have to just neuter it they would just have to they would have to just make it so that anything that gets even close to that conversation yeah would have to be just like shut down to the point that you can't even talk about superhero stuff right they would probably just limit it to the point of nothingness yeah which like yeah i mean if that's what they, I mean, we're again, we're talking about like children here.

23:22So like, yes, that seems like if that's what they have to do, then, then that in, in my view, anyways, is something that they should do.

23:41This is Mario Lopez with Mario Lopez daily podcast. Toyota is moving toward a more sustainable future by giving you the freedom to choose from a full lineup of hybrids, plug-in hybrids, regular gas, and 100 % electric vehicles, including stylish cars like the Camry and Corolla, as well as efficient and spacious SUVs like the RAV4 and Grand Highlander, all backed by Toyota's reputation for reliability. No pressure, just possibilities. Visit buyatoyota.com for a great deal on an efficient Toyota today. Toyota, let's go places. As New York cools off and I begin to actually going outside again, I'm always happy to pay for a sandwich at Court Street Grocers or a coffee from Birdie in Williamsburg with Square.

24:23The tech just works. It's easy to set up for businesses, giving them instant access to earnings with Square Banking, built-in marketing and loyalty tools, and powers over 100 ,000 different food and beverage sellers, from boutique bakeries to national coffee chains. It's incredibly easy to use for customers and businesses alike, and you don't need any special training or elaborate setup. And in my personal experience, I don't think I've ever had trouble with a Square terminal. You just tap it and go, or you slide a credit card in. In person it works, online it works. I just really like Square, and I'm happy that they're advertising with the show.

24:57Square keeps you up so you don't have to slow down. Get everything you need to run and grow your business without any long-term commitments. And why wait? Right now, you can get up to$200 off Square hardware at square.com slash go slash better offline. That's S-Q-U-A-R-E dot com slash G-O slash better offline. Run your business smarter with Square. Get started today. Morning people wake up for peace and quiet. McDonald's breakfast people, we wake up for the sweet rush of getting that warm, delicious breakfast right before it ends. With McValue at McDonald's, get any size pumpkin spice iced coffee for just 99 cents until 11 a.m.

25:38And try our pumpkin and cream pie. They're spiced and priced, so very nice. Price and participation may vary. Cannot be combined with any other offer. Residents at Brightview Senior Living Communities enjoy enhanced possibilities, independence, and choice. Brightview, Dulles Corner, and Great Falls offer vibrant senior independent living, assisted living, and memory care services through various daily programs, chef-prepared meals, safety and security, transportation resort style amenities and high quality care take financial possession of your apartment by december 31st 2025 and save discover more at brightviewseniorliving.com equal housing opportunity

26:23but again it's wild because like they don't usually hire lobbyists to oppose state level bills but the right the sponsor here uh rebecca bauer khan it's it's the lead act is is what it's called it's ab 1064 she's just like i've never seen anything like this they're like having lobbyists come and knock on my door and like you know yelling at me about this um and they're just like it's a full court press and we could talk a little bit after this about like how this is sort of like a part of a broader movement where they're you know silicon valley is sending its lobbyists out all across the country, but especially concentrated.

27:04Is this atypical to do on this level? I mean, Silicon Valley hasn't wanted regulation on anything for a long time, but the level of concentrated effort and sort of the campaigning is new, right? Like I've been a tech journalist for 15 years, and I've never seen anything like we saw over the summer where they where silicon valley tried to lobby for a ban on all state level ai lawmaking they they really got a whole sort of united front together they got stakeholders from the different fail though they did fail but they failed by basically one vote and they failed because of a republican in uh tennessee who represents nashville uh yeah and she was like wait a minute we have a law in the books that protects our country music industry from being slopified and would this overturn it?

28:00And they were like, well, yeah, but, you know, yeah, yeah. And then so she kind of came out against it. And if it wasn't for that, it would have, well, it would have passed. That is bonkers. It's bonkers. That's wild that country music saved us. Yeah, country music and Nashville's no AI music slop law. And they're going to try again, though. They're going to try again. And Ted Cruz keeps talking about it. He's interested in giving it another go. So it'll be coming down the pike. But to be clear, no, this has never happened. If you've never seen somebody say, okay, we're going to ban lawmaking around search.

28:38No search engines can have any laws made around them or social networks. This is its own thing. This is AI. Yeah. There just can't be laws. No, it's totally anti-democratic. It's totally absurd. And it tells you all you need to know, really, that the Silicon Valley interests are willing to push all their chips in and team up with the Trump administration and its allies to try to get this done. I mean, I don't know, fucking Newsom. No, he's – I mean, Newsom has vetoed a ton of solid AI bill. I mean, last year, Newsom vetoed a good bill that I think was good because it was on – so I don't know if you caught this.