AI and the Left

0:11Hey guys, it's Andrea Dominic and Emily Friedlander here with Cujo, a podcast about culture in the age of platforms. And today we are here with writer, researcher, technologist, and founder of the new platform All Models Are Wrong, Josh Davila. Welcome, Josh. Hi, thanks for having me. And we're having Josh on today to talk about this particular moment in which everyone seems to be freaking out about AI right now, especially if you are in the US. We'll also get into whether the total AI takeover is as inevitable as Silicon Valley's James Day language is making it seem, what regulating AI actually means or looks like, and the left's relationship to AI.

0:55So Josh, let's get into it. This is just a question for all three of us because we've been following this conversation, this situation in different ways. Why is everyone freaking out about AI right now? Yeah, I mean, at least in my view, there's like a couple reasons why. We're talking about existential risk around AI that has kind of come up a lot, I think in part because the IPOs of all these different of like at least Anthropic is coming up and I think a couple of others. And then I think the improvements of AI have gotten to a certain point where like all these weird niche conversations from effective altruists and rationalists, the only places where there was discussion about something like this or just kind of like pushed this weird niche thing that they were into into the limelight.

1:39Right. There might be a little context collapse going on where there's these conversations that have been happening behind closed doors in this specific world hitting the mainstream. And we don't even we don't understand what they're even talking about or we can't we can't get a read on it. Yeah, there's been like a kind of a spate of sort of public facing events related to AI that I think has also kicked off some of this, for lack of a better term, hysteria that's maybe been bubbling beneath the surface. So let's just break down a little bit like what's happened over the summer and the recent months, you know, both in the tech world and as part of some of the social media conversation that's maybe the peg to some of this aforementioned freak out.

2:17Yeah, I'll give a brief summary for people who haven't been following along and are wondering what we're even talking about. So a number of the like big AI companies have been experimenting with AI agents and conducting studies, usually in like sandbox environments, like safe, secure environments to see how they interact under different conditions or how they'll respond. And so OpenAI was doing a test on some agents and this group of agents, like an agent swarm, broke out of the secure environment in order to complete a task, which they weren't told to do. And then basically hacked the infrastructure of this other open source AI organization called Hugging Face, which is recently sold to NVIDIA.

3:15But they hacked into it. And then I think they kind of just went rogue. And there was like really interesting and strange behavior involved in this where apparently they coordinated with this elaborate series of like tens of thousands of messages. and they even like sacrificed some of themselves for the greater good of the project or like agents with fewer tokens left would be assigned more risky tasks or would like pass on instructions to others before they died, which is really crazy in science fiction. But basically they hacked into Hugging Face, Hugging Face contained the attack and then there's some conversation of like another part of the story where they then went back and tried to attack open AI.

4:01infrastructure. But basically, it happened over the summer and people in tech circles were talking about it, but there wasn't a ton of conversation about it. And like, it didn't break contain. And then it broke contain last week, basically. And then this guy named Jacob Coxon, who was a employee at Anthropic, who had also previously worked at OpenAI, did this very like public resignation from Anthropic where he was raising concerns about these technologies being super dangerous, specifically this idea of recursive self-improving AI that could escape. Once AI is able to train itself, essentially, it becomes harder and harder to control.

4:45And I don't know if he used the term extinction level event, but he was saying there could be this catastrophic consequence very soon of the development of these technologies not being slowed down. Then another guy, I think, from Anthropic was like, oh, yeah, I put the chance of a catastrophic event at like 10 percent in the next few years. So that's kind of why people are freaking out. And people obviously who are outside of this world are hearing terms like extinction level event. And they are like, what? And they also don't understand it because they're like, why are you continuing to want to work on this technology, if you think it's going to kill us all, it's really hard to read.

5:25No, yeah, definitely. I mean, it's definitely confusing. Yeah, it's kind of like a thing where for some people, especially on the outside, if you think it was going to kill humanity, wouldn't your behavior be a little bit different in some ways? If it doesn't happen, they also stand to potentially make a lot of money or make a career out of it or whatever it is. So it's really hard to judge. Are they sincere about what they feel the risks are? And then there's this other layer that I should have mentioned just within the story where amid this uproar, all this hysteria, Dario Amodai from Anthropic comes out with an article called We Must Pace the Frontier and kind of suggesting a path forward where he proposes a plan that is, I guess, getting an independent body to do oversight of the work that they are doing in-house.

6:19and having all of the major AI companies, at least in America, at first agree to have these people with like employee level access come in and make sure that they're that's what they when they say we must pace the AI, the idea is to have these people come in and make sure that they're proceeding and following proper safety precautions. And then eventually with a plan to extend this to like other, quote unquote, democratic countries. And then once the democratic countries accept it to quote unquote authoritarian countries, which is China. The implication is it's probably China. And then all the Elon and Sam Altman and other AI CEOs signed on.

7:00And that's kind of where we're at now. Right. Yeah. And it's worth noting that he's not suggesting let's just hit pause on this. And it's where a lot of like sort of tensions or controversies are maybe arising. he's saying let's slow down effectively so that the internal regulators we put in have enough time to keep up with the pace at which this is developing which is makes one wonder well why weren't you just doing that in the first place but then so that's the sort of like slow down thing on the table right now and then on the other on the full pause or full stop side of thing you also have a few pieces of legislation that have been introduced here in the U.S.

7:37on being put forth by Bernie Sanders and Greg Cesar. The legislation they put forth with jail scientists who work on superintelligence would terms it up to 20 years. There's also the AI Kill Switch Act, which was proposed by Ted Lieu and Nathaniel Moran. And that would require AI companies to build their systems so that they can be shut down on short notice, which raises a whole other set of questions about consideration to what enforcing these kinds of measures would actually entail. Sanders and Cesar's legislation would establish a new cabinet-level federal agency that's dedicated to AI safety, which then presumably would be surveilling the work of computer scientists.

8:18This nonprofit, the AI Futures Project, also published this thing called AI 2040, which is a scenario accomplished through the use of these hypothetical but physical verification devices that would be installed in data centers to detect whether or not computers are actually training new AIs. So that's the sort of other end of things of what's cooking amid this whole conversation in terms of policy, too. Yeah. Maybe some of this will happen in some form. Probably a lot of it is just throwing ideas, initial ideas, you know, at the wall and seeing what sticks. Right. Yeah. Like, I think at least as far as the like Bernie's proposal and his bill, what I think is interesting is I don't think they did it as a in a serious way thinking that it would become legislation.

9:01it feels more to me like they're doing it to start some sort of conversation because they do feel I think the incentives are fairly clear for Bernie it's like if this is something that hurts working class people that's his thing he's going to like find the thing that's going to protect working class people at least in you know in theory so I think it's like not surprising what's been interesting to me I think is that after that there was a lot of backlash against Bernie I feel like on the left about it, that like him doing that kind of gives credence to the worldview of these effective altruists and rationalists, Eliezer Yudkowsky, I think like maybe specifically, you can like point as like the most extreme version of that, where he wants like a, he proposes a complete pause on AI, because the creation of a super intelligence would cause the extinction of humanity.

9:50I don't know that that's like what Bernie specifically thinks. But I think the positioning of the legislation and how it's written kind of feels like associated with that and gives credence to that worldview. Right. So basically the left, certain subsets of the left are pissed at him for even proposing legislation that allows AI to continue existing effectively. I guess there are parts of the left who think that regulating AI then gives it legitimacy. Exactly. It's like a form of regulatory capture at the same time that like something like that would simply solidify the position of open AI or Anthropic as like the.

10:26To regulate is to condone. Because he did that like that one video he did a while ago where he was talking to the Anthropic or to Claude like on the phone. Like there's a lot of also a lot of people mad about that because that felt like he was then like I think there's a certain part of the left that is like absolutely not. AI is good for nothing. don't even use it you know there's this side of it that like to show a publicly interaction with it as like that it was like a faux pas of some sort right um and then there's like different positions of like do we regulate ai therefore the fewer companies that there are that are doing this the easier it is to regulate and keep control of it or is it that ai should be made open source and publicly available and that general intelligence is like a right to people in the same way that like water coming to your sink is like a right is that something in the future to come right like a utility perspective on it yeah we did an episode a couple of years ago with james muldoon who wrote a book about platform socialism i'm imagining that this sort of proposal would fit within that world he was talking about social media platforms more or like web2 platforms but public ownership, which some countries have been pursuing to some degree as we're running experiments in.

11:45So sort of to get back to where we started, there's so much to hit on here. But when people are saying that AI is advancing too quickly, what exactly is happening too quickly? Obviously, we've just established that some people on the left are like, it happening at all is happening too quickly or too much. But in the context of this specific conversation, What is happening too quickly? Should people actually be freaking out? Or what are you worried about personally? I mean, I think going too fast and far as like capability, I guess. The thing is, it's a bit of a roll of the dice, it feels to me sometimes.

12:25Like there's a possibility that an AI could make some sort of whatever biological weapon that, you know, can wipe out humanity. I think there's like another bifurcation also on this one is like, can an AI do that? Would the AI choose to eliminate humanity because it's super intelligence? There's like kind of this assumption, I think, in a lot of it of like, if something is more intelligent than humans generally, that they would then dominate like that, that being would choose to dominate, which also I, I personally don't agree that just because something is more intelligent doesn't mean they would choose to dominate.

13:03I feel like that's more of like a kind of projection of maybe how humans have treated other humans, perhaps. And then the other side of that bifurcation would be that someone who is bad, who is like a human would take it and then choose to do that. And that's the thing that would like cause some sort of human extinction event. I personally don't really know where I stand on. I feel like I would just have to spend a lot more time learning about how AI would help in the creation of biological material. I know that there are regulations within DNA sequencing machines. There are known DNA sequences that we know would be bad for humans if they were put out.

13:47And the machine is built so that it can't produce those things. So there are these things that already exist. I don't know how easy it is to get around it or what. But for me, I'm much more concerned about like the more immediate risks of like, I mean, it's already been happening, like excuse for laying people off tremendously throughout the tech world. And it's like hollowing out of knowledge workers and like the middle class, so to say, of people who did knowledge work is now like potentially in danger. I still think that AI doesn't really have, it's very jagged, you know, as they all say, in terms of like its capabilities.

14:23Something is really good at something that's not so good at something very obvious. but like at the end of the day, it's like humans that need to buy things. So will we need to rehire people again so that they can actually fix all the mistakes that the AI did? Or will we all just like kind of choose to live with the slop, so to say, in one way or another? I mean, I'm much more concerned about the creation of inequality of this thing. And I think that the financial bubble, I think also plays in this, that whether or not the AI is useful, it's creating this tremendous amount of inequality in a world where if it is true, then more and more wealth will go purely to capital than it will to labor.

15:01And we already live in a world that's like fairly unequal. So it kind of like then destroys a lot of fundamental assumptions about what our relationship with the state, what does the state provide, what types of services should be available to people and like who funds them. So it's like more of my, what I'm more afraid of. And then I think as well, like cybersecurity risk. If you can just put an AI to like find bugs everywhere all the time, that's probably not a good thing. The Culture Journalist is an independent project that relies on word of mouth and is funded entirely by listeners. If this pod had something meaningful to your life and you want to access everything we put out, please consider supporting our work by becoming a paid subscriber over at our sub stack.

15:45Paid subscribers receive access to our Discord community, the Cujoplex, full access to bonus episodes, and more goodies. You can get it all for just five bucks a month over at theculturejournalist.substack.com. Going back to the economic piece, there's like definitely conversation about like, I think the prospect of mass unemployment is still up in the air about whether that will actually occur. Some people think it's a possibility. Some people think it's actually not going to happen or maybe only a small percentage of people will experience unemployment. maybe a larger percentage of people will have their work changed pivot to a new field but if mass unemployment were to occur there's some conversation about like how would the economy even function if nobody has any money to buy any goods anyway like if a lot of people are out of work or wages are severely depressed how is this promise of infinite abundance going to even like It makes sense.

16:46Yeah. Because sure, like probably the ownership class will extract more and more of the money. But if most of the population has very little money, the economy would kind of collapse anyway because nobody would be able to purchase any of the goods that anybody sells. And so there's also conversations about like, do the companies need to getting rid of labor, laying people off, depressing wages is in their short term economic interest. But in the long term, that won't be in their interest because who's going to buy the stuff that they make? Yeah, basically. Cal Newport wrote a op-ed piece recently for The New York Times.

17:25It's called How Scared Should We Be of AI Right Now? And he kind of summarizes this really well. And they're in a couple of lines that I just want to flag where he says this style of anxious talk also allows AI companies to portray themselves as the reluctant stewards of an inevitable technology, which is a framing they can use to deflect blame for damage caused by their own products. To extend an analogy I've made before, if Ford automobiles were spontaneously catching on fire, we wouldn't tolerate the chief executives changing the subject to discuss our general lack of preparedness for a future with flying cars.

17:57We demand that the company address the acute problem happening right now. I thought that was just like so spot on. Oh yeah, I agree with that. It was a really well written piece. The piece is also talking about specifically the fearful language that Silicon Valley uses. I think it came out prior to Jacob Cox's resignation. Yeah, like just before, I think it came out on September 3rd. Yeah. But this sort of like fearful language that it uses and it connects that language to the popularity of rationalism in these circles. And I'm wondering, like when you listen to people like Jacob Coxon or, you know, executives of these companies talk, are they thinking rationally and rationally predicting real scenarios?

18:46Or do you feel like they're thinking more irrationally? Do you think that these predictions are grounded in science or is it more grounded in like a belief system that may or may not? I think you can debate whether rationalism is really grounded purely in science or if it almost is like a subjective belief system in and of itself. Yeah. Rationalism, by the way, is just a certain way of thinking about the quote unquote inevitability of AI and how it's dooming humanity that people have a lot of opinions on, but that has been very, very absorbed by Silicon Valley and has really reshaped their language and the way they talk and think and think about and approach AI in a way that once you understand that you sort of like read their tweets and statements they're making in a way that's completely different.

19:36so yeah i think i mean for me i feel like it relies on a lot of what ifs it's kind of like ifs kind of stacked on top each other of each other like if this then this and then if this then this and then it kind of like like within its own worldview of course it makes sense if you're surrounded by rationalism and like surrounded by the types of things that they're interested in and talking about is what it comes off to me i mean it's definitely something that like for a long time I've been kind of turned off by rationalism. I was never really attracted to it. Right. This is a capital R rationalism, by the way.

20:12It's not like rationalism is in terms of just rational thinking. This is like a way of thinking. It's tons of books. And they think they're thinking really rationally or that's part of it. That's what they believe. Yes. I mean, I'm pro sometimes being a little bit irrational, but it depends on the situation, I guess. So yeah, I think to me, I've been kind of in my head, I've been kind of comparing it to a type of like scientism as well in some sort of like taking, I think there is like scientism in like thinking of science almost like in through a moral or like spiritual lens in a sense, or just like, like as a way to justify what is the things that you ultimately want to do.

20:53Kind of like, I think scientism is like a, we should know things that we don't know type of thing. And I think rationalists, they take it in a kind of slightly different direction. Because what's interesting is that Yudkowsky comes from, from my understanding, he comes from extropianism. And extropianism is another weird niche ideology that in a different world, we wouldn't have to care about. But it's like this kind of worldview that every individual has a right to extend the ability of their body through technological means, however way that is. It's really it's also one of the underpinnings of a lot of the like crypto philosophy as well.

21:31But Eliza Yucatsui came out of this, is my understanding. And then he's the founder of rationalism. Exactly. And he was like, actually, these people are not taking the risks of AI seriously enough. So I think he kind of like already was coming from a world where an imaginary that like AI is going to become this like artificial super intelligence, like sci-fi from the 80s and 90s type of thing. It inherits a lot of extropian assumptions about how technology will develop and progress. And at the same time, rationalism is not the only niche ideology that has been thinking about this actually for a long time.

22:12There's also the CCRU, if you want to go into that rabbit hole. But yeah, my feeling is that without reading all of their fundamental texts, I guess, the impression that I get is that they're on a very wobbly tower of ifs, like if things happen and if things progress in a particular way. So they might have calculated or some of them might have performed calculations that there's a 10 % chance of this event happening, but it's a theory more than probably something they've proved. Yeah. And it sometimes feels kind of like, at least for me, it feels like self-fulfilling in a way. Yeah, it's very like Schrodinger's cat, you know, it's a Schrodinger's cat approach to like the doomsday scenario of AI.

22:51And I'm also not like definitely not the first person to point this out. It's probably obvious to people who have already been following this stuff for some time. But the problem also with focusing on these sort of extreme doomsday scenarios is that it sort of redirects the conversation to that rather than like more immediate harms that we can maybe already see happening right now or actually very soon. likely to happen. Like, yes, there's definitely proof that people are laying people off and saying they can be replaced by AI. Whether or not it's true that they can be replaced is debatable, but that's already happening.

23:35But obviously when you bring in these like big doomsday risks, that's what the brain focuses on. Even if it's like not grounded in reality, it's, you know, you can't look away. And the media, who also has some trouble sometimes catching up with AI literacy, they'll obviously grab onto that because they got a report on like, you know, the biggest story. And it seems like the biggest story, but it's unclear whether it's actually real. And so that seemed, we seem like we're in this loop right now. And it also seems kind of like, it kind of like allows the AI companies to be the ones to kind of tell us where this needs to go.

24:15Like I even was watching just like some kind of I think it was an interview with Jacob Cox and and one of the mainstream news networks. I can't remember if it was CNN where they immediately shifted the conversation to, oh, and open models are dangerous, right? Like we need safety. And one of the things that is dangerous is open models. And then the anchor obviously has no idea about this. And so he's like, oh, okay, so open models are really bad, right? When it's like, that's definitely not true. It's definitely not true that you can't regulate open models and keep them safe. But it works in the interests of these larger companies that kind of want to capture as much of the market as possible to say open models bad, private black box models good.

25:05And it also serves to sort of ground this in a there's the China of it all that we should talk about where our models are good and the way we do it is safe like we're the only ones who can keep you safe and you know we need more and more investment and we need to say how we're regulated or be regulated only in the way we want because we need to keep up with China who are the dangerous enemy and how they're building AI and just one more note about rationalism too it's easy to understand And why with everything you just said to why it's appealing to this tech cohort, it's appealing to them because by default, like looking at this AI doomsday scenario as inevitable, therefore makes the coders and programmers and executives who are helming this stuff like the saviors, right?

25:55So given certain personality types that gravitate towards that world anyway, it fits within sort of like a larger schema review of the world that we see how they operate. I would argue almost like a religious worldview. Where they are, you know, that is their sense of being a virtuous person. Yeah, like they're going to, AI is coming to destroy us. It's going to happen either way. So we've got to be the ones just develop it and stop it and control it. When of course it's like not playing out that way at all. Where is it? For me, it definitely reads as like a very, it's very American exceptionalist type of thing.

26:28Like only we can do it. We are the leaders of the free world type of thing. But it also coincides with Chinese open source models catching up to a lot of these labs. I think this is another factor that maybe we forgot in the beginning that I think also is bringing this up, bubbling up to the surface so much is because the Chinese models are also really good. They're doing approximately as well as some of the best models from Anthropic and Open AI. So they're like, this gap that used to be a lot bigger is just like noticeably smaller. And then you have, I think this division in the American capitalist class as well of like, do we have loyalty to American AI companies just because they're American and they're safer, but are much more expensive?

27:11Or do we just, it's just as good. Let's just use the Chinese open source models because they're open source and we can run it ourselves. And so like, in a sense, and this is like a case where at least I feel like if you take a neutral view at it, the Chinese side is a much more open ecosystem that gives people a place to use the tools versus the the American ones. Yeah, let's talk a little bit about that and compare and contrast to the American and Chinese AI industries a little bit. How are they doing things differently, both in terms of AI development and regulation? I know that, for example, you're referring to this, but instead of having all these different huge private companies who are developing their tech in a black box and kind of individually using all of this compute and resources and money to build the models, they will often publish their findings or make it open source so that companies can take what the others have built and then iterate on that.

28:12And that this has allowed them to sort of not repeat the research and build, in some cases, more cheaply. Right? Yeah, for sure. I mean, I think that's part of it. I think the other part of it is that like their strategy in China and like from Chinese companies, I feel like is more so to allow any company, anyone else to use their particular model in their own workflow as they want with complete control of how they would use it and therefore would be coming back to the same model or to that company for other types of services. Whereas they still offer, as far as I understand, like similar services as OpenAI and Anthropic like towards enterprises or like where you buy a specific subscription for their models.

28:57But they allow way more flexibility in terms of like the capture of you as a user. Like the American ones are very clearly their business model is to capture the user to be contained in their ecosystem. And so there's like kind of like two, there's this push and pull. So like there's this weird positioning where the more free market types often are very pro the open source models. And then the AI safety ones are much more pro these companies having custodianship and being regulated in a way so that we prevent these certain risks. But in China, you don't really have, based on my research, there are no Chinese rationalists.