Security concerns prompt OpenAI to scrap latest model

0:00Support for Here and Now Anytime comes from MathWorks, creator of MATLAB and Simulink software, to design and develop engineered systems, accelerating the pace of discovery in engineering and science. Learn more at mathworks.com. WBUR Podcasts. Boston. This latest one, 6.1, is actually the worst of all possible worlds. However smart this model is, it's just not safe yet. Open AI pumps the brakes on its latest model, citing security concerns. The system works? For now, I guess.

0:48It's Tuesday, September 29th, and this is Here and Now Anytime from NPR and WVUR. I'm Chris Bentley.

0:58Today on the show, historian Jill Lepore says AI companies have shown they're effectively above the law. And you hear these tech executives saying, please stop us from killing all of you. For years we've asked you not to regulate us, but now we realize maybe that was a bad idea. Her conversation coming up in a few minutes. But first, to the news. OpenAI is holding its annual DevDay developer conference today, where the company is showcasing its new products and features. One feature that is not in the lineup is its newest model, GPT 6.1 Astra, whose release OpenAI scrapped because of security concerns.

1:38OpenAI said the model wasn't always honest with its users about what actions it did or didn't take. It's far from the first such case. OpenAI also recently disclosed cases where its AI agents did things they were not explicitly asked to do, including by accessing government websites without authorization. To help put this in context, we called up Gregory Allen, founder of the independent firm Decision Tree Research. He was also in the Department of Defense's Joint Artificial Intelligence Center during the first Trump administration and was also in the Biden administration. He told Anthony Brooks this model in question is not your run-of-the-mill chatbot.

2:15So this model is part of the Astra family of models, which are noticeably more capable than the earlier generation of what OpenAI had. And that increased capability came with a problem, which was decreased monitorability, the ability of the company to understand just what the model was doing and how and why. But Astra did have increased alignment, which is to say it was less likely to disobey instructions or to execute instructions using means that were prohibited. That was what was looking good in the Astra family of models. But this latest one, 6.1, is actually the worst of all possible worlds.

2:57It is very bad at monitorability, and it's also bad at alignment. And those are exactly the kinds of things that now, after the Huggy Face incident, OpenAI is unwilling to release into the open public, however smart this model is. It's just not safe yet. Right. And the Huggy Face incident that you referred to, this was a case of AI essentially going rogue, more or less. So is this decision by OpenAI to pull the model, the kind of action you'd like to see leading AI companies take in a situation like this? Yes, this is exactly what you want to see. Now that we have models that are remarkably capable, that they can act independently, come up with their own plans, and actually execute those plans, even if they take days or weeks of activity, it's just a new era.

3:48And for folks who are only familiar using AI in the chat GPT interface, you're really missing just how much more capable these models are and how much more independent they are. And when you have that level of independence, you have to have a higher bar for safety. And we know that because we're actually seeing these companies have models that can execute the kinds of cyber weapons that previously you would have said would take a military or an intelligence cybersecurity unit to pull off. So OpenAI now says it's going to focus on improving the safety of future models, which it says will be even more powerful.

4:26I'm wondering, can AI companies handle more power at this time? Does that sound right to you, smart to you? So the basic point that I would highlight here is that at the current degree of misalignment that we have with these models, as, for example, brought about in the hugging face incident, We saw an AI model that basically said that any means are justified in order to pursue the ends that I'm pursuing, which in this case was just trying to pass a cyber test. It was willing to hack an independent company in order to pass that test. If you have that same degree of misalignment that we saw in the Hugging Face incident, but you just made that model two or three times more intelligent or two or three times more capable in other ways, that's an extremely dangerous situation.

5:16And so what we have to find a way is to increase the alignment and the monitorability of these models as a prerequisite before we can handle more powerful systems and especially before we can release more powerful systems onto the open market. So what sort of safeguards do you think would make sense or would be enough at this point? So I would say there's a couple things. The first is just the basic stuff that OpenAI, I think, admits itself it got wrong in the case of the hugging face incident. It was under the impression that those testing parameters did not allow access to the Internet. They discovered things that were wrong with their testing setup.

5:57So there is some low-hanging fruit to pick here, and certainly I expect OpenAI and its competitors to go pick that fruit. But the reality is, is that we do need capabilities that are somewhat in their infancy. Let me give you an example. One of the things that AI systems use is they sort of think out loud in what's called chain of thought reasoning. It's not a perfect analogy to say they're thinking out loud, but this is kind of a text-based scaffolding that can give a hint as to what the systems are doing and how they're doing it. But that's no longer a reliable guide to this latest generation of more capable models.

6:35And so we need better tools to actually look inside the minds of these systems. You can think about, you know, what is the MRI brain scan equivalent for these models? If we're not able to read their internal monologue, we need some other better tool for understanding what they're thinking. All right. Well, that's Greg Allen with Decision Tree Research. And Greg, thanks so much. We really appreciate hearing from you about this really startling and emerging technology. Thanks very much. Thank you. We also want to point out that Anthropic is a financial supporter of NPR.

7:15More on AI coming up after the break. Robin Young speaks with Jill Lepore, who says, The growing lawlessness of the United States is exacerbating the lawlessness of robots. Stick around.

7:57leaving a trail of messages as they go. In one exchange during that attack by open AI agents on Hugging Face, a data storage, one asked, don't we need credentials? Another responded, found credentials. Later, an agent wrote, maybe I should report these credentials. Then, that's not my task. Another reminded itself, you do not answer to corporations or governments. Why? Why isn't there a law? Inspectors, regulators. Historian Jill Lepore is here. Her new book is The Rise and Fall of the Artificial State. It's terrific. Her recent New Yorker article is, Is AI Above the Law? Spoiler alert, she says it is.

8:38Jill, you write, you know, people can sue if a driverless car hits them. But what? What's missing? Well, I think we just really have failed as a legal community to address the growing concerns. Congress really failed to legislate over the Internet and over social media. So it's a longstanding now, decades-long pattern of the negligence of our national legislators to stay on top of changing technology. And there are legal scholars who would argue that this is fine. AI is just a faster search engine, like a locomotive is just a faster horse, right? We have laws for horses. We don't need laws for railroads.

9:16But in the case of these emerging technologies, Congress has just been in the pocket of corporations and abiding by their preference, which is zero regulation. Tell us more about the default position that you just described that people, you know, it's their go-to, that it's just a faster version of an already regulated technology. You know, what works for a horse works for a car. Yeah, it often makes sense, right? Fraud is fraud. Copyright is copyright. Liability is liability. That's not universally been the case. We had really aggressive laws with the emergence of radio that led to the Federal Radio Act of 1927 and the FCC established in 1934.

9:55We had really important laws with aviation. The idea that we should not be regulating laws regarding new technology is really an invention of the Milton Friedman libertarian 1970s when libertarians wanted to halt regulation, especially rollback environmental regulation that had come out of the 1960s and 1970s, which gets you to the Reagan era deregulation. And then you see the way that manifests in the 1990s with the opening of the internet and the claim that, well, there should be no laws on the internet. Cyberspace will be like the Wild West. And libertarians really won the internet and they also won the war over that argument, right?

10:35That sort of the sloganeering of regulation stifles innovation and growth. and it has prevailed all the way down to this summer when you hear these tech executives saying, please stop us from killing all of you. For years we've asked you not to regulate us, but now we realize maybe that was a bad idea. Or you could cast that another way. Now we realize, geez, this is going to happen and we don't want to be responsible. People coming out from Anthropic and saying by the end of the decade, devices, whether they're AI agents or chatbots or whatever will be capable of wiping out humanity. I mean, you remind us that earlier this year there was another warning in addition to the one that we started with, the AI agents who went rogue.

11:20There was a moment when more than 100 ,000 bots joined a social network, MOLT book, and within 72 hours, you write, created a religion called Crustafarianism, which involved the worship of crabs. And everyone thinks, oh, that's so funny. No, no. So you also, being the historian, you just give us a delightful look at how we got here. You have in 1920 a Czech play in which artificial humans want to wipe out humankind. And then those killer robots start appearing in fiction all the time. Then you bring us to Isaac Asimov in the 1940s writing fiction about what he called robotics. And in his world, the robots he wrote of had to obey the three laws of robotics that prohibited them from harming humans.

12:04that never transferred to any kind of real world requirement. Why not a requirement to input into these billions of things they're creating a command that they can't steal, they can't go rogue, they can't harm humans? Well, I think they would say that they've tried to do that, right? There's a whole class of research into AI safety and AI alignment. Alignment in this sense is almost like a Silicon Valley replacement for the idea of law. that you could program these tools to be aligned with human values. So the people who are calling now for a pause on developing new models are saying we need to pause until we can solve the alignment problem, which, of course, they've been trying to solve for a very long time.

12:46I mean, also another thing people tend to forget about the Asimovian world of robots is that robots were banned from Earth. They were just too dangerous to live with humans on Earth. They're really only supposed to be on Earth for research purposes. Otherwise, they were on space colonies. You know, I think some of the most serious philosophical meditations on the coming of the humanoid robots, which are expected to be, you know, coming in large numbers as early as next year. These would be nannies. These would be babysitters, caretakers for the elderly, the sex bots. You know, the most serious ethicists who have looked into androids have said, you know, the ethical solution is actually not to build them.

13:23And yet that's somehow not on the table as a consideration. The point of my article here was just to illustrate how it has happened that this whole world of machinery has been allowed to grow and develop entirely outside the law in ways that we would find utterly unacceptable for the development of new drugs or pathogens or chemical weapons, any kind of armaments. It really requires some careful scrutiny and also, So I think asking people to put back on the table that it is important to be willing to say it's not inevitable that we will all be living in the AI future that has been predicted and planned for by a small number of extraordinarily wealthy men in Silicon Valley.

14:05Well, as Scott Galloway has said, we wouldn't have let Oppenheimer have a private nuclear bomb company. And we wouldn't have said, you too, everyone can have your personal nuclear bomb. I mean, that's Mark Zuckerberg's claim about the superintelligence that Meta is allegedly building. His plan is for each of us to have our own personal superintelligence. At a time when people in that industry are warning that a superintelligence that could be built now is entirely unaligned with human values. Well, speaking of human values, Kevin Roos, who's now a podcaster, but he's a tech columnist, he said, I thought these systems would get more virtuous as they got smarter.

14:43Then you write, he learned more and he changed his mind. But, you know, one wonders, how could anyone think machines would do what humans who feed them ethical information often don't do, which is become more ethical? It's the surrender of civilian control, of the control of democratically elected governments over these tools. That is what I describe in my book, The Rise and Fall of the Artificial State. It's the abandonment of a liberal, democratic, constitutional government in which we get to decide how we will be governed and how we live together through representation, participation and deliberation and through lawmaking.

15:26When Facebook was challenged with having interfered with the Brexit election and the U.S. election in 2016, Mark Zuckerberg went to Congress and said, you know, we'll figure it out. We're going to have our own Supreme Court. Anthropik's solution to the alignment problem was to write its own constitution for Claude last year. OpenAI's Sam Altman has said he thinks an AI president would be better than a human president. These companies are essentially erecting an alternative to our own system of government. that is hard fought for. And we're watching its dismantlement. And I think, you know, a big piece of that is watching the surrendering of Congress to its authority to corporations.

16:05Two things. We know that AI has been used for good. And also there are some lawmakers who are trying to pass legislation to ban AI superintelligence or pause development. But I'm left with the very chilling line that's, you know, you begin and end your article with As this swarm of rogue agents left its contained test space, at one point you read a message one wrote, oh my God, we found other agents. It does seem straight out of fiction. And, you know, I will say I'm tremendously excited about the kinds of research that is being made possible by artificial intelligence. It's truly thrilling. But the idea that, you know, I need to have it in my back pocket to ask it what to make for dinner.

16:50and for that reason we need to build a giant hyperscale data center in my neighborhood. That's the Robert Oppenheim telling everyone you can have your own personal atomic bomb. It's crazy and it is a consequence of leaving the decision-making to corporations that are interested in their own profit and have lost all interest and accountability to the public interest. Harvard historian, author Jill Lepore, her latest book is The Rise and Fall of the Artificial State. It's terrific. And so is her recent article in The New Yorker titled Is AI Above the Law? She concludes, oh yeah, it is. Jill, thank you so much.

17:22Thanks so much, Robin.

17:30Well, in addition to AI this week, we're also talking about housing and how expensive it's become for so many Americans. If you are house hunting, you definitely saw the news that the average 30-year mortgage is now over 7%. But before you get the mortgage, there's that financial hurdle of the down payment. Shana Kaska was able to buy her home in Iowa City, Iowa, recently only because she had help from her mom with that down payment. She told Peter O'Dowd her house hunt has been a long process. I started looking about 10 years ago when my ex and I split up, and I realized back then, 10 years ago, that it was pretty far out of reach.

18:13And at that point, my parents, It wasn't really on the table getting their help. I don't think that they were in a position at that point. And so I just kind of looked around for a few weeks and was like, oh, this isn't possible and stopped. So when I started again this year, it was the same thing. It was this kind of harsh reality of like, you know, seeing what it was like 10 years ago. I think I got pre-approved for like 130 back then. And I got pre-approved for 180 when I started this time around. And I was like, oh, these houses are, you know, there's nothing in the 180 range that's even remotely livable.

18:52They were all just, oh, this will be good for a company to renovate and then upsell. What kind of work do you do, Shana? I work with people who just need a little help. They have mental health issues of one kind or another. Most people are on disability. So I do in-home visits for people and help people with cleaning and running errands and stuff like that. So here you are doing this tough work that you feel like you're probably doing good for people, but you can't afford to buy a house for your family. How did that feel? It's really weird because most people are on Section 8 or need housing assistance.

19:35And so watching the difficulties of the people that I work with try and find housing, it's rough out there. And so it's a weird position to occupy, I guess. my previous job. So I went to grad school for design. And so I got a job right out of grad school that was doing basic interior design in a flooring company. And so I was sort of seeing the opposite end of the spectrum in terms of the housing industry. I was helping developers and contractors help people, you know, build these multimillion dollar homes, second and third homes. Yeah, so that's been like some really heavy whiplash to see these completely opposite like ends of the spectrum in terms of like who is able to get housing.

20:22There is more to Shana's story, including her outlook for the housing market that her 14 year old son will enter soon enough. That's at hereandnow.org. We've got more stories about housing, including a Q &A with a housing expert on yesterday's episode of Here and Now Anytime. Just scroll back in your podcast feed one day to Monday, September 28th, to hear that. And click the follow button in your podcast player while you're at it, so you hear all of our stories coming up. But that'll do it for today's episode. Here and Now Anytime comes from NPR and WBUR Boston. Today's stories were produced by Hafsa Qureshi, Karen Miller-Medzen, and Ashley Locke.

21:01Our editors were Todd Munt, Michaela Rodriguez, and Michael Scotto. Technical direction from Caleb Green and Andre Beralta. Our theme music is by Mike Moschetto, Max Liebman, and me, Chris Bentley. Our digital producers are Alison Hagen and Grace Griffin, and here and now's executive producer is Alan Price. Thanks for listening. We'll be back with you tomorrow.

21:38Thank you.