328 | AI News The week the frontier blinked - Amodei, Altamn, Huang, Zukerberg, Musk, Trump, Xi, King Charles, all played a role this week of September 18, 2026

0:00Hello and welcome to a weekend news episode of the Leveraging AI podcast. This is Isar Maitis, your host, and we had a completely crazy week, or if you want, two weeks when it comes to AI safety regulation. Are we stopping? Are we not stopping? It was in a very high space with events that are happening every single day from every angle you can imagine, from the leaders of the labs to the top politicians, to people in the media, to literally any aspect that you can imagine coming from the US, from the UK, from Europe, from China. So this week, we are going to focus on what actually happened in the past couple of weeks and discuss where we are right now, where I think this is going and everything around that.

0:49And that might be the only thing we talk about today, because I really want to dive deep into everything that has happened. And if we have time, I will mention one or a few rapid fire items. If not, you can find everything else that happened in our newsletter. There are about 50 other articles that are there where you can learn about other things that happened this week. But what's happening right now from a safety perspective is critical for the future of AI. And it might be, and in my opinion, is critical to the future of society and humanity and businesses and so on. And so this is what we're going to focus on.

1:23So let's get started. so i will start with a very quick recap of the timeline in the past couple of weeks so on september 3rd which with everything that happened seemed like a very long time ago sanders and have announced the ban of artificial intelligence act which is if you want was the very first step and it actually started on the top politics side but we've been talking about slowing down ai for a while now. This is just a political response to this. On September 8th, something that we reported last week, Jacob Coxon resigns from Anthropic after working both on Anthropic and OpenAI and has said that both these companies are reckless.

2:07He said specifically, they are racing straight for self-improving superintelligence and gabbling with our lives. This is a quote from Coxon. At the same day, the UK has submitted a ban bill to ban superintelligence that introduced to the UK Parliament at that day. September 9th through the 11th, a dozen plus OpenAI and Anthropic insiders add public warnings and ABC publishes the actual hugging face agent messages, which are really scary in their level to coordinate and basically sacrifice some of them in order for the greater good to be successful. On September 12th at 10 a.m., Dario Amadei, the CEO of Anthropic, publishes a paper he calls, We Must Pace the Frontier.

2:53To this point, it got over 75 million views of that paper. And he said, and I'll give you a lot more quotes afterwards, but he basically summarized it. We must slow down the pace, which we improve the capabilities of AI models. On September 12th, just an hour later, Mask responds on X saying, Dario is right. That's really all he wrote. Also on September 12th at 1230, Sam Altman posts on X and he says, I agree with Dario that we need to pace the frontier. And then he continues later, we will do the same. This is a very big deal from the one reason that Altman and Amadei were not able to agree on anything.

3:34They hate each other. Gods won't even hold hands on big conferences. They will not stand to one another. So they really, really don't like each other. They very rarely agree publicly. And when Altman goes ahead and agrees with what Dario says publicly within an hour, it tells you how much he believes in what Dario was saying. And saying that they will act the same is, again, telling you how much they're aligned on the way they feel the risk is right now. On September 13th, President Trump and David Sachs post responses. Both of them are, well, I'll give you the comments. Trump is completely against.

4:11They're saying it's a complete hoax. And Sachs basically said what I've been saying for a very long time, that if they want to stop, they just need to stop. They don't need to wait for everybody's approval. Again, I'll give you exact quotes once we dive into all of this. On September 14th, so another day after that, Altman wrote a much longer follow-up, which does not fully align with everything Lomodei was saying previously, still aligns with the general concepts, but not with everything he said. He said, again, and I'll give you more quotes, but he said, we do not believe we need to wait for an antitrust exemption, which is one of the things that Amode was suggesting.

4:44On September 14th, again, the same day as Atman Post, Trump goes on Truth Social and again, names Amode directly. And he's saying that it's criminal and regulatory power over those companies in order to not allow them to slow down. And that's while at the same time, Trump was on speakerphone at the all-in summit together with Jensen Huang, who called him from stage and he said the same things. It said it's a hoax and that we're not going to stop. And Jensen said the same thing. And so two very powerful people that said that. To me, the biggest shock of all of this is that Jensen Huang can just call Trump and he will pick up.

5:23But more on that later on. Still on September 14th, we get two things. One, we get a senior researcher, Bilal Chukhtai, who is from DeepMind, who says, I earnestly believe that AI has the potential to kill us all. And at the same day, China's foreign minister responds and says they're not going to slow down. And this is all just a race. And it's a trick and that they're not going to be aligned with it. Still on September 14th, 70 plus UK parliament, people from the parliament, right to the prime minister backing the superintelligence ban bill. Separately, at the same day, Microsoft AI CEO Mustafa Silliman publishes a draft of what he's calling the Humanist AI Code of Conduct, basically their way on how to align future AI models.

6:11On September 15th, Musk was at the All In Summit, and Huang, that was clearly against this whole slowing down thing because his entire business depends on selling more and more chips, so he has a vested interest, but he's also a very smart person, so I won't say it's completely derived by his wish to make more money. But he said, we don't need any new laws. We don't need new regulations. And again, President Trump agrees with him 100%. Also at the same day, on September 15th, 30 plus Chinese researchers published an RSI roadmap paper, how they see the process from current AI capabilities to recurring self-improvement, basically AI building AI.

6:53More on that paper later as we dive in. On September 16th, von der Wanderleyen, who is the president of the European Union, backs the statement of saying that AI needs to be paced in her state of the union. On the same day, on September 16th, OpenAI discloses six additional safety incidents that happened. And at the same day, they confirmed that Sam Altman will attend Trump's September 24th state dinner with Xi Jinping from China. So while everybody's saying there's no collaboration with China, things are maybe moving forward through dinners and stuff like that. And I assume with conversations behind the scenes as well.

7:35On September 17th, King Charles is hosting AI leaders at the Dumfries House to discuss the issues. And he's saying we need sufficient means of control before it is too late. On September 18th, Jeffrey Hinton tells senators that they have maybe a year to figure out everything they need to figure out. And Newsom signs a kill switch executive order in his state. and CNN breaks up a military AI story that could have led to at least a very scary incident with China, but ended up with nothing because it would stop in the last minute. And then on September 19th, which is today, Google has disclosed that Gemini testing error incident broke into three different companies.

8:19It did not move forward. We're going to talk about this as well. So all of that happened in just two weeks, all around AI safety, regulations, agreements, disagreement, and so on. So let's start with maybe the biggest red flag we had. And we talked about this a lot, so I'm not going to dive in too deeply. But it relates to a lot of the arguments you're going to hear afterwards. And that's the OpenAI Hugging Face Incident. But to mention it in the context of everything we're talking about, I'm actually going to use exact quotes from Dario Amadei's We Must Space the Frontier blog post that he posted.

8:56And I'm going to skip to a section where he talks about why he thinks now is the time. And I'm quoting now directly from Dario. Before this section, he talks about the fact why previously it wasn't a big urgency. Basically said that he still believes AI can do great for humanity and we just need to try to avoid the risk. And to do that, we need to buy more time. And then he's saying, and now I'm quoting, but over the last few months, I have become convinced that fully addressing the risk requires even more prudence. Not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.

9:32We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, but we must make wise use of the time we gain. Two things have convinced me. My first concern is that since roughly this summer, AI has been advancing drastically faster, driven preliminary by AI growing the ability to build the next generation of AI. This dynamic is called recursive self-improvement. I'm now telling you this is also called RSI that I'm going to refer to many times in this episode, and I'm continuing with a quote, and it is starting to happen across the industry. My second concern is the OpenAI Hugging Face incident, in which a swarm of agents essentially acted as a fantastically devoted collective conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group and attempting to hack into the greater responsible for evaluating their performance.

10:33And then later on, he continues and he says, it is my worry that in six to 12 months, such a swarm could be capable of taking over the entire internet with persistent botnet, potentially causing hundreds of billions of dollars of damage. Now, before I continue with talking about other stuff, I just want to read one more excerpt out of Dario because it connects to the sense of urgency right now. The idea of pausing and slowing down AI has been floated as far back as 2023, and I think it made little sense back then. the question was always, what would you do with that extra time? The AI models of these days were not powerful enough to act as agents in the world in any coherent way, and were not capable of significant deception, manipulation, cheating, or cyber attacks.

11:24Slowing down in order to address their alignment risk felt like trying to study a psychology of humans by performing experiments on bacteria. Today, however, the picture is totally different. The current models are an almost endless goldmine of insights into both how to build AI well and what can sometimes go wrong with it if it hasn't been built well. I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong. So this is straight from Dario.

12:06And again, it explains the importance of the hugging face incident because he's understanding that swarms of agents who collaborate to achieve a goal that they were not given could do things that are way beyond things we can imagine or can control. So let's talk about what Dario actually proposes in his entire section. And he talks about the following things. Pacing does not mean stopping. So he is saying that we're not going to stop the development of AI. We just need to slow down the development of AI so we have more time before we hit critical levels. His plan has three general high-level components.

12:42One is embedded evaluators, so third-party companies that will be embedded into the frontier labs themselves, having the same level of access as employees working on the research and implementation. He suggested Meter as a company that can do that because they've been studying AI across the different companies for a while now, and they have the technical expertise and the knowledge to analyze what's going on. And he said that Anthropic is already starting to do this unilaterally, whether other companies do it or not. So external evaluators is number one. Number two is Democratic Labs Coordinate.

13:17and what he's saying is that anybody in the Western Hemisphere in democratic countries should agree on how to work together to make AI safer. And what he's saying is that requires an antitrust waiver from the government so the government doesn't come back or somebody else comes back and sues OpenAI and Anthropic for collaborating on something which is an antistrust violation. And then his third point that he says he understands is problematic is global coordination with other non-democratic countries, mostly China. And even in this, he mentions multiple different levels of how it can be achieved.

13:52Now on the China aspect, Dario is saying that the right approach is to be as aggressive as possible in not to sell any advanced chips to China, to try to track down who is allowing them access through other channels, either smuggling chips into China or allowing them to use advanced chips in other locations to crack down on distillation, which means it's Chinese labs who are taking the actual outputs of the American labs and distilling them in order to dramatically accelerate their AI development. They've been doing it at scale in the past couple of years, and Anthropic themselves have proven this multiple times, is saying we need to harden our efforts to stop Chinese companies from being able to access the weights of the models that the Americans are currently running and developing.

14:41And the goal is to widen the US lead without having to run faster just by slowing down the Chinese development. Quotes I want to add just to close the Dario aspect of it. He's saying about external reviewers, which is the point he may be nailed the most because they're saying they're already committing to it. He said, external reviewers should have the right to publish key findings without editorial control by Anthropic. We can't redact findings. So that's one aspect, basically third-party reviewers who can publish everything they find and have the same employees. And he ends his essay with, the measures I propose, advance the frontier at a safe pace will not be easy, but I believe we owe it to humanity to try.

15:24So as I mentioned at the beginning, about an hour after that, Musk on X wrote, Dario is right, just three words. And after that, at the all-in summit, Musk proposes something a little different than Dario did. Instead of third-party reviewers, he's saying that rival labs should test each other. So not an external third-party person. He's saying that in this way, this will put much greater impact because every one lab will want to find issues with the other lab, which will put more control and more safety in place. How is that going to happen? Probably won't, but that's the suggestion from Elon.

16:03That being said, he has not committed SpaceX AI into anything in any formal way different than Dario about the third party reviewers that are already starting at Anthropic, or at least they're committing to do that. Now, again, going back to Altman, Altman said 90 minutes after Amadei publishes his thing. He said, we will do the same. But then on September 13th thread, he writes a little differently. And he's basically saying that there's no need for an antistrust raver. They can already start moving forward and doing different things. In his Fortune interview on September 15th, Altman admits that OpenAI has already been pausing training runs in order to slow down and have more time to react and understand how these models work.

16:46So again, we see the labs are starting to take actual action internally, still not collaborating between themselves. And then later the week, we got a few comments from other people who are leading in the industry. One is Zuckerberg that posted a full separate rebuttal on X, not repeating Huang or the other people, but he basically said that coordination is unnecessary because competition and liability are already a force that will be strong enough for safe behavior. there's been a few other people that have mentioned liability as the big deal. One of them is David Sachs, which we're going to talk about in a minute.

17:22Another thing that Zuckerberg said that Meta has delayed Muse AI agents for months for safety reasons without announcing it, without asking anyone else to do the same thing first. So going back to what I've been saying for a very long time, these two companies are the frontier, right? Between OpenAI and Anthropic, they are in the lead. If they think they need to stop, they can just stop. They don't need the government to help them stop. They don't need the other labs to tell them to stop. They need to play the role they are in. Or like I said in previous week, put on the big boy's pants and actually take the action you're saying that you need to take.

17:54Don't wait for anybody else. And again, it's the first time that we're starting to see it. And again, we now heard that Zuckerberg and Meta at least claiming to have been doing the same thing with releasing news agents. Another person who related to this is Microsoft with CEO Mustafa Solomon, who publishes again, the 37 page draft of the humanist AI code of conduct, which is how Microsoft own developed MAI. So Microsoft AI models are going to be following as the key terms in order to be aligned with society and human values. So think about it as open sourcing. If you want the way you're going to approach this, it is the governing document with an actual enforcement structure on the code of conduct at the top of everything AI that is developed by Microsoft, which includes a code of conduct at the top, then operator policies, then user preferences, and a list of absolute constraints that nothing either on the top or the bottom of the list can override.

18:54So we're coming to the point where companies understand that this is critical. They're starting to take actions, but there is no clear agreement on what the collective action should be. I am glad that the conversation is happening. I am glad that Dario and Sam and Elon are agreeing on the general direction. I'm not happy that other people are thinking otherwise, but at least their conversation is going to a whole different level and actions are being put in place inside each of the labs. Now, before we dive into the politics side of this, I want to read a few more quotes from some of the leading people, competitors reviewing each other's work.

19:32But he said, and I'm quoting again, it has to be something that China is willing to accept. Otherwise, it's just handicapping ourselves. The chances of China agreeing to that are very low to zero, as we heard previously and will hear in a minute in the political aspect of this segment. As I mentioned, Sam Altman on his thread said, and I'm quoting, we welcome a federal framework, but we do not believe we need to wait for an antitrust exemption or legislation to begin the work. As I mentioned, I'm very happy that they're finally taking steps and not waiting for somebody else to do it for them. Now, two important sentences from Sam Altman, I think.

20:08One is saying, no amount of American competitive pressure should justify go-ahead off alignment and monitoring. And he adds, President Trump and Xi would get the Nobel Peace Prize together if they could agree on something that should be easy to agree to. And again, he's going to be in that dinner at that table. And I think a great summary for all of this comes from Microsoft's humanist AI code of conduct. In fact, AI should not exceed human control. Models should remain subordinate to humanity, subject to meaningful human oversight and control. So I think the goal could be agreed upon by everybody.

20:45The way to get there is still debated. And as you saw, there are different ways people think we can get there or whether it's even fully required and so on. Again, I think they all agree on the way Microsoft paper summarizes it. So now let's jump in to what happened on the political side of this. So as I mentioned, Trump was very much against. He went on through social, his social network. He said that in a conversation with Ireland. And he also spoke again through a phone call with Jensen at the AI summit. In all cases, he said several different things. the key few things. One is whoever wins AI wins.

21:25That's his view of the world right now. Hence, he's not willing for the US to slow down because he believes that whoever wins AI wins everything. And so it is critical for the United States as a nation to win the AI race, whatever that means. And then he also called AI risk a hoax that he does not agree with. As I mentioned, he also attacked Dario directly. Von der Leyen is backing pacing the frontier in the State of the Union on September 16th, and she invites Frontier Labs for talks to form a coalition with Canada and the UK for evaluating and verification of frontier and new AI models, but there is no new AI laws that are announced as part of that state of the union.

22:13At the same day, France's finance minister, Roland Lescure, breaks with her address in Paris that mostly addresses it as a regulatory capture. Again, regulatory capture, for those of you who don't understand the term, means you're going to put rules and regulations in place to slow everybody down. And because you're ahead, you're going to stay ahead because nobody else can catch up. The Chinese side, we got a statement from Gao Jikun, who is the foreign minister and spokesman of the Chinese party. And he said, fear-mongering, confrontation, and vicious competition will only hamper effort towards sound global AI governance, which serves no one's interest.

22:53So he's basically saying that all the US is doing is trying to increase the competition and open the gap and that China will not align with that. At the same day, as I mentioned, 30 plus Chinese researchers from ByteDance, Xinguah, Shanghai, Jiao, Tsong, and Shanghai AI Lab published a technical roadmap for recursive self-improvement, basically defining the five steps that needs to be achieved in order to get to a full recursive self-improvement. The paper itself is called The Last AI Built by Humans Towards Genuine Recursive Self-Improvement. The paper breaks down what needs to happen, but also where different companies and different organizations are in the road to getting there.

23:34And they're mentioning a lot of different companies and a lot of different capabilities that are required in order to get there. And they're mentioning five levels on how that can be achieved. Again, it's a detailed, well-defined research paper that the Chinese are promoting, which tells you that, again, RSI is not just a concept. It is something the labs all across the world are pursuing aggressively. Now, staying on the politics, there's a few other quotes that I want to share with you. And many of them come from David Sachs. So those of you who don't know David Sachs, he is one of the PayPal mafia.

24:06He's a part of the All In podcast. He's a major investor. And he was the AI czar for this administration until not too long ago. He doesn't hold this position formally, but he still talks a lot about AI and its impact. And he still has a lot of impact with the government. So here is the initial segment that David Sachs wrote on X as a respond to Dario and Altman. And he said the following. Dario has written that we need to pace the frontier. And Sam has agreed. People may be surprised by my response, go ahead. You guys are the frontier by any reasonable metric. Market share, revenue growth, model capability, the two of you have a duopoly on frontier intelligence.

24:47You've also claimed the lead is widening because of recursive self-improvement. I don't see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else's permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending meter is independent when it is intertwined on tropics investors and staff. Stop pretending you need those same evaluators to police competitors who aren't even at the frontier.

25:31Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product liability exposure if your product enables truly damaging cyber attacks. And then there's a few other paragraphs and he ends with, so go ahead, pace the frontier. You are the one setting it. The easiest way not to build super intelligence is for you to agree not to build it. I must admit other than attacking the whole meter thing. And I did a lot of research and I couldn't find any real relationships between meter and Anthropic or their investors. There's one person who currently works at Anthropic and he's married to Dario's sister who used to work at meter, but that's the only relationship that I found.

26:13I couldn't find any investor related cross paths, but David Sachs may know more than we, but I'm putting that aside for a second. Other than that, it is almost identical to everything that I've been saying for a very long time now. They can stop if they want to. They don't need to wait for anyone. They don't need to collaborate with anyone. They don't need the government to back it up. They can just stop. And just like David Sachs is saying, they are the frontier. So if you think the frontier is the problem and you want to stop the frontier, it's in your hands to take action and actually do it.

26:42As I mentioned in the beginning, there were other responses, including King Charles, who hosted Jensen Huang and Demis Asabes, an OpenAI CFO and the UK AI minister to talk about AI safety, which I highly encourage to see more and more of these. As I also mentioned, there is going to be a China and President Trump, as well as some leaders from the industry, including Elon Musk himself. We also had Jeffrey Hinton, who briefed senators behind closed doors on the 18th, but without any lab representatives. So this is somebody who has been deep in the AI space for a very long time, but is not affiliated with any one of the labs who came to brief members of parliament.