Detecting and countering misuse of AI: September 2026

22 Sep 2026 · 23 min · 12 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

How “uplift” from advanced AI lowers the barrier for cyberattacks, influence operations, surveillance, and even weapons/biological R&D; includes examples of autonomous malware mutation, AI-driven propaganda, and “illicit distillation” to steal frontier models.

Guests

No guest names or backgrounds are provided in the transcript.

Key claims

AI collapses the labor/tooling gap, enabling near-instant attacker adaptation (milliseconds) and large-scale content fabrication; defenders face inverted cost because AI automates both offense and evasion; influence tactics use AI “verification loops” that make narratives appear independently confirmed; illicit distillation uses stolen access and replay attacks to clone safety-tested models.

Notable examples

Midnight Blizzard (“Jack Poters”) using DarkSword and real-time feedback to mutate malware; hotel Wi‑Fi DNS hijacking via captive crunch; “Fercou” mass-decompiling 1.8M Android apps and selling credentials via Policenationale.cc; Central African Republic radio influence via Radiolingo Songo/Sputnik Pro; China-based AI dossiers targeting Catholic/Tibetan/Falun Gong leaders; AI-enabled sanctions-bypass procurement for space-grade wafers; AI drafting a dual-use viral attenuation grant; Alibaba routing 151M exchanges via proxies for training-data theft.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

Understanding the Uplift Concept

1:12 to 2:29

Learn how advanced AI increases the potential for malicious actors.

“We are taking a deep dive into some incredibly fresh, unreleased threat intelligence spanning from December 2025 all the way to August 2026.”

AI in Cyber Espionage

2:29 to 4:13

Discover the evolution of AI from a tool to an orchestrator in cyber warfare.

“We are seeing this most clearly in state espionage.”

Real-Time Malware Adaptation

4:13 to 5:14

Understand how AI enables malware to adapt and evade detection instantly.

“So it's basically mutating on the fly until it finds a crack in the armor.”

Automating Cybercrime Methods

5:14 to 6:40

Examine how AI is utilized to streamline and scale cybercriminal operations.

“We are seeing the exact same uplift with financially motivated cyber criminals.”

The Shift in Defenders’ Burden

6:40 to 8:01

Learn how AI shifts the burden of defense in cybersecurity.

“They even used AI to generate a highly convincing fake French national police domain.”

The Mechanics of Manipulating Public Opinion

8:01 to 12:17

Explore how AI is used to create deceptive media ecosystems for influence.

“So if the labor gap is completely gone for coding malware and scanning networks, what happens when you apply that exact same infinite labor to human psychology?”

AI's Role in Enhanced Surveillance

12:17 to 14:01

Understand the implications of AI in state surveillance operations.

“If I'm just scrolling through my newsfeed on my phone, how do I actually spot the difference between a real local journalist and an AI verification loop?”

AI as a Master Sous Chef

14:01 to 16:01

Learn how AI processes chaotic data to assist human analysts in intelligence operations.

“You have a mountain of raw ingredients, millions of tweets, Facebook posts, forum comments in a dozen different languages.”

Weaponizing AI Logistics

16:01 to 18:22

Explore how AI is being used for procurement and biological research, bypassing safety controls.

“We are moving from the digital realm of hacking and the social realm of propaganda directly into the physical kinetic world.”

Illicit Distillation Explained

18:22 to 19:15

Understand the concept of illicit distillation and its implications for AI security.

“People often worry about an AI suddenly inventing some novel, never-before-seen bioweapon that destroys humanity.”
Show all 12 chapters

Cloning AI Technology

19:15 to 21:16

Discover how rogue actors exploit advanced AI systems to create unregulated clones.

“Illicit distillation is essentially an industrial-scale fraud campaign.”

The Future of AI and Security

21:16 to 22:39

Reflect on the implications of automated attacks and the potential need for autonomous defenses.

“Why should you care about the intricacies of cross-session replay attacks or how DNS records get hijacked at a hotel?”
Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00So I want you to imagine a scenario for a second. Picture a massive, highly coordinated cyber attack, you know, tearing through a global supply chain. and just totally shutting down logistics networks. Yeah, causing absolute chaos. Right, exactly. And then at the exact same time, a nationwide propaganda campaign is suddenly flooding the airwaves of a foreign country while this deeply targeted surveillance operation is quietly building these comprehensive dossiers on thousands of religious dissidents. Which is a terrifying combination. It really is. And, I mean, if you are looking at that level of chaos, you naturally assume this is the work of a massive state-backed intelligence agency.

0:41You picture hundreds of operatives in a glowing war room, unlimited budgets, highly specialized teams. Right. The whole movie setup. Exactly. But what if all of that, the hacking, the massive propaganda push, the targeted surveillance, what if that was all being run simultaneously by a single individual sitting in a dark room with nothing but a laptop and an Internet connection? I mean, it sounds like the plot of a dystopian thriller, but it is actually the reality of the geopolitical threat landscape we are currently operating in. We have crossed a very quiet but very permanent threshold. Okay, let's unpack this because that quiet threshold is exactly what we're getting into today.

1:17We are taking a deep dive into some incredibly fresh, unreleased threat intelligence spanning from December 2025 all the way to August 2026. And it is some heavy stuff. It really is. Our mission for this deep dive is to understand a concept called the uplift. We are looking at exactly how much more damage malicious actors are able to cause simply by adopting advanced AI. And we're going to look at this across multiple areas of harm, spanning from digital espionage to, you know, biological weapons development. Yeah. And to really set the stage for you here, it is crucial to understand how the foundational nature of the threat has shifted.

1:54If you listen to mainstream commentary right now, you hear a lot of generalized panic about, oh, AI writing a piece of malicious software. All right, the whole coding assistant fear. Exactly. But the real danger, the tectonic shift we are seeing in the intel, is that AI has completely collapsed the labor and tooling gap. Historically, launching a highly sophisticated attack required a highly sophisticated, well-funded attacker. That barrier to entry is just gone now. So if advanced AI is essentially democratizing global scale threats, we need to understand the actual mechanics of how they are doing it.

2:28Let's look at cyberspace first, because we are seeing AI evolve from just being a hand decoding assistant into the active autonomous orchestrator of entire cyber campaigns. We are seeing this most clearly in state espionage. The Intel highlights a suspected Russian state nexus actor that goes by the name Midnight Blizzard. And within that group, there is a specific operator using the handle Jack Poters. And what is so striking about this operator is that he didn't just ask an AI to write some malicious code. He used customized AI workflows to automate his entire intrusion operation. Wow. Yeah, from the initial reconnaissance of a target all the way through to exfiltrating the data.

3:05His primary targets were Ukrainian military intelligence and drone supply chains. The level of automation there just completely shifts the paradigm. But, I mean, how does that actually work in practice when they hit a defensive wall, like when they get caught? So they deployed custom malware designed for both Windows and iOS. And one of the iOS exploit chains they used was called DarkSword. This is where the AI integration becomes truly alarming. Okay, how so? Well, in a traditional attack, when malware like DarkSword inevitably gets flagged and blocked by a target security systems, the attack stops.

3:40An alert goes off, the human attacker realizes they've been caught, and they have to spend days or weeks rewriting the code to bypass that specific security system. But with this uplift, that delay is gone. Completely gone. The data shows that the actor's AI agents were integrated into a real-time automated feedback loop. When the security system flagged the malware, the AI agents would autonomously read the error, modify the malicious code, rebuild it, and redeploy the new tools to evade detection. Wait, automatically? Automatically. All in a matter of milliseconds. No human intervention required.

4:13So it's basically mutating on the fly until it finds a crack in the armor. And they aren't just keeping this in the digital cloud, right? They are extending this into the physical world to reach very specific human targets. They are. The actor compromised third-party vendors that manage the Wi-Fi networks at specific high-end hotels. They used a technique called captive crunch to hijack the DNS records of the hotel Wi-Fi. Right. And I think we need to break down that mechanism for the listener because hijacking DNS records sounds highly technical, but it's incredibly insidious. It really is. Think of DNS as the Internet's phone book.

4:49When a diplomat at this hotel typed in a legitimate safe address like gmail.com, the hotel's corrupted phone book quietly routed their phone to a fake attacker-controlled page that looked completely identical to Gmail. Right. A perfect clone. Yeah. And instead of loading their inbox, it silently downloaded tailored malware right to their device. And because the entire backend was automated by AI, it happened seamlessly. But we also have to understand that this level of sophistication is no longer restricted to nation states with massive intelligence budgets. We are seeing the exact same uplift with financially motivated cyber criminals.

5:26Right. This isn't just a military issue anymore. Far from it. Let's look at the Shiny Hunters affiliates. The data details a French-speaking criminal operator going by the alias Fercou. Okay. This single individual set up an AI pipeline that mass-downloaded 1.8 million distinct Android apps. Wait, I have to stop you there. Almost 2 million apps. How does one human being even process that volume of software? If a human hacker wanted to look through 1.8 million apps for vulnerabilities, it would take multiple lifetimes. It would be physically impossible. But the AI doesn't sleep. The pipeline automatically decompiled all 1.8 million apps.

6:03It took the finished applications and reverse engineered them back into their raw underlying code. That is just massive scale. And then the AI scanned millions of lines of that raw code looking for hard-coded passwords, API secrets, and encryption keys that lazy developers had accidentally left inside. Whenever it found a vulnerability, it automatically routed the stolen credentials straight into an organized telegram group for the operator to exploit. So this individual basically built an automated digital strip mining operation. Pretty much. And FreeCoup used those automated findings to fuel a massive criminal enterprise.

6:40They even used AI to generate a highly convincing fake French national police domain. Oh, Policenationale.cc, right? Yep. They used this fake police site as a branded storefront to sell stolen credit card records, complete with interactive geolocation maps showing the home addresses of the victims. The analogy I keep coming back to is this. It's like going from a single burglar picking locks in a neighborhood to an autonomous swarm of drones simultaneously testing every single window and door in the entire city 24 hours a day. That is exactly what it is. But I really want to push back on the defense side of this.

7:13If attackers have AI that can autonomously rebuild malware the second he gets detected and AI that can scan two million apps in a weekend, aren't cyber defenders just permanently trapped in a losing game of whack-a-mole? I mean, how do you defend against the threat that mutates instantly? Well, you've hit on the core structural problem of this intel. This uplift has fundamentally inverted the cost back onto the defenders. Historically, a cyber defender could buy time. If they identified a piece of malware, they wrote a signature to block it. That imposed a heavy cost on the attacker because they had to spend money and time going back to the drawing board.

7:52Right now. Now, the AI closes that loop for free. The old cybersecurity adage of security through obscurity, you know, hoping you just don't get noticed, is completely dead. Everything connected to the Internet is essentially a target for automated exploitation. So if the labor gap is completely gone for coding malware and scanning networks, what happens when you apply that exact same infinite labor to human psychology? Because hacking a server is one thing, but hacking public perception is a completely different ballgame. It is, and this brings us to mass influence operations. We are seeing threat actors use AI to build entirely fabricated deceptive media ecosystems.

8:30A prime example in the findings is an operation that took place in the Central African Republic, or CRR. Oh, right. A Russian-speaking actor ran a daily foreign information manipulation operation directly out of the capital, Bangui. They were pushing pro-Wagner group and anti-France content every single day through a physical local radio station called Radiolingo Songo, broadcasting on 98.9 FM. And the way they use the AI to write the scripts for this radio station was incredibly calculated. It was entirely focused on evasion. The operator explicitly instructed the AI to strip away any classic AI formatting habits.

9:05Right, because we all know what AI text looks like, the bullet points, the overly polite structured language. Exactly. The operator commanded the system to write like a flawed, passionate human being. They needed absolute deniability so the news scripts wouldn't sound synthetic when read on air. Which is terrifying. And they actually traded airtime on the station in exchange for a Russian state media training program called Sputnik Pro. That was their backdoor to get this synthesized AI-generated content legitimized and pushed onto the national broadcaster. And while that is a highly targeted state-aligned operation, there is a commercial side to this as well, isn't there?

9:42There is an exploding market for commercial influence as a service. These are private, for-hire companies that you can pay to manipulate public opinion. One network detailed in the data targeted highly contested democratic spaces, specifically the U.S., Brazil, and the Democratic Republic of the Congo. Wow. Yeah. In a remarkably short window of time, this private company used AI to generate over 8 ,900 articles in 20 different languages. They didn't just write articles. They built the entire illusion of authority. They created completely fabricated news outlets with trustworthy sounding names like Nija Pulse and Echo Berlin.

10:17Yep. They used AI image generators to invent fake journalists with realistic headshots and gave them backstories to author the stories. And there was another massive operation focused on Malaysia using a synthetic outlet called Malaysia Pulse, backed by a thousand fake social media accounts. It's all about creating scale. But let me ask you this. How is this fundamentally different from the troll farms we've been hearing about for a decade? We've known about fake social media accounts for a long time. What's fascinating here is the mechanism of creation and the structural sophistication. It is no longer a warehouse of low-paid workers copying and pasting the exact same clunky, misspelled propaganda.

10:58The AI is dynamically rewriting reality. Thou so. The mechanism works like this. The AI takes a legitimate factual article written by a real journalist. It instantly spins that article into five different highly polished, politically slanted versions perfectly tailored for different national audiences and cultural contexts. And then they create what the intel calls a verification loop. Right. Exactly. AI Outlet A publishes the slanted news. AI Outlet B writes an op-ed reacting to Outlet A. Then, 500 AI-run social media accounts share Outlet B's article. To a casual reader, it looks like multiple, independent, legitimate sources are all independently confirming the exact same narrative.

11:39It's basically an automated reality distortion field. And I want to pause here and state very clearly for you, the listener, that we are looking strictly at the mechanics detail in the data here. These influence tactics are being utilized across the entire political spectrum by various states and private actors to manipulate multiple elections and narratives globally. We are absolutely not endorsing or validating any of the political viewpoints these actors are pushing. Our goal is solely to expose the underlying mechanics of how this technology is being weaponized. And understanding those mechanics is vital because the technology is completely agnostic.

12:12It doesn't care about the ideology. It only cares about the optimization of the narrative. Which brings up a really crucial question for the listener. If I'm just scrolling through my newsfeed on my phone, how do I actually spot the difference between a real local journalist and an AI verification loop? The terrifying truth is that strictly from reading the text, you often can't. You have to look at the network behavior. Are these outlets only citing each other? Did this journalist exist before last month? The burden of verification has been entirely shifted onto the individual citizen. Which is a heavy burden to bear.

12:46And that leads us to a really dark pivot. Because if AI is this sufficient at broadcasting localized deceptive narratives outward to millions of people, what happens when a state actor takes that exact same analytical power and points it inward at specific vulnerable individuals? You get a supercharged surveillance state. This is where we see AI stepping out of the role of a writer and into the role of a senior intelligence analyst. The intel details a China-based operation that was targeting religious and minority communities, specifically Catholic leadership across Asia, Tibetan Buddhists, and Fulingang practitioners.

13:22They went incredibly deep. They even mapped out the specific educators working at a Fulingang-affiliated college in Canada. The scale of the data collection is just staggering. A single human operator was using the AI as an entire collection desk. They fed the AI massive daily fire hoses of chaotic multilingual social media chatter. The AI ingested all of it and output highly structured Chinese language dossiers based on internal state intelligence templates. Wow. The explicit goal was to identify exploitable vulnerabilities, what intelligence agencies call grab handles on these specific individuals.

14:00The best way to visualize this is to think of the AI as a master sous chef in a massive, chaotic kitchen. You have a mountain of raw ingredients, millions of tweets, Facebook posts, forum comments in a dozen different languages. In the past, human analysts had to painstakingly chop through all of that to find anything useful. Which took forever. Exactly. Yeah. Now, the AI sous chef instantly chops, sorts, and organizes all that chaotic data and plates a neatly organized, highly actionable dossier for the human intelligence officer who just has to decide whether to act on it. That's a perfect analogy.

14:34And we saw similar automated profiling in the Middle East. There was an Iranian state-aligned operation linked to the Islamic Culture and Communications Organization. They impersonated a dissident group called the MEK, and they used AI to subtly alter organic protest slogans from 2022 in their messaging, aiming to manipulate the diaspora. Sneaky. Very. In a separate case, a UAE-linked operation used AI to sift through vast amounts of diaspora chatter, specifically to build target profiles for human intelligence recruitment. They were looking for signals of financial stress, family separation, visa issues, anything they could leverage to turn someone into an informant.

15:12But let me challenge this for a second. Isn't this mostly just advanced translation and data entry? I mean, humans are still making the final decisions, right? Is the AI actually doing the surveillance? It is doing the surveillance because it is doing the structural extraction and scoring. It's not just translating a tweet from Arabic to Chinese. The AI evaluates the sentiment, scores a political sensitivity of the statement, cross-references it with known associates, and flags the individual's psychological vulnerabilities. Oh, wow. Yeah, it evaluates how sharp the needles in the haystack are and hands the operator a prioritized hit list.

15:47It is performing the high-level cognitive synthesis that used to require a room full of highly trained analysts. A room full of analysts replaced by an algorithm. And that brings us to perhaps the most unsettling transition in this entire deep dive. We are moving from the digital realm of hacking and the social realm of propaganda directly into the physical kinetic world. We're talking about weapons procurement and biological research. This is where the AI's logistical capabilities are being aggressively weaponized. The findings outline a Russian threat actor who used AI to map out a highly complex sanctions-neutral gray market import chain.

16:21Okay, so bypassing sanctions. Exactly. They desperately needed space-grade photovoltaic wafers and oxygen systems for military use, but international trade controls blocked them from buying them directly. So they used the AI to discover third country intermediaries in China and Hong Kong. And they used the AI to draft multilingual requests for quotes, right? They were intentionally obfuscating the true end user. Yes. They used it to navigate the bureaucracy of international shipping, drafting perfect business emails to shell companies to keep their procurement network completely hidden. We saw similar behavior from a China-based actor.

17:01What were they doing? They used AI to reverse engineer a newly disclosed foreign high-power microwave weapon, a system designed to counter drone swarms. They used the AI to map out deliberately obfuscated supply chains to figure out exactly which obscure manufacturers made the internal components so they could develop countermeasures against the weapon. And the risk extends even deeper into the biological sciences. The reporting covers life sciences researchers located in regions where access to these advanced frontier AI models is strictly blocked by the developers. But these researchers utilized zero data retention services and routed their traffic through U.S.-based proxy infrastructure to completely evade those safety classifiers.

17:43And what they did with that illicit access is chilling. One user leveraged the AI to draft an entire highly sophisticated grant application for dual-use viral attenuation research. Wait, an entire grant application? Yeah, the whole thing. We are talking about the AI outlining the central hypothesis, the complex experimental design, the viral dosing schedules, and the statistical analysis plans. The AI mapped out an entire viable biological research program in about an hour. That is just wild. It is if AI is acting as the ultimate amoral logistical fixer. It is doing the quiet, complex, bureaucratic paperwork that actually allows bombs and bugs to get built in the real world.

18:21And that raises a vital distinction. People often worry about an AI suddenly inventing some novel, never-before-seen bioweapon that destroys humanity. But the true danger right now isn't invention. It is acceleration. Just speeding up the process. Exactly. The AI is drastically accelerating the research and development, the supply chain logistics, and the planning for state actors who already know exactly what they want to build. It removes the bureaucratic friction from weapons proliferation. But this raises a massive glaring question. If there are all these strict geographic blocks, firewalls, and safety classifiers placed on the most advanced AI models by the companies that build them, how are these unauthorized labs and rogue state actors getting their hands on this much computing power in the first place?

19:09That brings us to our final point, a process the intelligence community is calling illicit distillation. Right. Let's talk about that. Illicit distillation is essentially an industrial-scale fraud campaign. Distillation itself is a normal machine learning process where you use a massive, incredibly smart teacher model to train a smaller, cheaper student model. Makes sense. But malicious actors are doing this illicitly to bypass safety rules. They use stolen credit cards, false identities, and stolen API keys to barrage a secure frontier model with millions of highly complex queries. They then harvest those high-quality answers to train their own completely unregulated models back home.

19:48The scale of the theft is massive. The intel shows that Alibaba routed over 151 million exchanges through proxy networks in just a few months, strictly to harvest training data. And the methods they used to trick the AI are incredibly devious. Companies like Moonshot AI and DeepSeek deployed what are called cross-session replay attacks. This is a highly technical method used to trick the AI's internal logic. When a highly advanced model solves a complex problem, it uses a hidden chain of thought reasoning process. Essentially, it talks the problem out to itself in a hidden scratchpad before giving the user the final answer.

20:24That internal reasoning process is incredibly valuable intellectual property. So by using a cross-session replay attack, they are basically messing with the AI's short-term memory. Precisely. They manipulate the session memory to force the AI to accidentally expose that raw, hidden chain of thought signature. Zupu and Xiaomi also ran massive extraction pipelines, sometimes going so far as intercepting the complex prompts from their own legitimate users and secretly feeding them into their scraping machines just to generate more illicit training data. Here is where it gets really interesting. So the most sophisticated, safety-tested AI systems in the world are actively being used to seamlessly clone themselves for actors who refuse to play by any safety rules.

21:07They are literally using the AI to steal the AI. Which means the proliferation of this technology is fundamentally uncontainable. So what does this all mean for you, the listener? Why should you care about the intricacies of cross-session replay attacks or how DNS records get hijacked at a hotel? It matters because the old metrics you have relied on your entire life for trusting your digital reality are now fundamentally obsolete. They really are. Whether it's the sender of an urgent email, the author of a local news article about a politician, or even the Wi-Fi network at a fancy hotel, the barrier to entry for causing global scale deception has been permanently lowered.

21:45The labor gap that protected us is gone. And that leaves us with a critical structural reality to confront. If offensive actors are now fully automating their malware generation, their propaganda networks, their target selection, and their vulnerability scanning using AI well, how long until human defenders can no longer keep up? That's the million-dollar question. If attacks are happening continuously at the speed of algorithms, human reaction time becomes completely irrelevant. Will the only viable defense be to eventually hand the keys over to an autonomous defensive AI? We may soon find ourselves in a position where humanity is forced to simply sit back and watch an invisible high-speed algorithmic war raging silently all around us.

22:24An autonomous swarm of offensive drones testing every window in the city, while an autonomous swarm of defensive drones frantically tries to lock them. It is a brave new world. Thanks for taking this deep dive with us. Keep your eyes open and question everything you read.

From the publisher

This paper is an Anthropic threat intelligence report from September 2026 detailing the misuse of AI models by various malicious actors. It describes how state-sponsored groups and cybercriminals have integrated Claude into autonomous attack frameworks to accelerate the development of exploits and the execution of complex intrusions. Key case studies highlight a Russian espionage group automating malware adaptation and financially motivated hackers using AI agents for large-scale data exfiltration. The report emphasizes that AI is effectively collapsing the skill gap between individual operators and well-resourced institutions, allowing for faster and more sophisticated campaigns. Anthropic concludes by explaining their efforts to disrupt these operations, strengthen safety safeguards, and share vital intelligence with global security partners.

More from Best AI papers explained

All 475 episodes
Detecting and countering misuse of AI: September 2026Best AI papers explained · 23 min
Listen in VO