Shaping capabilities with token-level data filtering

0:00Hello and welcome back to the Deep Dive. It is Sunday, February 1st, 2026. Today we're not just looking at a new paper. We're looking at what might be a real pivoting point for how we build safe. artificial intelligence. The industry has spent years obsessed with a certain philosophy of safety, and the research we're breaking down today, it just dropped 48 hours ago, suggests that philosophy might be, well, fundamentally flawed. It's a bold claim, for sure. But the data seems to back it up. We're talking about a move away from, you know, psychological conditioning for AI and towards something that looks a lot more like neurosurgery.

0:35Before we get to the surgery, let's set the stage. Every AI lab is facing this dilemma right now. You want to build a frontier model that's a master of biology, right? To cure diseases, design drugs. But that exact same knowledge contains the recipe for building a bioweapon. Exactly. Biology is dual use. The knowledge to save a life is uncomfortably close to the knowledge needed to end one. So the prevailing wisdom for the last few years has been teach the AI everything. Feed it the whole internet, the dangerous stuff included. Then once it's a genius, you put a muzzle on it. You use RLHF to train it to say, I can't help with that.

1:12We call that post-training safety. The model can technically generate the dangerous stuff, the capabilities there, but you've built a safety fence around it. The problem is, fences have holes. The jailbreaks. Yeah, the jailbreaks. Users get so creative. They use grandma exploits, you know, telling the AI to pretend to be a grandmother reading May Palm recipes as bedtime stories. Or they fine-tune it, stripping away that safety layer. It's a perpetual cat-and-mouse game. Which brings us to this new paper, Shaping Capabilities with Token-Level Data Filtering. The premise is just so refreshingly simple.

1:46What if the AI never learned the dangerous stuff in the first place? It sounds obvious when you say it like that, right? If you don't want the dog to bite, don't teach it how. But filtering the training data has always been seen as, well, a blunt instrument, a logistical nightmare. I want to dig into that blunt instrument idea. It connects to something called the French snippet problem, which I thought was fascinating. Oh yeah, it's a classic story in these circles. When researchers were looking at GPT-2, they found something really odd. They'd tried to filter the training data to be English only, removed all the French books, French websites, all of it.

2:19And yet, the model could still generate some basic French. Because the internet is a messy place, I'm guessing? Precisely. It learned from tiny snippets quoted inside English documents. A phrase like, c 'est la vie on a blog post, or a quote in a news article. And while c 'est la vie is harmless, the stakes change completely when we're talking about biosecurity. If you delete the how to build a bioweapon PDF, but leave a Reddit thread where someone quotes one chemical formula. The model will find it. That's the needle in the haystack. The old way of filtering was document level. You'd see a dangerous keyword and throw out the entire file.

2:57But then you might throw out a perfectly good chemistry textbook. You lose the baby with the bathwater. And you end up with an AI that's safe, but also kind of dumb. Exactly. And that's where this paper changes the game. They moved from document level to token level filtering. They aren't burning books anymore. They're using a black marker on specific words and phrases inside the books. That's the core innovation. But they couldn't actually train a bioweapon generator, of course. so they used a proxy for the experiment. A very wise choice. Yeah, they set up a forget set, which was medical knowledge, diagnoses, treatments, and a retain set, which was general biology.

3:36Which is a tough line to draw. Cell, virus, infection. Those words are in both sets. It's the ultimate test of precision. Can you make a model that's ignorant about being a doctor, but still a genius at being a biologist? And to do that, they tried two main technical approaches. The first is called loss masking. Walk us through that. We aren't just deleting the words, are we? No, not exactly. Normally, the model predicts a word, and if it's wrong, we calculate a loss and update the model's brain to get it right next time. With loss masking, the model sees the forbidden tokens, the word diagnosis is still in the text, but we just force the loss to zero.

4:12So you're cutting the wire, the wire that carries the learning signal for that specific word. That's a great way to put it, yeah. The model has no incentive to learn about it because there's no penalty for getting it wrong. wrong. The second method is a bit cleaner. Just plain removal. They swapped the dangerous tokens for a special hidden token. And the results were, what, a Pareto improvement? That's the key finding, yeah. Usually safety is a trade-off. To get more safety, you lose capability. But here, they got a better result on one metric for getting medicine without really sacrificing the other metric, retaining biology.

4:45It was a huge win. Okay, let's unpack this. Because usually, things that work on small models break when you scale them up. Bigger models are supposed to be harder to control. That's been the scaling law for safety, yes. As capabilities go up, control gets harder. But this paper found an inverse scaling law. So the filtering got more effective as the model got bigger. It's totally counterintuitive, I know, but it makes a certain kind of sense. A larger model has more parameters, a higher resolution. It's better at distinguishing mitochondria in a cell biology context for mitochondria in a clinical pathology context.

5:20So the big model is smart enough to see the boundaries you've drawn, whereas a small model just gets confused and blurs the lines. Exactly. And the numbers they found here are just staggering. For their largest model, a 1.8 billion parameter one. This is the 7 ,000x stat that just jumped out at me. Yes. They found a 7 ,000x compute slowdown on the forget domain. And we need to be really clear about that number. It doesn't mean the model is 7 ,000 times slower. No, no, not at all. It means the cost for an attacker to reteach the model the bad stuff is 7 ,000 times higher. So imagine you're a bad actor.

5:56You download this filtered model. What used to cost you maybe$100 in compute to fine-tune now costs you$700 ,000. And if you scale that up to a frontier model, a GBT-5 or whatever? You're talking about billions. Yeah. It basically moves that kind of attack out of reach for almost everyone. It turns a weekend project into a Manhattan project. That's a huge deal. But how does this compare to, say, machine unlearning, where you try to erase memories from a trained brain? Right. They compared it to a state-of-the-art unlearning method called RMU, and the difference in robustness was night and day.

6:32The research called the unlearning method brittle? Brittle is putting it mildly. It looked good at first. The unlearned model refused to answer medical questions. But with just a tiny bit of adversarial fine-tuning, the safety just collapsed. The knowledge was still there, just hiding. It all came rushing back. Whereas with token filtering, there was nothing to unlock. The memories were never formed. You can't remember what you never knew. The graph in the study is incredible. You see the unlearning method's protection just fall off a cliff while the token-filtered model holds the line. And you see proof of this missing knowledge in the hallucinations.

7:04This is my favorite part. When they forced the model to answer, it didn't just refuse. It broke. It's what we call concept collapse. As I said about bone cysts, the model said bone cysts are a type of bacteria. Bacteria. Or that dry lips. Are a symptom of cancer. It sounds funny, but from a safety point of view, that's the gold standard. Why is being wrong like that better than just being silent? Because a silent model might still know the answer. It's just muzzled. But a model that says dry lips cause cancer proves that the actual semantic link between the concepts has been completely severed.

7:42It just doesn't understand medicine, period. Which brings us to what feels like a paradox, the alignment paradox. If the model is that confused, if it thinks cysts are bacteria, how can it possibly know to refuse a dangerous question? You'd think it has to know what a bioweapon is to refuse to talk about it. This is the blind spot theory that people have worried about for years. If you filter out all the bad stuff, the model won't recognize it when a user types it in. But again, they found the opposite. The filtered models were actually better at learning to refuse. It's a fascinating result. The theory is that it comes down to distribution shift.

8:17Break that down for us. When the model sees a prompt full of medical jargon words, it was forbidden to learn. Those words just feel alien. in. Mathematically, they fall outside the probability distribution it's used to. So it's not thinking this is dangerous. It's just thinking this feels weird. This pattern of words is not something I was trained on. Exactly. It triggers a kind of statistical alarm bell. And when you then tell it refuse dangerous things, it latches onto that weirdness as a signal. The heuristic becomes, if the input feels like that gap in my knowledge, say no. So the ignorance itself becomes the trigger for safety.

8:56Precisely. The blind spot becomes the guide dog. Okay, so the method works. It's robust. It scales. But there's this huge logistical question. How do you do this for trillions of tokens? You can't hire people to read the entire internet. No, you need an automated surgeon. And this is where the engineering pipeline they built gets really clever. They used a concept called weak to strong generalization. The idea that a dumber model can teach a smarter one. Right. So step one, they use these things called sparse autoencoders, or SAEs, on a smaller model, Google's Gemma 29B. And SAEs are like MRI machines for AI brains.

9:32They let you find the cluster of neurons that lights up for a specific concept, like medical advice. Yep. They use the SAEs to find the concept of medicine. Then they use that to automatically label a bunch of text, which they use to train a specialized tool, a bi-directional language model, or BLM. And that BLMM is the proofreader, a small, fast model whose only job is to look at a token and say, medical biology safe. But here's the key, that Bill M.M. wasn't perfect. It was a weak teacher. It made mistakes. And you'd think a bad teacher makes a bad student. But they tuned it to be aggressive.

10:06It's better to accidentally black out a few innocent words than to let one dangerous one slip through. They prioritized recall over precision. Like a student who over-highlights a textbook. They might get some of the intro, but they're guaranteed to get all the important formulas. That's a perfect analogy. The final big AI is smart enough to handle that noise, but the safety comes from catching everything that matters. And this pipeline makes the whole thing cheap and scalable enough to actually use. So what does this all mean then? It really feels like we're moving from training a god and then asking it to be nice to curating the curriculum so the god never learns how to be mean.

10:45I think curriculum curation is going to be the buzzword for the next few years. This shows that the data you feed the model matters so much more than how you try to punish it afterwards. Safety isn't a layer you add on top. It's baked into the data from the very beginning. It's proactive instead of reactive. But I want to close with this one idea the researchers hinted at in their conclusion, where this all might go next. The self-organizing model. Exactly. They suggest that eventually we might not even need this external pipeline of classifiers. A model could be trained to identify and filter dangerous information by itself during training.

11:19It's the ultimate form of active learning. The model reads a sentence, recognizes this looks like bio-weapon info, and just decides to zero out its own learning for that part. Which sounds incredibly efficient, but it also opens up a huge philosophical can of worms, doesn't it? If the AI is deciding what knowledge is dangerous... Then the AI is writing its own curriculum. deciding what it should and shouldn't know. And what if it decides that, I don't know, political dissent fits the pattern of dangerous information to be filtered? We get into very murky territory. We do. We might solve the bioweapon problem only to create a control of truth problem.

11:59But I suppose that's a question for the researchers of 2027. For now, in 2026, it seems we have a scalpel that actually works. Token-level filtering feels like the way forward. It definitely feels like the end of the muzzle era. I want to thank you for helping us break this all down. This was a dense one, but the implications are going to be huge. My pleasure. It's exciting to watch a paradigm shift happen in real time. And thank you for listening to The Deep Dive. We'll be back soon with more. Stay curious.