Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning

0:00You know that feeling when you ask one of those really high end AI models? is a tough question. I'm talking about the ones built for heavy lifting, you know, complex math, coding's a kind of deep reasoning stuff. Oh, yeah. You hit enter and then it's just nothing. The spinning wheel. Exactly. The spinning wheel. And it just spins and spins. And you're sitting there staring at the little thinking indicator and you're wondering, is it actually solving the secrets of the universe or did my Wi-Fi just cut out? It's the modern sound of a dial-up modem. Something amazing is supposedly happening on the other end, but the wait is just agonizing.

0:37Right. And don't get me wrong. Usually when the answer finally shows up, it's brilliant. But it feels like we've hit some kind of speed limit. We've built these digital geniuses, but they think at the speed of, I don't know, a leisurely Sunday stroll. It's a real problem. We call it the latency tax. But what if I told you that the slowness isn't really about the computer chips being too slow? It's about how the AI organizes its thoughts. That sounds like a management problem. It is, in a way. And today we're diving into this fascinating new approach called divide and conquer chain of thought, or DCQN for short.

1:09Okay. And the headline here is that researchers have found a way to make these deep thinking models nearly 40 % faster without making them any dumber. 40%. That's huge. That's the difference between a pause and a real conversation. It is. And the secret isn't better hardware. It's teaching the AI to stop acting like a lonely genius. and start acting like a coordinated team. I love that image. So our mission today is to figure out how you take a linear, slow-thinking machine and turn it into, well, a high-speed organization. But to appreciate the solution, I think we really need to understand the bottleneck.

1:46Why are they so slow right now? So imagine the smartest person you know. Okay. Now imagine they're sitting in a room with a single pen and a single piece of paper, and they have a rule. They can only write one word at a time in order. They can't skip ahead. they can't write the conclusion before the introduction. That sounds excruciating. That is exactly how large language models work. It's called stream of consciousness processing. To generate token number 100, the model must have already generated token 99. It's a sequential dependency. So even if the computer running it is a supercomputer, the model itself is stuck in a single file line.

2:23Precisely. And recently, to make models smarter at math and logic, we've been using something called long chain of thought. We're encouraging them to, you know, show their work, verify their steps, backtrack if they make a mistake. Which is great for accuracy. I mean, I want the model to check its work. Absolutely. But the token budget, the amount of internal monologue required, it's just exploded. We used to think 32 ,000 tokens was a lot. Now for these complex reasoning tasks, we're seeing budgets of 256 ,000 tokens just to get to an answer. Wow. So it's writing a whole novel length internal monologue just to solve a math problem.

2:57Exactly. And because of that one word at a time, physics, a 256 ,000 token thought process takes a long, long time to generate. That is the sequential bottleneck. OK, so the thinking is linear, but is the work always linear? That is the million dollar question. The researchers behind this new method realize that actually, no, most complex problems, especially in math, coding or logic, are naturally parallel. Give me an example. Okay, take a really nasty algebra equation. Maybe it involves two different variables or two different cases, like solve for x if y is positive and solve for x if y is negative.

3:35A human mathematician looks at that and sees two distinct tasks. Right, case A and case B. And working on case A doesn't really affect case B at all. Exactly. But a standard AI model has to solve case A, finish it completely, write the summary, and then start case B. It's doing sequential work on a parallel problem. It's like a chef insisting on cooking the eggs, plating them, washing the pan, and then starting the bacon. And by the time the bacon is done, the eggs are cold. It's inefficient. The hypothesis behind DC Code Divide and Conquer is that we can teach the model to recognize these forks in the road.

4:07We wanted to say, hey, wait a minute, I can split this up. So instead of the lonely genius with one pen, we get a room full of geniuses. We get a director and a team of workers. Okay, let's unpack this architecture because I'm picturing a little digital office building. How does it actually work in practice? It starts with a model acting as the director. It looks at the prompt you gave it, say that complex math problem. It starts thinking sequentially, just like normal. It analyzes the structure, maybe does some initial simplification. But it's constantly scanning for an opportunity to delegate.

4:40It's looking for the split. Right. And when it finds a spot where the problem naturally decomposes, it outputs a very specific special token, spawn workers. That's the magic word. That's the signal. The moment that token appears, the system pauses the director and spins up multiple copies of the model. Let's say three workers. Now, this is the crucial detail. Each worker gets the original problem, plus everything the director is snot about up until that point. So they all have the same context. They aren't coming in cold. Exactly. But then the director gives each one a specific, different assignment.

5:16Worker 1, you handle the case where x is 0. Worker 2, you handle the case where x is 1. Worker 3, check the boundary conditions. And this is the part that blows my mind. These workers are all typing at the exact same time. Parallel execution. They're independent threads. So if worker 1 needs to write 5 ,000 words and worker 2 needs to write 5 ,000 words, in the old way, I'd be waiting for 10 ,000 words of time. But here, you only wait for 5 ,000. You only wait for the longest path length. we stop caring about the total number of words generated and start caring about the critical path from start to finish.

5:50That becomes our new proxy for speed. That makes total sense. So the workers finish their jobs. Then what? Do they just shout the answer at you? No, because they might just have pieces of the puzzle. Or they might disagree. Once the workers are done, the director comes back online. It reads all the outputs. That's phase three aggregation. And it synthesizes them. Okay, worker one says it's five. Worker two says it's impossible. So the answer must be. And then the director writes the final conclusion. It's a cycle. Think, delegate, execute, aggregate, finalize. It sounds incredibly logical. In fact, it sounds so logical that I have to ask, why hasn't this always been the way?

6:28If we know how to make managers and workers, why couldn't we just show the AI a bunch of examples of this director script and have it learn the pattern? Ah, now we get to the really interesting part, the failure part, because that is exactly what they tried first. They call it supervised fine-tuning, or SFT. Which is usually the bread and butter of training AI, right? You show the model good behavior, and it mimics the behavior. Right. They took a very capable base model deepscaler, and they used an even smarter model, Claude, to rewrite thousands of sequential math solutions into this parallel director-worker format.

7:03They basically created a perfect textbook on how to be a manager. And the model learned it. It learned the format perfectly. It used the spawn workers tag. It assigned tasks. It looked like it was doing the job. But it got significantly dumber. The accuracy just degraded. Why? If it's mimicking a smart strategy, why does the result get worse? Because mimicking the structure of reasoning isn't the same as understanding how to reason. It's like, imagine you watch a surgeon. You see them wash their hands, put on gloves, and ask for a scalpel. You can mimic all those movements perfectly. But if I don't know anatomy, the patient is in big trouble.

7:38Exactly. The model was role-playing. It was cargo-culled reasoning. It would spawn workers and give them tasks that didn't make sense. It might tell worker one to solve the first half of a sentence, and worker two to solve the second half, which is impossible because the second half depends on the first. So it was splitting things that shouldn't be split just because it knew it was supposed to split things. Correct. SFT teaches form, not function. It taught the model to wear the manager suit, but not how to make management decisions. So the monkey-see, monkey-do approach failed. How did they actually solve this?

8:12How do you teach a model to be a good manager if you can't just show it examples? You have to stop showing it what to do and start rewarding it for results. They moved to reinforcement learning, or RL. The carrot and the stick. Exactly. They set up a system where the model plays a game. It gets points for getting the answer right. But, and this is the key, it gets a penalty if the longest path is too long. So if it solves the problem correctly, but does it in a long, slow, linear way, it gets a slap on the wrist. It gets a lower score. To get the high score, it has to be correct and fast. It has to figure out on its own, oh, if I parallelize here, I finish quicker and I get more points.

8:52But RL is notoriously tricky, isn't it? You can't just tell the model, be faster and expect it to work without side effects. Oh, it was a struggle. They started with a standard algorithm called DAPO. It's a common way to train these models. And it worked for a bit, but then the accuracy just plateaued. It stopped getting smarter. Why? It comes down to something called entropy. In the context of AI, entropy is a measure of randomness or uncertainty. DAPO tends to increase the model's entropy. It made the model too creative, I guess, too chaotic. It was trying so many different weird ways to split the problem that it confused itself.

9:27Essentially. In math, you need precision, not chaos. The researchers found that the entropy was skyrocketing. The model was becoming unsure of its own reasoning. So they had to stabilize it. They switched to a different algorithm called CICP. It forces the model to stay closer to its original stable reference point while still learning the new behavior. That fixed the stability issue. But then they ran into another trap, the laziness trap. Of course. If you give an AI or a human a shortcut, they are going to take it. Exactly. Initially, they trained the model on a mix of problems. Some were really hard, but some were pretty easy.

10:04And on the easy problems, do you really need a director and three workers? No. And the model realized that. It figured out, hey, on these easy ones, if I just blurt out the answer instantly, my longest path is basically zero. I get huge points for speed. So it stopped trying to learn the parallel strategy because it could game the system on the easy questions. It prioritized raw speed over the mechanism of parallel thinking. It wasn't learning to manage. It was learning to rush. So how did they force it to learn? They implemented a remove-easy strategy. They filtered the training data. If the model was already getting a problem right consistently, they deleted it from the data set.

10:40Wow. You're good at this. Great. You never see it again. It's the only way to force growth. They left only the problems where the model struggled, The problems where a linear approach was too slow or just prone to error. It forced the model to practice parallel reasoning because it was the only way to survive the training gauntlet. That is a fascinating insight into AI psychology, if we can even call it that. You have to curate the curriculum to prevent laziness. So after all this, the SFT failure, the entropy chaos, the lazy filtering, what was the final result? The payoff is significant. DC Seaco achieved the same accuracy as the base sequential model.

11:17So it didn't lose any IQ points, but it reduced that longest path length by 35 to 40 percent. So if I'm waiting 10 seconds for a complex answer and now I'm waiting six, that feels massive. It is. And remember, this is on hard benchmarks like the MEM math competition. You're getting the deep intelligence of a reasoning model in nearly half the time. And I noticed in the notes there's something called high length penalty. That's if you want to push it to the limit. If you crank up the penalty for length during training, really punish the model for dawdling, you can squeeze the time down even further.

11:50It becomes a Pareto improvement. You might trade a tiny fractional sliver of accuracy for a huge gain in speed. Now, I have to ask a question that I think a lot of our listeners might be wondering. Is this just majority voting? You know, with ChachiPT, sometimes people just ask the same question three times and pick the most common answer. Is DC Cote just that? That's a great clarifying question, and the answer is a firm no. Majority voting is independent sampling. It's like asking three strangers on the street the same question. They don't talk to each other. They don't coordinate. Whereas DC Cote is a meeting.

12:24It's collaborative decomposition. The director assigns specific different tasks. Worker one is doing something totally different from worker two. They aren't repeating work. They are dividing the workload to build a single coherent answer. So it's much more efficient. But here's the kicker. You can combine them. You can have majority voting of these teams. Exactly. If you use DC Co-T plus majority voting, so running the whole director worker process three times and voting on the results, it beats everything. It beats the base model. It beats standard voting. It's the new state of the art for this efficiency frontier.

12:57It really feels like we are moving away from the idea of one big model, just hallucinating an answer in one breath. That's the big takeaway here. We are moving from linear AI reasoning to organizational AI reasoning. We aren't just building a brain anymore. We're building a project manager. It's interesting because usually when we talk about making AI faster, we talk about better chips, more GPUs, faster hardware. Hardware is brute force. This is algorithmic elegance. Latency isn't just about raw speed. It's about efficiency of thought. If you can do three things at once, you're faster than the fastest person doing three things in a row.

13:34It's parallel processing, but at the cognitive level. Precisely. And what's exciting is that this is just the beginning. I mean, right now the model splits into three workers. Why not 10? Why not a hierarchy managers managing managers? Oh, man. Don't invent middle management for AI. We have enough of that in the corporate world. Fair point. I can't answer your prompt. I'm stuck in a committee meeting. Exactly. But jokes aside, the principle is powerful. We are teaching AI to structure its own thought process. That seems like a higher order skill than just predicting the next word. It's metacognition.

14:07It's thinking about how to think and doing it in a way that respects the user's time. So practically speaking, for us users, this means the wait time for deep reasoning is going to collapse. It means those models that can solve PhD level physics or debug complex code are going to start feeling as snappy as a standard chatbot. Which changes how we use them. If I don't have to wait 30 seconds, I might ask more follow-up questions. I might treat it more like a tool and less like an oracle. Speed changes utility. When the friction drops, the use cases explode. Before we wrap up, I want to leave it with a thought that popped into my head while reading about this director-worker dynamic.

14:46Go for it. We have this director model, and it spawns these workers, and then it reads their reports. But what happens when the workers disagree? Not just about the answer, but about the reality of the problem. It's an internal debate club. But faster than we can blink. If an AI splits its consciousness into three parts to solve a problem, and they come back with conflicting truths, the director has to decide what is real. We are engineering a synthetic version of cognitive dissonance. And solving it in milliseconds. It raises a fascinating philosophical question. Is the truth what the director decides?

15:22Or is it in the consensus of the workers? And if the director has a blind spot, does it ignore the worker who found the right answer because it didn't fit the director's initial assumption? That is a deep rabbit hole. We might need to train the directors to be better listeners next. There's always another training run. Always. That's all for today's deep dive into divide and conquer reasoning. Next time you're tackling a big problem, maybe take a page out of the AI playbook. Don't just think harder. Think wider. Delegate, even if it's just to yourself. Thanks for listening and stay curious.