AI Trends 2026: OpenClaw Agents, Reasoning LLMs, and More with Sebastian Raschka - #762

28:35Sebastian Raschka:Yeah, I would say so. I mean, I mentioned my native Mac apps, even like something, I have a Mac app where I just put in a PDF and it exports the PNG, WebP and PDF versions in a certain resolution. And it took multiple tries, even with, there was back then Codex 5.2 to get really everything, all the buttons working correctly. Like you said, it was not one-shotted at all. It was multiple iterations to get it to work and even something simple like that. And then I sometimes wonder, are my instructions maybe bad? Or maybe I wasn't clear. Maybe you have to say, please test everything thoroughly and make sure everything works and blah, blah, blah.

29:14Sebastian Raschka:Maybe you have to be super explicit about that. And we are not that explicit because we kind of assume it would make sure everything works. Or maybe these cases we see are just lucky. You know, like sometimes on certain things, it just happens to work very well. So I don't know for sure, but I agree with you that it's not all what it seems when someone shows you, oh, I one shot at this. I don't think that's reflective of how things work today. So let's switch gears a little bit and talk through some of the key areas that you expect to see continued innovation around with LLMs in the upcoming year.

29:55And then, you know, for each of them, we'll dig in and talk a little bit about the recent history and where you expect to see things going.

30:03Sebastian Raschka:What are the big themes for the year? I would say it's still going to be the reasoning. We can maybe go into more detail there because it's a very broad topic. So like pushing more on the reasoning front, the post training. The second one, I would say, is also inference scaling, like more sophisticated techniques that they are partly related to training, but mostly how to use the LLM after training. And then I also think we will see more of this agentic type of use, because right now, mostly LLMs are focused on like a turn by turn and how to, like people will double or companies will double down on this loop, basically running an NMS loop like Moldbot and optimizing for that.

30:53Sebastian Raschka:And I think these three things will be mainly the biggest, I guess, focus areas for companies. So let's dig into reasoning to set the stage for where you think we'll be heading in 2026. What do you think were the big advancements in 2025 around reasoning? So yeah, the biggest advancement was, I mean, first OpenAI 01, which got everyone excited about it. And then OpenAI 01 was using both inference scaling and, I mean, no one knows for sure because there's no paper, but likely also training techniques. But then R1, DeepSeek R1, they published their reasoning pipeline. And I think that was like really something that took off where a lot of other companies also doubled down on that.

31:40Sebastian Raschka:But it's still very new in the grand scheme of things. It's just like a year old. And I was recently working on a chapter on reasoning. There were so many improvements to the algorithm. I mean, just the other day, I've compiled a list of 15 different tweaks and improvements from basic things changing sequence level, log props to token level, but then also GDPO by NVIDIA. Lots of progress there, and I think we will see more of that. First, also, one reason is that with pre-training, we have seen basically that, I mean, it still works. and I think it's still the biggest part of the pre-training, of the whole training pipeline because it's just so much data and very expensive.

32:22Sebastian Raschka:But the R &D, like the research and development of the focus of the research team, I think it's more focused nowadays on the post-training, like getting more performance out of that because it's more like the newer paradigm and there are still low-hanging fruits to be picked, where in pre-training it's already pretty sophisticated, where, yeah, you still need a lot of data, you still need a lot of compute, but there is no, let's say, like there's nothing you can really do much there compared to post-training in terms of changing up the algorithms to get more performance. Of course, you can still do that and you will still get better results if you use more data, optimize the data mix, maybe multi-token prediction and these types of things.

33:04Sebastian Raschka:But most of the interesting things are happening now on the post-training front in the reasoning realm, basically. So I think we will see more there. On the reasoning front, the one topic that I heard about, I heard come up quite a bit last year, is the idea of verifiable rewards. And I think that led to a lot of the advancements or contributed to a lot of the advancements that we saw in terms of coding models. Can you talk about that as a paradigm and some of the big milestones that we've seen there over the past year? Thank you for the question. It's actually a really, really important point.

33:49Sebastian Raschka:So the reasoning training is essentially mainly based on verifiable rewards, which means there are tasks where you can verify the answer. So for example, in DeepSeq R1, the verifiable rewards were coding and math. So with math, for example, you ask the model to output the final answer in a boxed format in the latex. It's like a latex command, like boxed. And then you can have like a deterministic, like a regex or deterministic code to extract the answer. and then you can use something like Wolfram Alpha or SymPy to compare this answer symbolically to a reference answer. Like if 2 over 3 matches 2 over 3 or 4 over 6 matches 2 over 3, it's essentially the same answer.

34:36Sebastian Raschka:But you can symbolically double check the answer and get a reward signal whether it's correct or not. And this is actually great because you can kind of like infinitely, you can evaluate infinite numbers of answers because before with, I mean, it's still an important point, but the reinforcement learning with human feedback, yeah, you need human feedback, essentially. You can train a reward model to approximate that, and it's part of the training where you get a score for each answer, but it's not quite as, let's say, accurate as a really correct answer, like where you can verify the answer, like there's an absolute, it's math, it's either correct or not.

35:12Sebastian Raschka:And you can have, if you have something like that where you can verify the answer deterministically cheaply, you can have the LLM generate infinite answers. You can say, okay, generate 60 ,000 answers for this problem. And then you can calculate the reward on all of them in a fraction, like a very short time. It's still expensive to generate these answers, but you don't have vagueness and you don't have to have, let's say, human evaluating these answers. And so I think that helps scaling these things. And the same with code, where with code in the DeepSeq R1 paper, where the original approach was to have the code and make sure that the code compiles, basically, like if it compiles correctly.

35:54Sebastian Raschka:And you can use also a code interpreter for that. I think, I mean, both are great, but I think this is just the beginning. I mean, we will probably see this being extended to have more than just the correctness reward. I mean, there are already other types of rewards that are being added. For example, a formatting reward where you want the model to use, I mean, it's not required, but some companies prefer to have the thinking in the think tags. So they have like a token think and then a closing token think closed, like in HTML, like the opening and closing tag. It's not required, but it can be helpful to have it because then you can parse out the intermediate stuff and do something with it where you can train the model to output this structure.

36:46Sebastian Raschka:And it's called a format reward. So you can have multiple types of rewards added to this thing in addition to the correctness reward. And I think we will maybe also see interesting things there where people will come up with formatting rewards or like auxiliary rewards that help the overall model to learn. And one thing is also they tried that in a DeepSeq R1 paper to evaluate the answer explanation instead of just looking at the final score or if it's correct or not. Making sure or like evaluating if the reasoning, like the explanation is correct or not. Is that a process reward? Exactly, yeah.

37:27Sebastian Raschka:They use, this is called process reward model that it's basically another model that you train to give a score for this explanation. But I remember, I mean, it's been a while since DeepSeq R1 came out in the paper they had a section that they listed that as a failed attempt or unsuccessful attempt. So they tried it, but they thought, okay, this increases the chance of reward hacking. And then it was just not worth it. It's more expensive. It resulted in reward hacking, the model exploiting thing, because it's easier that way for the model to kind of cheat, to kind of mislead the model that evaluates the model, basically.

38:07Sebastian Raschka:And so it is still tricky to do that. But I think also in the recent months, there were some more interesting success stories like DeepSeek Math version 3.2. They use something like that where they evaluate also like the whole answer with a rubric, have another model for that. And then they have another model that evaluates that model that has the rubric and so forth. It's like multiple levels. And, but that seems to work. And they had like ablation studies that show that this is actually helping. And I think we will see also more of that. It's just like a very new paradigm, like making the reasoning training more sophisticated, essentially.

38:46Right now, the verifiers are focused on like math and coding, and that's because, you know, for a given response, there's a concrete ability to verify. Do you see this verification paradigm expanding beyond math and code? And I think, you know, in part, you know, the focus on math and code is successful because, you know, even though not all LLM responses are about math and code, those things kind of have an inherent, you know, logic or reasoning capability in them. And so the ability for the model to reason generalizes to, you know, non-math encoding problems. but do you see a focus on expanding this idea of verification beyond math and code types of problems?

39:42Sebastian Raschka:Yes, so it's actually a very interesting and important point. How can we, I mean, you mentioned that if you train the model on math problems, reasoning on math, it will also become better at reasoning in general but then it would be even better if you have a target domain to train the model specifically on that target domain, on reasoning in that target domain. I think you're right. There will be more of that. For me right now, I just lack the creativity right now to come up with examples of problems that can be verified. But I would say maybe something even like biology related where for like a drug design, like a pharmaceutical drug design, a protein structure modeling, where you have like physical constraints So there are like the angles between atoms, they can only have a certain angle and so forth, where you could probably have like a physics type of equation that double checks whether the generated molecule adheres to these certain types of formats and then have that as a form of reward when you're training the model.

40:51Sebastian Raschka:I mean this is maybe not a typical case of reasoning because well what is the reasoning explanation when you're generating a model right I mean but in general like something like that for other fields and in the worst case you can always I mean this is more like a rough approximation but you can always train another model that provides the correctness reward this I think this is more challenging though because it's susceptible to reward hacking even going back to back in the day, generative adversarial networks, where it's easy for the generator to collapse. You have the discriminator, which says, is this image real or generated?

41:34Sebastian Raschka:And then it was like the setup where you train a generator to fool the discriminator. And the discriminator gets better at distinguishing. And you have almost like a similar setup. You can use it to say, give a reward or not. but then the model may or may not exploit it at some point, learns a trick if I only generate this one word or something like that, then I fool that evaluator. But I think maybe we'll see also more of that, like developing AI-based reward models, essentially, that can be used in other fields to train better reasoning models. beyond increased focus and and tweaks to the verification models are there other areas that you see as contributing to stronger reasoning going forward um yeah i do think it's also i mean the training is one part but the other one is the inference scaling that you can get much better performance if you use simple let's say not simple i mean simple in quotation marks but If you, after training, nothing is simple.

42:44Sebastian Raschka:If you spend, let's say, if you spend more compute, essentially. Like inference scaling is all about, the definition is essentially spending more compute after training during inference when someone uses a model to generate the answer. And you can do it in multiple ways. I mean, reasoning models themselves, they are already kind of like a form of inference scaling because they generate more tokens than regular models. The explanation is longer than a regular model provides. But it helps often the LLM to reach the correct answer. But this is more like a sequential inference scaling. You can also have parallel forms of inference scaling where you just generate multiple answers.

43:26Sebastian Raschka:And that's called self-consistency. So for example, if you have a math problem, you can have the LLM with different temperature settings, answer the question multiple times, and then you take a majority vote or something like that. I mean, there are different ways you can do it. There's also, there are different scoring methods or other LLMs that look at all the answers and give you the most likely correct answer. And with that, you can also boost the performance of the model. It's more expensive though. So it's always like this, yeah, it's not like a one size fits all. You don't want to use it all the time.

44:01Sebastian Raschka:You use it when you need it. But I think what will be interesting is to improve the way to tell when it's lean. I think when ChatGPT, was it 5.1 or 5 launched, they had like that automatic setting that we talked about in the beginning. It was very bad at the beginning, but I think it got much better over the months and years. And I think I'm not quite sure we have anything like that in the open source, open weight ecosystem. Maybe listeners may correct me here, but I think something like that I can see also being more important because on the one hand we are developing these very expensive models that can solve very hard problems like in this math olympia but we don't want to use them all the time because they are slower and more expensive and there's also going to be more like a focus at the same time on cheaper models so for example just the other week Gwen3CoderNext or next coder sorry Gwent 3 Next Coder came out, which is Gwent 3.

45:02Sebastian Raschka:Gwent 3 is one of the most widely used open weight models because they have like a lot of really high quality models in all different types of sizes. But they also, the next model, it is essentially like a hybrid. It's not like a pure transformer anymore. It has, like, it's inspired by states-based models to make things cheaper. But then it's like always this trade-off. People are developing higher accuracy models. people develop cheaper models. And I think, yeah, I mean, one way would be changing the architecture to control the quality and price. The other one is inference scaling. But I think right now it's in the open-weight ecosystem, it's not quite as popular yet.

45:45Sebastian Raschka:So I think we will also see more of that in local tools and so forth. I don't know that I know of any, like a open source project or a model that incorporates this. But from conversations, I do get the sense that a lot of companies that are building around, you know, the Quinn models, for example, and these open weight models commonly have like a router component in their architecture that tries to assess the complexity or category of a prompt and routes it to the right, you know, model and prompt. that is either most economical or maybe post-trained for better responses, that kind of thing. My sense is that that's the common approach to addressing this challenge that you're describing.

46:39Sebastian Raschka:Now that you mentioned that, another example came to mind. It's the GPT-OSS model, the open source model by OpenAI, which came out last summer. And in that model, even if you use a very simple inference or like a simple tool like OLAMA or any comparable tool, you can set the reasoning effort in the system prompt. So you can say no mild, medium, high reasoning effort and then scaling inference based on the reasoning effort. But I don't think there's any other technique really automatically incorporated like self-consistency or self-refinement. It's mainly you have to, as the researcher, do it yourself most of the time.

47:19Can you talk a little bit more about the self-refinement and self-consistency and how folks use those techniques?

47:27Sebastian Raschka:Yeah, so self-consistency and self-refinement are two examples of inference scaling. I would say the biggest difference between the two is one is a parallel technique, where self-consistency as a parallel technique generates multiple answers. And you choose, let's say, the correct answer based on majority vote, or you can have a scorer who assesses these answers. but then people call that technique best of n like best of n answers. Okay best of n or quorum or that kind of thing. Yeah it's essentially an ensemble technique like classic ensembling almost and the other one is self-refinement where you have the LLM generate the answer and then you feed the answer to another LLM or to itself and say, here's the answer.

48:17Sebastian Raschka:This is the question. Write a summary if the answer is likely answered correctly and what are weaknesses and like a rubric. Almost you provide a rubric with certain things that the LLM should check. And then it gives you back a report and says, well, this could be better. This is like the incorrect explanation doesn't match the final answer. and then you feed that output back to the original LLM and say, hey, look at that report and refine your original answer based on the report here. And often this can lead to the LLM improving its own answer. It's almost like this phenomenon. Sometimes you ask Chachapi something and it gives you something and like, wait, that can't be right.

49:03Sebastian Raschka:I don't know, you ask something about like when was a certain model released and you know, okay, this can't be right. The year is totally wrong. And you tell it, hey, you are incorrect. You made a mistake. And oh, yeah, you are right. I made a mistake. And then it tries again and it's better next time. And it's almost like that same mechanism where it self-refines its answers. It can, I mean, based on my experiments, it can also sometimes make answers worse. like it will overthink or it was originally correct, but then the feedback is weird or bad and then it makes the answer incorrect. So it's not like a foolproof technique.

49:42Sebastian Raschka:It's also with caveats, but in the DeepSeq Math version 3.2 paper where they had a self-refinement in a more sophisticated way, where they had a third model evaluating the evaluator, they really showed, again, a nice graphic or plot where they showed how much the accuracy can improve. And basically from a, I don't know the numbers off the top of my head, but if they cranked up the self-refinement and self-consistency, they were able to have like gold level performance in certain math competitions, which was very impressive, given it was still the same model as they used before, but they just cranked up the inference scaling, basically.

50:22One thing that's interesting, kind of reflecting on these themes that we're discussing is just how they're all very interrelated. You know, so reasoning is a key theme. Reasoning is enabled by inference scaling. Inference scaling, a lot of what we're hearing as we talk about this is like loops and recursion and those kinds of ideas. And those are key ideas. And the third key theme that you mentioned, which is kind of agentic uses of LLMs. With that as a segue, talk a little bit about what you've seen thus far around agentic and what you think is exciting in that space.

51:08Sebastian Raschka:I would say, yeah, the agentic use cases, it's even like simple, I mean, again, in quotation marks, simple, things like codecs or cloud code where it does just multiple iterations to solve a problem. It's not just like one shot. But it's more like doing a task rather than just providing an answer. I mean, Maltbot would be another example of agentic systems. Agentic is also, I would say, almost like a not well-defined term because people use the term differently. But for this podcast, maybe we can think of agentic as something that runs in a loop. And I think, yeah, that is something we will see more of.

51:55Sebastian Raschka:Recently, Cloud Code and GPT 5.3 Codex, the app, they added a lot of these tasks where you can even schedule something and it does something on a reoccurring basis, for example. And I think we will see more of that. It's just like the beginning. It will be more like plugins. And I mean, it's still the same LLM. It's just like how we use the LLM and how to get the most out of it, out of the context, feeding back the context. And I think there has not been that much focus on this in the open-weight, open-source community. The focus there is more on developing the LLM itself, where companies though, like OpenAI, Cloud, they are more like, okay, let's build these tools so we can actually do more and more impressive, bigger things with these LLMs.

52:43Sebastian Raschka:And I think maybe by the end of the year, we will have systems that can reliably book a trip to, you know, some holiday vacation destination where this becomes more and more common. I mean, there were already tools that promised to do that. I forgot the name, but I think it was called Devon, something like that. It might still exist, but I think this is just the beginning. Yeah. Oh, yeah. Manus, right. Yeah. But I think it's just like the beginning. And also most people, I don't think they need like a full-blown thing that can do everything. They just maybe need like a plugin for Excel to have certain intervals where it updates certain things.

53:25Sebastian Raschka:And then Excel spreadsheet goes into the, let's say it goes to the internet and pulls the recent stock price or something like that. But like in a type of loop type of setting, essentially. One of the things we heard a lot about in the context of agentic uses of LLMs, I think over the past year, maybe two years, is the idea of like multi-agent systems and like decomposing a problem into independent agents with kind of their own personas and that kind of thing. And I think that, you know, the whole open claw idea, like even today, I'm seeing a lot of, hey, I created my AI team and there's this employee, my AI employees, right?

54:20And there's this employee, that employee, that employee, and they talk to each other using, you know, Slack or notebook or whatever. And like, what have you seen with regards to, you know, kind of from a concrete, you know, builder or technical perspective, this like multi-agent use case? Are you finding, you know, folks getting a lot of value out of that?

54:46Sebastian Raschka:To be honest, I wish I had a really good answer or interesting answer, but this is something I've not explored personally where most of my experiences are like single use case where it's one LLM that provides solutions or tackles a specific task, but it's mostly not interacting with other agents. I think here, though, I mean, I also see it's more like a context engineering problem where how do you, I mean, so the LLMs themselves, I don't think they are the bottleneck. It's more about how you, let's say, provide the result, get the results and provide them to another LLM. I mean, in that sense, it's almost like a form of when you do image or video generation where you have one LLM parsing the text or improving the input and then passing that to the part of the model that generates the output, the diffusion model part or the transformer based diffusion part, where I think it's more like a sophisticated form of that.

55:51Sebastian Raschka:how do we provide the right context to the different agents? And it could be from basic databases to using Slack where one model outputs something there and via the API, the other model ingests it. And I think, well, that is, I think, also something that is just getting started also with Moldbot and OpenClaw. And I think we'll be seeing a lot more of that. But yeah, so that's all I can say to it because I personally don't have any concrete experience. I haven't worked on this myself yet. Do you have a sense for where we'll see focus and innovation around these kind of agentic uses in the upcoming year or maybe what the gaps are, what really needs to be worked on in order for them to kind of come into their own?

56:44Sebastian Raschka:I do think it's still like each L &M has its own kind of like failure rate at some point where so the progress is usually measured how long the LLMs can work autonomously like how long can they work until they fail and the more models you add the higher the risk that one of them fails if they depend on each other and I think improving the model itself here will also help improving the whole system basically as the main way to improve the performance. But then I can also see right now, most, as far as I know, based on what is publicly available, these are still the vanilla LLMs that are in Cloud or in other APIs.

57:31Sebastian Raschka:They're not specifically trained to interact in a multi-agent setting. And I think in that sense, if you prepare data for training these agents in a multi-agent setting, like a fine-tuning type of situation, I think you can also get more performance out of them. We have seen that, I mean, even for simple things like codecs. So GPT 5.2 or 5.3 codecs is not the same as GPT 5.2 and 5.3. These are models that they forked off and then specifically trained to work with the codecs app, basically. And I think something like that we will also see for these agent models. It's just harder for, let's say, the consumer to do that because we don't have access to these models.

58:18Sebastian Raschka:So we're kind of like dependent on the person who owns the LLMs, who hosts the LLMs to do this type of training, basically. So, yeah, I think I can see companies also developing something like this. I mean, if I had to bet Claude and OpenAI, they really paid attention to what Mold, what OpenClaw is doing and maybe coming up with their own version of that. that is maybe even more capable because they control the model and they can fine tune it for certain interactive multi-agent types of environments. One of the things that's interesting looking back is that a lot of the, you know, things that we might look back and see as big advancements over the past year or two years, from an architecture perspective, they're relatively incremental, like the fundamental core architecture you know there have been a handful of proposals of like where we might go beyond LLMs but the core has been fairly stable you know do you agree with that where do you see you know how do you think about the future of LLM architecture?

59:26Sebastian Raschka:Yeah that's a interesting question so I would say everything I'm saying here with an asterisk because DeepSeq version 4 is not out it might It changed everything completely in terms of what I'm saying. But if we just look at 2025 up to the second week of February, I don't think there were any fundamental changes in terms of the state-of-the-art architecture. So I think one thing we have to distinguish between is there are architectures that are more geared towards doing the same thing more efficiently. And there are architecture changes that are geared towards let's get more modeling performance accuracy out of the model.

1:00:07Sebastian Raschka:If we, first, if we look at those models that push the state of the art, the modeling performance, there haven't been that many changes really recently. I mean, looking at 2025, a mixture of experts models have been making a comeback. I mean, there were other models like Mixtral and DeepSeq MOE before, but they really became popular after DeepSeq version 3 came out. And DeepSig version 3 became popular because of DeepSig R1, which is basically a fine-tuned version or post-trained version of DeepSig version 3. But then a lot of companies adopted this architecture. I think Kimi straight up used that architecture and they scaled it from 670 billion to 1 trillion parameters.

1:00:52Sebastian Raschka:Or even like the European company Mistral AI, they used DeepSig version 3 architecture. So a lot of people are, I would say they are not gambling in that sense. And let's try something different. They take something that works and try to make progress or differences in terms of changing the data and the algorithms. But that doesn't mean there are no new ideas. So, I mean, DeepSeq version 3, besides MOE, the mixture of experts, they did have the multi-hat latent attention. I think it was also in one of the previous papers, but multi-head latent attention is essentially like a tweak of the attention mechanism where you have like a intermediate, smaller, compressed state of the queries and values.

1:01:43Sebastian Raschka:I mean, all of them actually are, but the keys and values are important to compress because then your KV cache becomes smaller. so you don't store the full keys and values in the KV cache, but a compressed form. But then you reconstruct the keys and values from the compressed form in inference. So you are basically trading off compute with memory. But also maybe to explain this a bit better, you can think of it as a LoRa, like the low rank adaptation. So basically you project it down into a compressed space and then you project it up again. So that's basically a multi-latent attention. So that's like an interesting tweak, I think, in 2025, 26 that people adopted.

1:02:27Sebastian Raschka:And then it was, again, DeepSeq version 3.2 that had another, I would say, tweak, sparse attention. I mean, sparse attention is also not new, but there's always been this research on how we make attention cheaper because it scales quadratically with a sequence length. and there have been hundreds, if not thousands of papers. But, you know, with papers, I'm always a bit, I mean, the ideas are interesting, but I'm always a bit careful and I'm waiting always to see that in production, in quotation marks. What I mean is to see it in a flagship model, like because the idea might work well if you are only focused on a small model, but things may fall apart once you scale the model to 500 billion, 600 billion, 1 trillion parameters.

1:03:19Sebastian Raschka:So, and DeepSeq here is a nice case study because they do have this flagship model. And if they use something in that flagship model, you basically know it works at scale. And they have their own version of sparse attention. I think they call it literally DeepSeq sparse attention. And it's, yeah, it's instead of, so they have like a lightning indexer, like a small cheap model in a sense. Instead of for one token paying attention to all the previous tokens, it's more selective. It selects which tokens it pays attention to. So it's kind of like a mask. So you are calculating a mask over all the tokens to select a subset to make it cheaper, to make it scale sub-quadratically basically.

1:04:04Sebastian Raschka:And there have been like these types of tweaks, but it's not fundamentally changing how attention works. It's still the same attention mechanism, But how do we, yeah, how do we make it cheaper, basically? So I think that is something where people hone in on what works at the moment. But we will see maybe in 2026, maybe one of the flagship models will have a fundamentally different approach. Little companies, I mean, not little, sorry, companies, but little changes have also been made in terms of alternative architecture. So we mentioned Quint 3 earlier. So GEN3 is one of the flagship models. Right now, it's maybe not at the top anymore because it's a bit older.

1:04:48Sebastian Raschka:It came out in summer. But usually when GEN3 or when GEN models came out, they are usually top of the leaderboards. They had also a parallel version of their model. They called it GEN3 Next. And that one tried something different. They had like a hybrid attention mechanism with a gated deltanet basically to have more like a state-based model approach where it's more like linear where people are trying, but it's not necessarily like their flagship model. Like they are in parallel trying other things. And I think this makes sense because, yeah, you don't want to put all your eggs in one basket, basically.

1:05:26Sebastian Raschka:You want to have a good model and then maybe on the side try something and then maybe scale it up later if it works well. What about continual learning? That comes up frequently as an opportunity, particularly before we got really good at incorporating in tools and the ability to do searches because models would get stale very quickly. but you know there's still this interest in having a model that you know we can you know we can keep its training data updated we can delete we can delete things we can incorporate new knowledge like do you foresee significant innovation in that area yeah i think this is like maybe the biggest dream in the sense of like hey how can we make the model improve itself like the biggest uh i guess achievement right now that could be made if that gets if someone finds out a way that this works but i think right now there is no even no pathway to this uh like there's no yeah there's nothing really that is like where you would say oh that's the thing that will give us a reliable or continual learning but that being said, I think, I mean, there are already forms of continual learning, I would say.

1:06:49Sebastian Raschka:I mean, even something like, well, I mean, it's more like controlled, like instead of the model automatically updating itself, people would collect data from like the recent internet or recent tasks and then carefully update the model, essentially. I think it's more like that where it's not that we don't update models, but we also don't do it fully automatically. It's like a semi-automatic almost type of thing. I think also it's like that not only because that's more reliable. So because, yeah, it's risky to just update a model on new data, but it's also because of resource constraints. Because, for example, I don't know how many copies of the model OpenAI has, but I mean, you can't definitely not have a single copy per user.

1:07:43Sebastian Raschka:That would be way expensive. I mean, everyone would have to have a little supercomputer at home or like a$100 ,000 computer to have like a big flagship model. and so companies can't like just update everything on the fly for each user because that would be just infeasible basically and so unless we have ways that the models only run on the personal device i don't think we have or can have really good continual learning essentially because yeah and then the other thing is yeah you have to be really careful how you update it you don't want to make the model worse. Because it's such an important, expensive product if you have random, if you just even think about feeding the data back to OpenAI and then OpenAI automatically updates the model and maybe there's a better update and it disrupts everything for everyone.

1:08:34Sebastian Raschka:And so I think it's more of infrastructure, security, that type of issue. But otherwise, I mean, if you look at the reasoning training, what we talked about, the reinforcement learning with verifiable rewards. If you run this based on correct answers and that type of setting and you just keep it running, it is kind of a form of continual learning in a sense where you can technically just keep running this. And it's just like you don't want to, you want to be more selective, basically. And do you think that long context or longer contexts kind of alleviate some of the pain or need to do continual learning?

1:09:15like, you know, in your case of like the personal, personalized models, you know, one approach is to take new information and kind of, you know, continually learn against it. You know, another that I think folks have played around with is to create like personal LoRa adapters for a model. but then a third is to just put that new information into the context and use it at inference time.

1:09:46Sebastian Raschka:I would say yes and no. So I do think long context, they have enabled so much recently, like where before people were building RAC systems, like the retrieval augmented generation systems, and now it's almost, I wouldn't say they are obsolete, They still are very useful if you have a fixed big database or document set, and if you use it repeatedly. But if you're a regular user and you have like, even if you have a thousand page PDF, you can technically often, most of the time, I mean, thousands may be stretching it a bit much, but like a 200 page PDF, you can have it in the context. You don't need to train the LLM in terms of fine tune it on that data.

1:10:30Sebastian Raschka:You don't need to have a rack. you can do a lot of stuff in context and like you said the same is maybe true for information where you could technically just provide all the relevant new information in context but i think it only gets you so far because you also as a user have to know what to provide this information but then if you couple that with tool use for example if you ask about a historical event and let's say the data cutoff is 2025 and you ask about a 2026 historical event, the LLM can still use a web search. It can still use a tool and look it up on the web. So you don't necessarily need to update the LLM for that particular historical event.

1:11:14Sebastian Raschka:But then if the historical event has a lot of ramification and affects a lot of other things around it, that might be missed then. And you only get certain facts from a tool core, but not that whole interaction with other data points. So it's not fully replacing the updating, but it is making it maybe less necessary to do it. Or it's not necessary to do it quite as often, I think, maybe. So, you know, your kind of big picture thoughts on like where the field will be focused over the next year is, you know, again, reasoning, inference time scaling, agents. Any other thoughts or predictions that, you know, come to mind for you?

1:12:07Sebastian Raschka:Yeah, I will be curious to see. I mean, it's a little thing, but you know, like we talked about, there is no big alternative to the transformer architecture. But there is, for example, there are things like text diffusion models. And Google, for example, they had like a, there's like a waiting page, they are planning to launch a text diffusion model, like a small, I mean, not small, but like a alternative where I'm really curious, it's more like something I want to see. maybe that's going to be replacing like the free tier of LLMs. Maybe that's a really interesting thing. I'm mainly interested in that is because there's been a lot of research on text diffusion models.

1:12:47Sebastian Raschka:So it's like a different take instead of generating the text sequentially. It's more like a BERT model, a B-E-R-T model where you have masks and then you gradually denoise or replace the masks by text. I just want to see how it performs at scale because right now most of the models are research models and just things like that. It's nothing, you know, I think people should get excited about in terms of cutting edge performance, but it will be maybe cheaper and faster. And maybe that will be making the, like it's like an everyday maybe improvement for even like the Google summary and Google search, which is also LLM based, but it's not the best.

1:13:26Sebastian Raschka:And like these little quality of life improvements. I think, well, why we're recording this is before the Chinese New Year. And historically, around the Chinese New Year, there have been always a lot of model releases, open-weight model releases. So maybe there's like a little surprise in there. Maybe we'll see DeepSeek version 4 and maybe there is a bigger change. So I'm kind of like interested in following that and seeing that. But yeah, right now, top of my head, I think we covered pretty much everything. Let's maybe switch gears a little bit and update us on what you've been working on personally.

1:14:05You've kind of referenced chapters of the book. Talk a little bit about your current book and where folks can learn more about it.

1:14:14Sebastian Raschka:Yeah, so I think last time I was on your podcast, we talked about my build a large language model from scratch book. So it's basically the whole journey from building the architecture to pre-training a model and then also doing instruction fine tuning. and the goal of that was not to let's say build your personal assistant that does all the things at home for you so because that would cost like uh 50 000 100 000 dollars and be a lot of work and it's it's i mean even though it's simpler nowadays to train your own llm it's not something you can do routinely on a weekend but the goal of that book was to teach people still how that workflow works to understand how LLMs work because that helps you to, let's say, use LLMs better to understand what is the context, what's the limitation of the context, how does, you know, attention work and why is it more expensive if my input gets longer.

1:15:08Sebastian Raschka:And it's just like, if you build the LLM yourself, you kind of like get a real clear understanding compared to just explaining it in a more, I would say, freeform based approach. And so, yeah, a lot of people like that. And it's like a very popular textbook also for teaching now. And I was then really excited to kind of, because it's only one book, it could only cover so much to work on the sequel. So right now I'm working on build a reasoning model from scratch, which is kind of like the sequel. There's no overlap between the books. It's basically, I mean, it can be read as a standalone book, but it's mainly focused on the reasoning techniques we talked about, the reinforcement learning with verifiable rewards, the GRPO algorithm, inference scaling, like all these techniques that once you have a pre-trained LLM, so the book starts by, starts given, so there's a given pre-trained LLM, we use QEN3, the smallest QEN3 model, and then adding inference scaling and the reinforcement learning.

1:16:08Sebastian Raschka:so the first 360 pages are already in the early access and I'm hoping to finish I mean there's only one more chapter left by April I mean the chapter is a lot of work because you have to run all the experiments so I've been running a lot of experiments especially for the GRPO algorithms because there have been so many different papers and improvements and trying them out in practice it's been a lot of fun but it's also a lot of work so yeah so I've been mostly running experiments in the last couple of weeks and months. And yeah, it's quite exciting actually. And so can folks pick up the second book and run with that?

1:16:46Or do you expect folks to have read the entire first book before they start with the second?

1:16:51Sebastian Raschka:I would say either way works. You don't have to read the first book. So the second book, it uses a pre-trained LLM. So you don't have to pre-train your own LLM or you don't need the first LLM to, sorry, the first book to train the LLM for the second book. So it's kind of independent like that. But the second book doesn't explain in detail the pre-training or the architecture. I mean, I have an appendix on explaining the architecture, but it's not quite as detailed as the first book. So I think if people want to understand the whole, let's say the whole life cycle of an LLM from pre-training to post-training, I think it would make sense to read them sequentially.

1:17:30Sebastian Raschka:But you could also start with the second book, learn about inference, scaling and reasoning. And then if you're interested in the pre-training, you can fill in the gaps later on. I think either way works basically. Well, very cool. Sebastian, it's been great catching up with you and we need you to do it more often than every three years. But thanks so much for jumping on and sharing a bit of your perspective on kind of where things are and where things are going. Yeah, thank you so much for the invitation, Sam. I had a great time. I love talking about LLMs and AI. So, well, that was a treat. And thanks for having me on.

1:18:04Thank you.