State of LLMs 2026: RLVR, GRPO, Inference Scaling — Sebastian Raschka

26:35And then that whole thing, it's not really worth it according to their experiments. There are a lot of people who try to make it work. And I think it is promising and we will see it working at some point, I think. So it's just like right now, it's still tricky to make it work, but I am quite sure we'll see it as part of the standard repertoire at some point. And there was, for example, end of the year last year was the DeepSig Math version two paper where they had actually a nice study. They had something like that where they had like a second model that was checking the answers and explanations of that first model.

27:10And it's almost like turtles all the way down. They had yet another model. So they had three models. They had one model generating the answer, one model to grade the answer and the intermediate steps. And they had one model to grade the grader, basically, to say, oh, is the grader actually doing a good job? So there were like three models in a row. It sounds a bit excessive, but based on the performance of that whole setup, it performed really well. So they were cranking up also the self-refinement steps and iterations. And they got gold-level performance on some of the math benchmarks. One could say, okay, maybe, well, cheating, the data was public or whatever.

27:48I don't know, that's a different question. But the fact that this performed better than the model without it tells me whether, let's say, it's really gold level performance is a different question. But it is doing better than just the plain model. So it is actually adding usefulness to the whole process. And I think we will see more of it. It's just expensive because now you have to have more models, more training, more stuff. But that's what I meant also earlier with that's where you make the bigger gains rather than scaling the model size. I think that's one of those things where you will see more progress coming from.

28:19And speaking of math, I think that's one of the key questions going forward for RL, whether you can expand this beyond math encoding to other domains. What's your take on that? Yeah, I think what's attractive about LRVR is that you don't have to have, let's say, humans checking the solutions. You have a verifier that deterministically checks for math is the answer correct. Like giving two fractions, are the fractions the same? You know, that two numbers with decimal points are the same if I round them up. And so it's like very easy to check programmatically, algorithmically. And the same for code.

28:56So you have code and you like code problems and you can. So in that case, you can compile the code if it compiles or you have unit tests. It checks. It works. It's very nice to check. It's no, there's no subjective aspect. It's very objective. You can say, okay, it compiles. It doesn't compile. It's very clear cut. The question you had is, does that, what happens now in general to other fields? Is it like specific to math or code? And I think we will see also expansions of that to other fields. I'm personally not an expert in other fields, so I don't know what that would look like for medicine.

29:30I mean, I have like a computational biology background. I know a little bit about the drug development pipeline and so forth, but it is, I think it's not quite as clear of what the reward looks like. but you can also be more creative so you don't it doesn't have to be strictly verifiable through an algorithm it can be verifiable maybe through an lm you know it could be something like that so for example i can see i don't know for research maybe um training and model to give correct citations and you can maybe check the citations you could have another model that goes through the url and say oh this is indeed the correct paper giving the correct title of the paper or something like that it's a correct url and stuff like that so i think there are lots of these things where we can expand uh rla vr too and uh yeah um train on those things yeah so i think we will see a lot of that the thought that crosses my mind as you describe this is that uh i've heard people say that rl is very difficult to scale is very finicky and hearing what you describe about different techniques put together i'm starting to get a sense for for why is that why it's complicated is that because it's a basically a bunch of different things and models talking to one another that it is hard to scale or is there another reason well i just implemented before we are recording this uh grpo lvr from scratch in a jupiter notebook uh i wrote up a chapter 39 pages so it is not super complicated i would say it's like uh you can fit it into a jupiter notebook it works um and it trains fine i'm trying to say is if you can figure out pre-training the scale at pre-trading, you can figure out this.

31:05Because if you also look at the numbers of how much it cost, just GPU hours, DeepSeek version 3, they had like a$5 million price tag on that given the, I think$2 per GPU, they assumed. Whether that's a correct assumption or not, it's a different question. But if you compare it relative to the cost of R1, I think R1 was about $300 ,000 when they trained it. They had a number in the Nature version of the paper. So it's basically more than 10 times cheaper than pre-training. So, and it's the same infrastructure where you have to make sure, I think the complexity comes from making sure that GPUs don't crash.

31:41If they crash, that you can resume and so forth. And also during pre-training, you might have bad losses where you want to, you know, re-roll the old checkpoint and the same things apply. You have multiple models, but yeah, you are right. There is a bit more, I would say, trial and error in RLVR, where just due to the nature of the updates and so forth, you can, I mean, that's what I observed when I was training my models. You often get, I mean, not that often, but every so-and-so many hundreds or thousands of steps, the model gets bad. You know, like, so the model works totally fine and you train long and long and suddenly the model is really bad.

32:18And so you just go back to the previous checkpoint. But it's not new in terms of a new thing that's happening all the time in pre-training as well. But I think vanilla GRP or the original algorithm, it is pretty flaky. Like where it is, you have to babysit it. Over the course of the year, many people had these tips and tricks where some people were saying, remove the KL divergence term. Like if you just drop it for math, it performs better. Remove the standard deviation, the normalization term. Or if all the rewards look the same, you can skip them to make it faster. you know like there's like there's a lot of tips and tricks like these tricks of the trade that make it more stable and i think like if you apply all of them together it is actually a pretty stable okayish algorithm just like last week uh nvidia also had a paper on g gdpo i think yeah gdpo so they were focused on also algorithmic improvements with respect to multiple rewards so if you have more than one reward it could be something as simple as you have the accuracy reward but you usually have also a format reward because you want the model to put in the final answer and you don't have to but you can put that into these think tags so there's like a token a think token it's more honestly for stylistic purposes i think the advantage is some people develop models that are hybrids which are capable of a normal mode and a thinking reasoning mode so thinking stands for reasoning the appeal here is you don't always want to use reasoning modes because it's expensive it uses a lot of tokens and sometimes you have a simple answer and you don't want to spend 2000 tokens on the simple answer.

33:51And so you can, with these think tokens, steer it a bit. For example, in Quint 3, you can add empty think tokens. So you have an opening token and a closing token. If you add that, the think whatever in between is empty, and then the model will not generate any reasoning chain of thought. And long story short, so during training, you can teach the model to adhere to these different formats. So then you suddenly have a second reward. So one reward is correctness. Is the answer correct? The second reward is, does the model output something that fits my formatting here? And then you have two rewards and how you combine them.

34:27Usually, originally, you just add them up together, but then there were some downsides in the GRPO instability. And so GDBO had some algorithmic improvements to improve the stability. And so there are lots of these little tricks over the to make LRVR more stable, but it is a newer paradigm. So it just takes a few iterations to find the canonical one. It's similar to optimizers with Atom. So Atom is, I mean, right now there's AtomW, there was SGD and all the other RMS prop and how they were called. And they kind of all converged to AtomW by adding more and more tricks. And I think that's the same right now with LRVR, With GRPO, we're adding more tricks to get to something that kind of like is pretty stable across a lot of different scenarios.

35:17It's fascinating to hear you talk about tips and tricks and different techniques that triggers the thought that you had a nice way of putting it in your blog post and taking a step back for a second from the weeds. But you talked about a meta lesson for all the things in 2025 and where progress actually comes from. And do you want to get into that? I think that'd be interesting. Yeah. And so meta lesson would be essentially here that, well, the whole, I think we are talking right now for half an hour about different things. So I think the theme would be, well, there's no one thing that fixes it all.

35:51It's a lot of little tricks and tips, tips and tricks all over the place. And if you add them up, that will give you the progress. But I, yeah, I think there's no magic lever, no magic, I guess, bullet that gives you everything. I think it is kind of tweaking things here and there and making things more robust. I think the tweak was the transformer architecture back then. And now it's essentially, let's, you know, make it even better, I guess, refining it. And a little bit of post-training here, a little bit maybe improving the quality in pre-training, maybe some architecture tweaks, algorithmic tweaks.

36:23It's all a little bit of everything, basically. It's also that I would say you don't have to know all these things because like in practice, it's like a big team at a big company. and everyone has a specialty. Like everyone is either like on the post-training team or the pre-training team. It's not that one person has to know everything and all the tricks because that would be really impossible. And so I think it's also just due to the nature of work because it's so much work. It's a lot of work to train these big models that you kind of like separate these roles and then everyone is working on, can work on everything at the same time, which is also nice.

36:59And then you bring back together all these improvements into the model. And you're confident in the industry's ability to keep coming up with tricks and tips going forward? Yeah, that is a good question. I mean, if I look at DeepSeek, for example, because I mean, I'm always picking here DeepSeek in this podcast because I think they have a really nice trajectory of models. I wish I could also talk more about Gemini and Chachibiti, but they don't really release the techniques. So hard to talk about it. So picking on DeepSeq here, I mean, if you look at version three and then R1, and then they had version 3.2 model with the sparse attention mechanism, and then also this math version two with a self-refinement and everything.

37:41So they do have right now still a track record of improving things. And they are rumored to release a new model in February, the DeepSeq version four. But I think so far, I think we are still on that trajectory where we haven't run out of ideas. So I think the only thing we're running out of is really benchmarks. So the improvement on benchmarks, it's kind of like harder to measure. And I think maybe, well, maybe it's not the one shot problem anymore, where it's not really answering knowledge question. It's not really solving math problems in one iteration of the benchmark. It is maybe more like the agentic cycle, like where you have like a more like an objective that is not, let's say, answer the question, but more like design something, blah blah blah and then it goes off and how long it can or how long it needs or how long it can run until the problem is solved and i think it's maybe more towards that how we measure progress rather than whether we get 90 or 95 percent or 97 on a benchmark yeah yeah you had this nice uh expression uh bench maxing uh you know in some of your posts what what do you mean by that yeah so So benchmarks maxing, so I'm often reading things on X because that's like where a lot of the AI community is.

38:56I think benchmarking is one of the ones that came up in 2025, like a newer generation term. And so loosely what that means is essentially that, well, it's almost like exploiting the benchmarks. Like people train models, do well on the benchmarks, but it doesn't really translate to real world performance. Like a popular example was the Lama 4 model. I mean, based on rumors, I heard they had a separate model just for the benchmarks, the leaderboards. But let's say even that aside, if someone, let's say, trains a model on leaderboard performance, it doesn't mean the model necessarily performs better in real life because leaderboards are susceptible also to style.

39:33And so with leaderboards, the tricky part is because humans compare which model they prefer. And if I have, let's say, a very complicated math problem, I ask an LLM and let's say I don't even know the answer. And then it should help me with my text report or something like that. And there's one LLM that gives me a really nice explanation. Maybe the result is wrong, but the explanation is really nice, easy to follow. I probably like that one. And so I would probably give it a thumbs up because, oh, it's understandable. It's reasonable because I don't know what's correct because I'm not a text expert.

40:04And I think that's one problem with leaderboards. it rewards the style more than the correctness because there is no correctness check. It's like, yeah, as an expert, you have to know whether it's correct or not. Yeah, and so also LLM developers, when you're training the LLM, it kind of gets biased to follow a certain style and the style of people who use those leaderboards. And in that sense, you end up with models that have, let's say, have been benchmarks. They have been getting really good benchmark scores, but they might not do better than previous models. And then it's kind of like a tricky thing.

40:37You know, it's like how to measure progress this way. And you think people do that just out of largely economic incentives? Like the companies need to raise more money and people need to have successful careers and therefore they want to look good. Is that the driver? I mean, I don't want to accuse anyone. I don't know for sure. I mean, I only know what is known on the internet. I read on, let's say, Reddit a few times that Lama 4 was a separate model. So there might have been some company leader decisions that have led to that. I honestly don't know. And maybe incentives, getting good headlines and that stuff.

41:15But, well, I think the open-weight community is a pretty smart community. So I think it's not worth risking something like that. And I think most people don't risk it. It's just implicit. It just happens. It's like if you iterate too many times, it's a classic deep learning problem or machine learning problem. But the nice, the beautiful thing here was actually, it's not a big concern because it happened to all the models. So all of the models performed like 5%, 10 % worse on this new data. It was pretty consistent. So if you were to rank those models, the ranking would still be the same. So in LLM terms, let's say JetGPT and Gemini, let's say they cheat on the benchmarks and the models are 10 % worse.

41:55But if the ranking is still the same, let's say Gemini is still better than GPT, then it's not a problem if both of them do that. So I think we have now that in LLMs where I wouldn't say they are cheating. They are just using the data a lot. And from using the data a lot, well, the data leaks in a sense. So you kind of like you're biased in a sense. But then if they're all biased, then it's again fine because the ranking is still the same. But I think, yeah, the problem still remains. The benchmarks are saturated and it's hard to demonstrate or detect or have any type of notion of progress. It's really right now.

42:33Honestly, personally, I stopped looking at the benchmark numbers. I just use the model and see for a few days and I see if it's better or not. Like I can't say, OK, this is better by so and so many percent. It's more like, oh, I use it and I feel like it's doing a better job. I can't even put it into words. And I think that's the challenge we have right now. how do you put that into words to communicate the progress? And I think that will be, in the upcoming years, the more difficult problem to solve, how to actually evaluate what you're using. I mean, the power of LLMs is that they are so free-form, but that's also the downside for evaluations because evaluations, if you want to be numeric and precise, well, free-form is not so easy to deal with, basically.

43:10Super interesting. So going back to tips and tricks to make sure that we cover the state of LLMs in, I guess, early 2026. So we talked about post-training. How much in the recent progress in the last year or so do you think comes from non-architectural and non-post-training stuff? And I'm thinking in particular inference scaling and then tool use. I do think a lot. I mentioned previously post-training, but honestly, I think one of the biggest drivers this year has been also the inference scaling. So inference scaling essentially means you don't change the weights of the model. You just expend more compute during using the model, during when the consumer or the person, the user uses the model.

43:55And a beautiful example or chart was back in October, 2024, when OpenAI 01 came out, they had like this chart where they had two subgraphs. One was for scaling the training and one was for scaling the inference. And you could see for both, both were going up with a similar like increase and so you can basically invest either more money during training which is a one-time cost and then you have with a fixed size you never have to pay money again later on but then that breaks a bit in a sense because reasoning models they generate more tokens so they are also expensive to use during inference so inference scaling includes that it includes like models that generate more tokens because if you generate twice as many tokens it's twice as expensive because now you have twice as many steps that is one form of inference but it can lead to more accurate answers the other one is parallel sampling you just have the more you ask the model multiple times more like a majority vote and most people if you see the benchmarks they do that it's called best at something like that like best of five or something or best they say i think an at sign five or something running it five times and then selecting the answer.

45:05So you can do majority vote, but it's five times more expensive now because you have to run the model five times. There are also methods where people have a judge model that judges these results. Like if you can't do majority voting, you can have a score and then score the highest answer. It's a bit brittle because, well, that model can also make mistakes, but there are all these types of tricks or self-refinement where you have multiple iterations. Self-refinement is also So like basically like this iterations where you have one LLM write the answer and then you say, okay, take a look at this answer.

45:38And then, oh, I made a mistake here and it self-refines. I mean, reasoning models do that internally as a chain of thought also sometimes, but you can also have an explicit version of that. Or another really cool paper that came out in January was RLMs. And so what they do is they take that prompt. So instead of processing it all at once, they chunk it up into several smaller prompts or the LLM decides it learns how to, or sees how it should chunk it up in code and then runs a prompt on each of those again. So basically making one prompt into smaller prompts and then having multiple requests. And this, I would say, is also a form of inference scaling because I, well, it depends, but I do think it can be more expensive because now you have more LLM calls and each one, if you want to go deep, you can end up spending more tokens.

46:23Not more tokens in one request, but in the sum of all the requests. But there are all these things, I think, that are underappreciated in a sense because they're not, I mean, in this case, it was a popular paper, but often inference scaling is not talked about that much. But I think it is a big driver of making LLMs perform well. And I think why I think that is if you use DeepSeq locally or use the platform, let's say you use a local LLM and use ChatGPT. I think ChatGPT, of course, is a really good model, but I do think the leading open-weight models are not that far behind. But if you use them locally, they don't feel as good.

46:59And I think that's because ChatGPT has a really good interface. Like the platform they have, it's not just running the LLM, it's maybe cleaning up your prompt. Again, this is like, I would say, a hypothesis or like I'm guessing here. But instead of having the LLM learn, of course, it can and it does learn how to deal with misspelled words. but you can also just clean that up, the input in certain cases where that might improve the accuracy. And I think all these little engineering tricks, not just inference scaling, but just cleaning up the prompt, how to manage the context, the history and everything.

47:32I think that all contributes to a lot of progress that is felt by the user. Yeah. And another example would be tool calling too. I think it was also a big one in 2025. I don't remember when ChatGPT introduced tool calling, but it might have been early 2025 or 2024. But GPT-OSS, so they had that open source model in let's say summer 2025. And GPT-OSS has tool calling support. So tool calling means that the LLM can call a web search or it can call code interpreters and so forth. And that is very, very powerful because I think this is one of the ways You can mitigate, not totally mitigate, but let's say reduce hallucinations because then the LLM suddenly doesn't have to remember everything anymore.

48:21You can outsource a lot of things that are hard to tool. So like we humans do, right? We humans, we use calculators. We use the web search. We don't try to memorize everything. And so I think in that sense, that is really a big unlock. The only problem is, well, you have to trust the LLM to run on your computer. which is why right now I think it's more like confined to these proprietary LLMs like Gemini and ChatGPT because, well, it's not your computer it runs on. If it goes somewhere, executes some code and messes it up, well, not your problem. But I think we will see more of that in the upcoming years when the open source tooling kind of like, I would say, gets more robust and people have more trust in running that on their own computer, maybe in a Docker container still, but yeah, something like that.

49:08I mean, right now, a lot of people already run code agents on their computer. They're usually kind of more restricted, but I think people are more and more trusting these to do things they wouldn't have trusted them to do like a year ago and give them permissions. But as LLMs get better, people develop more trust or, I mean, run it in a secure virtual environment. And I think that will be a lot of also leaps we will see, even though the LLM doesn't get bigger or anything like that. And you can actually go to the GPT-OSS release block, and they did have benchmarks to show how the performance on the benchmarks is with the same model with tool called enabled and disabled.

49:46And you can actually see there is, I mean, it's not like two times the performance, it's maybe 1.2 times the performance, but you can see there's definitely a jump in capabilities just by allowing the model to use tools, basically. And that's part of where you see the world go, right? This combination of like open source model and private data. I think you call that the edge in your blog post. What distinguishes different LLMs right now? They are all kind of similarly good, I would say. Like open-weight LLMs, I love LLMs that are similarly good. I mean, personally, I don't use all of them all the time.

50:21But so I usually use one LLM at a time. but like if you use or compare Chachapiti, Gemini, Cloud, Grok, I think they are all pretty much on the same level. And I think that's because they're trying to do everything, like the generalist models for a general person to do a lot of things. I mean, Cloud is a bit more specialized to code now, but the other ones, they are more like general models. And I wouldn't say one is significantly better than the other. They have like small, I mean, differences and so forth. So if you want to really distinguish them and make them better in certain industries, I do think, yeah, the private data is what helps.

50:57Like all the treasure troves of data that a finance company has over the years, over 100 years or 50 years collected, or medical data, like medical records from patients. I think JGPD had like a contract now to process them, to make it secure and private. But I honestly think these companies, they don't want to just give away that data first. they can't. I mean, it makes also sense. You just really, as a patient or a customer, I would feel really bad if someone gives my health data to some other company that I didn't agree with, or agree with this sharing. But then also the companies, they don't want to just give away all that data because once they do, well, then maybe they become then really kind of obsolete in that sense where it's all, all the treasure is basically all their, what makes them different from other companies is now taken, basically.

51:49And so over the last month, I mean, people reached out to me also. I know for a fact that big companies are training now LLMs in-house. Really, like big companies who have the financial means to train a Chachapi-like model are hiring people who train LLMs. And I think that is also what we will see, that instead of going to this big LLM provider and giving them the data, people will try to make their own LLMs for their own company and private data. It's fascinating, right? So back to the future, because initially people thought that they were going to train the models and then they kind of like gave up.

52:27But what you're saying is that you're seeing people going back, maybe with a better state of open source LLMs that they can build on as a building block. That's what you're saying, right? Yes and no, I think you're right. I mean, no, you bring up a good point. I think open-weight and open-source models are very popular like a few years ago and are still very popular and I love working with them. But I think, well, there's still a gap between a chat GPT model and an open-source, open-weight model. Maybe now with DeepSeq version 3, not so much, but that's almost a different community. Like the Tinkerer community, like me, like small system.

53:03Well, DeepSeq version 3 would be way too expensive to run for me every day. I would have to spend thousands of dollars on just hosting costs every day, every week. And so I toy around with smaller special purpose models. But what I meant is, so first, yeah, the open source community in that sense will have maybe a comeback at these companies. But I even mean a step further that they actually develop models from scratch, like really big models. And what's different from, I would say, the regular open source here is that it is really large scale. It's like a chatGVD scale data center, large LLM.

53:42It's not something you run on your computer, basically. It's really like a big data center style LLM. So I know that there are, I mean, I can't say any names, but I know people are interested in that. Like they are exploring that. Whether it will work out or not, I don't know. But I think, I mean, right now, if you are in college, you are learning about LLMs. That's the thing. That's the big thing. So you start with open source, you start training small LLMs, and you work your way up. And then at some point, well, you probably want to get hired either by Gemini, JetGPT, or so forth, and do the big model development there.

54:18But not everyone can have 100 ,000 people doing that. So there will be people distributed across different companies who will do something similar. And also, on the other hand, people who are, I think, at OpenAI and Gemini at some point, I mean, big finance companies have a lot of deep pockets. They will make it attractive to do something similar at their company, too. So I think we will see right now it's very concentrated at these companies, but we will see, I think, the knowledge being a bit more spread out where other companies will develop models. We will probably not hear about it. You know, it's like on the news or anything, maybe the news, of course, but not, there won't be papers.

54:57There won't be big announcement because it won't be this general audience model. So it's more something they will do internally. Right. And especially if you're a big hedge fund or defense company, like I assume that's the kind of companies we're talking about. Yeah. Those are of a very secretive DNA. Okay. Fascinating. Fascinating. What else do you see happening this coming year? I was interested to read your thoughts on continual learning, which was sort of the talk of the town, the NeurIPS, but you viewed it or you view it as a 2027 thing, not necessarily a 2026 thing. Yeah, the continual learning is an interesting one.

55:36I think it was discussed there very heavily, but also in general, if you went to social media, AI related topics, continual learning was there all the time everywhere. And to be honest with you, I don't know why exactly it was such a hot topic in 25, because I don't think there was a big breakthrough. I mean, maybe it's the hope for breakthrough or the like, hey, nothing has changed. So maybe we have to focus more on that to force some change. But yeah, so continuing learning, I think is an interesting topic because it sounds attractive if you have an LLM that self-improves or like an agent that does something, fails and learns.

56:15I don't think anything like that is feasible this year. I mean, so right now, I mean, well, you could technically do continual learning if you wanted to with the data you have, like when you think even of RLVR. So you could technically, it's just updating the model, but the problem is still the catastrophic forgetting. Also, you don't want to train the model on garbage data. So people kind of do, I think, I mean, I would call it continual learning, but in a more controlled setting where, well, instead of just updating the model, letting the model update itself, they collect failure cases and data and then construct the data set and then do it in a more controlled manner.

56:51But you can see based on the model releases, it happens more frequently than it used to. Because I mean, back then it was GPT-1, GPT-2, GPT-3. And now it's GPT-4, 4.1, 4.2 or 5.5, 5.1, 5.2. And all the models, they are iterating now or even the same with DeepSeek. So it's like this same architecture. You iterate multiple times, but still in a more controlled way. And I think it makes sense because it's such an expensive thing to do. I don't even know how you would do continual learning. If you have this model, it is hosted in a data center. You can't just update it. It's a very expensive model.

57:27You can't just update it on good luck. So it's like the risky thing to do. And you can't do it even as a single person. You have to be really careful monitoring a lot of things. I don't know how that would work. Because right now we are still in this era where everyone uses the same model. People don't have custom models. When I go to Chachapiti, I have the same model as you do. And yeah, the prompt is a bit different, like the memory and everything, but it's all in the prompt. It's the same model weights. And as long as we have that, I don't think we will see anything like continual learning.

58:01I mean, there are companies, I guess, like, I mean, Tinker API. It is something where it is democratizing a bit like the training where through an API, you can now train your model more cheaply or instead of having the hardware, it's on their data centers. people have their own copy but you know it's i think very very far from continual learning it's just making available what other companies have in terms of training on a large cloud instance without you having set it up but i don't see anything 2026 that really makes continual learning like the big breakthrough in efficiency or i don't know and so 27 is even ambitious so i don't know So maybe we'll see something there in 2027.

58:46Given that it is such a big topic, an important topic, there's a lot of smart people thinking about this and working on it. So we'll probably see something or at least, I don't know, some ideas that are fresh or things that prototypes that work or interesting, but really hard to say anything concrete without having seen anything. So it's a prediction that I just put out there. maybe we'll see more continual learning stuff in 2027, but with a grain of salt. Yeah, maybe it becomes a self-fulfilling prophecy, right? If you have enough smart people that decide it's the thing, then maybe it does happen.

59:25All right, so maybe to close the conversation, let's talk about your work and how you do your work. So you published a book this past year in 2025 on how to build LLM from scratch. I believe that you are writing the sequel currently, How to Build Reasoning Models from Scratch. Is that the title? And you produce an incredible amount of work. So people should find you on your website, on your Substack letter. How do you absorb all of this knowledge? to which extent are lms part of your workflow just curious how you work these days good question so i must say like uh well i don't have like a magic well approach or anything i think i i think the thing i have maybe is i get very excited about things and then when i'm excited about something it goes very easy and very fast i don't know it's like well if you notice maybe i write only about certain topics i don't cover image models at the moment for example because I am just very excited about LLMs.

1:00:36And then I don't know, it's just, I can't help it. I get very excited, read all the things about it, write about it. And that's mainly, I almost go by intuition basically. But I'm kind of lucky in that sense that like with my blog, what I find interesting, other people also right now find interesting. So I think there's like a lucky coincidence that I honestly write only about things I find interesting. So I'm not kind of trying to force myself, oh, I have to cover X, Y, Z because, well, it's something that should be covered. It's more like, oh, how does this, let's say, recursive language model work?

1:01:09Let's just read the paper and then I write about it, you know, like more like, yeah, getting excited about things. And yeah, the book writing is, well, it's also a bit different because my blog is more research paper focused where I put all the, like when I get excited about something, I read about it and put it in there. For the book, I'm similarly excited, but that's more like a coding book where it's like the fundamentals. It's like, because I think that's like, to be honest, the best way to understand something is to see, to see it actually working. It's not any like hand wavy figures. I mean, there are a lot of figures like right now for chapter six, just I finished it the other day, 21 figures and they take the most work.

1:01:44Maybe one day LMS can help me with that. But figures help to explain the code and everything. But code basically doesn't lie. If it either works or it doesn't work, you know, and I think that's a very useful way to learn also. and it's just also a lot of fun. For me, it's like when I write code and I see it working, it's very satisfying and you have something that actually works. So I should say I'm not building in that book any production level systems. It's called build an LLM from scratch or I mean large language model from scratch, but it's not like an LLM that you would use in production.

1:02:17If you make a few tweaks, you could use it in production, but the goal is code readability and teaching LLMs basically too. because I think to see actually how do I format my training data? How does it get processed? What is the loss function? What gets updated? I think this explains so much more than if I say, oh, it does next token prediction and then hand wavy, hand wavy here and there. You can actually literally see how it does that and what feeds in. And the same with LVR. We had like a hopefully not too bad explanation in this podcast at the beginning of a GRPO. but if you actually see the diagram, I have the numbers in there, but the numbers could be wrong though if you have a figure and you draw arrows and everything.

1:03:00But then if you implement that in code and you get exactly the same results and the model trains and you get 50 % accuracy, it's like a nice thing where you can, oh, it's actually working. It's not just made up numbers. It's actually, it actually works, you know? And also that's how I learn about LLM architecture. So I have this block, the big LLM architecture comparison with now I think 13 ,000 words because I keep extending it. I read the paper. I look at the architecture of an LLM. Do I really understand it? So I draw the architecture, but then do I really understand it? And often, unless it's like a one-train parameter model, I often code the model.

1:03:36And so I have still my GPT-2 architecture and they are relatively similar. And so if there's a new architecture, I take the most similar one and make a few changes to it. But then the beautiful thing here is, so someone already implemented that in a hugging phase, the Transformers architecture. So I have a reference model I can run. I run my model and I can see, do I get the exact same logits, the same numbers if I have the same prompt? And so with that, you can actually self-check yourself. Did I implement everything correctly? Are the results correct? And I think that's just a lot of fun. It's a lot of work, but it's a lot of fun and it doesn't lie.

1:04:09It gives you the correct answer. Perfect on some prompts, someone else extended that and I found a bug and now I have a better understanding how they implemented the yarn scaling. That is something you would never understand by just reading the paper. You have to really, I don't know, look at the code and toy around with that. And so, yeah, so that's basically how I try to work. I try to combine, you know, like reading and coding. And LLMs I also use, but I try to use it. So I would say for blog writing, on book writing, not so much because honestly, for fun, I tried it out. It's just, it generates okay text, but it's, I don't know, it does not, I can ask it to generate text like me, But it's almost like then I don't like it and I end up editing it.

1:04:51And then it's almost faster for me to just write it out the way I want it. And you had interesting thoughts on LLM burnout, how using LLMs tends to deplete energy. I would say first, also the thing is it's not super satisfying if you just ask the LLM to do it. It's, you know, like cheating at homework. I honestly, I understand there are jobs and people where it just matters how much you get done and how fast you get something done. And then it makes sense to use an LLM to do the job for you. But I think that different types of people who enjoy different things, for example, I enjoy doing the work more than managing.

1:05:30When I was a professor, I did research, but I had to also, you know, supervise other students. And I liked working with other students, but I noticed I actually liked doing the research myself more than telling other people how to do research or like managing. And so I think if you use only LLMs to just generate everything and I wouldn't say useless, but I would feel maybe empty. Like, okay, you use that pride. I think the pride, oh, I did something that worked and it's cool and you're proud of this. And so what I try to do is generally when I use LLMs, I try to make my work better. Not to like, not necessarily to make more or make it faster.

1:06:10I mean, to some extent I do, but then I try to like, how can I make what I do kind of better? So what I use LLMs for is more like, hey, I use actually the GPT-5 Pro when I have written an article and put it in there. Hey, can you find any mistakes or typos? Often I have mislabeled figures. I go like figure 11, 12, 15, 16, and things like that. I can check myself, but it's just faster for an LLM to find all these things, like how to make things better. Are there any things that are unclear? I mean, I'm not a native speaker. So sometimes I have problems with a sentence. I'm tired. I just can't get it right.

1:06:44And then it suggests me, oh, yeah, that is maybe not a bad way to say it. And I would then take that sentence, for example, like things like that, where I'm trying to make, let's say, the work better without fully replacing myself. I mean, maybe that's like short-sighted because LMS will eventually be able to do everything. But I kind of like enjoy the work too much to just give everything away. if that makes sense. And I have the luxury that this still works for me. So I know there are some businesses where, yeah, it is really important to execute faster. And, you know, so, yeah. Thank you very much for your work.

1:07:19Very popular for the quality of your writing, all your tweets. You have like a big X following and your name keeps coming back in conversations about where people go to learn. Thank you for doing this part. Really appreciate your time. That was super fascinating and really educational, really insightful. So I really appreciate it. Thank you so much, Sebastian. Thank you for inviting me. I had a lot of fun. I think it was maybe one and a half hours just talking about LLMs and AI. I mean, this is, well, that's the dream, right? Thanks for having me. Thank you. Hi, it's Matt Turk again. Thanks for listening to this episode of the Matt Podcast.

1:07:57If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.