What’s Next for AI? OpenAI’s Łukasz Kaiser (Transformer Co-Author)

27:50And I was like, no, no, we're going to do all tasks in one model. And people were like, no, no. So there was a lot of pushback against the idea? Not against the idea. Google was also an amazing place at that time that they would very happily let you work on whatever you wanted. But I don't think there was widespread belief in doing multiple tasks with the same model, not to mention, you know, this idea that you, I still find this idea that you take basically the same model as Transformer. Like now there is a bunch of changes to it, but you could in principle take the same architecture as the decoder from the paper, train it on all of the Internet, and it will basically start chatting with you.

28:30That would have back then definitely sounded as a worthy dream we maybe had as a dream, but not a reality that you expect five years later. It's very lucky that it actually works so well, right? Talk about the transition from Google to OpenAI and perhaps how those two cultures are different. So Ilya Suzkever was my manager at Brain. Then he went on to found OpenAI. He asked me a number of times whether I would like to join in the years. I found it a little bit too edgy at the time. Then Transformers came, so we had a lot of work with that. And then COVID came. And COVID was a tough time for the whole world, right?

29:13But Google totally closed. Google was reopening extremely slowly. So one part of me was I find it very hard to do remote work. I much prefer to work with people directly. That was one reason. But the other was also Google Brain, when I joined it, was a few dozen people, maybe 40, something like that. When I left, it was 4 ,000, 3 ,000 people spread across multiple offices. It's very different to work in a small group and to work in a huge company. So with all this, Ilya was like, you know, OpenAI, though, is in a much stabler state. We're doing language models. You know something about this that may look like a good match.

29:52And I was like, OK, let me try. I've never worked in any company other than Google before, other than the university. So it was quite a change to the small startup group. But I like working in smaller groups. It has its pleasure. It has a little bit of a different intensity sometimes. In general, I found it very nice. On the other hand, Google has merged, made the Gemini, and I hear it's also a very nice place. I think in general, the tech labs are more similar to each other than people think. There are some differences, but I think if I look at it from the world, you know, from the university in France, the difference between this university and any of the tech labs is much larger than between one lab or the other.

30:43How are the research teams organized within OpenAI? They're organized. They're not very organized. I mean, we do organize them. Some people have managers and sometimes talk to them. No, but mostly people find like projects, there are things to do, right? Like improve your multimodal models, improve your reasoning, improve your pre-training, improve whatever this part of the infrastructure. People work on it. You know, as we go through these parts, right, there is infrastructure, pre-training, reasoning. I think the parts are the same for most of the labs. So there will be teams doing these things.

31:27And then sometimes people change teams. Sometimes new things emerge. There's always some smaller teams doing like more adventurous stuff like diffusion models at times. Then, you know, some of the more adventurous stuff like video models gets big and then maybe they need to grow. Do people compete for GPU access? I don't think it's so much people that compete. I think it's more projects that compete for GPU access. There's definitely some of that. On the other hand, like on the big picture of GPU access, a lot of this is just determined by how the technology works, right? Currently, pre-training just uses the most GPUs of all the parts.

32:13So it needs the most GPUs, right? RL is growing in the use. Now, video models, of course, use a lot of GPUs too. So you need to split them like this. Then, of course, people will be, oh, but my thing would be so much better if I had more GPUs. And I've certainly said that a number of times, too. So then you kind of push it, you know, I really need more. And then some people may say, well, but, you know, there's only so much. There is never enough GPUs for everyone. So there is some part of the competition, but the big part is just decided by the fact how the technology works currently. Great. What is next for pre-training?

32:53We talked about data. We talked about engineering, a big GPU compute aspect to this. What happens to pre-training in the next year or two? Pre-training, as I said, I think it has reached this upper level of the S-curve in terms of science, but it can scale smoothly. Meaning if you put more compute, you will get better losses if you do things right, which is extremely hard. And that's valuable. You don't get the same payoff as pushing HRL, but it generally just makes the model more capable. And that's certainly something you want to do. I think what people underestimate a little bit in the big narrative is, you know, OpenAI three, four years ago, I joined even before that, was a small research lab with a product called API.

33:50But, you know, it was not such a big, there was no GPU constraint on the product side, for example. All GPUs were just used for training. So it was very easy as a decision for the people to say, you know, we're going to train GPT-4. This will be the smartest and largest model ever. And what do we care about small models? I mean, we care of them as to make like to debug the training of the big model, but that's it. So GPT-4 was the smartest model and it was great, right? But then it turned out, oh, there is this chat and now we have a billion users. And, you know, people want to chat with it a lot every day and you need GPUs.

34:30So you train the next like huge model and it turns out you cannot satisfy this. Like people will not want to pay you enough to chat with the bigger model. So you just economically need the smaller model. And this happened, of course, to all the labs, because like the moment the economy arrived and it became a product, you had to start thinking about price much more carefully. than before. So I think this caused the fact that instead of just training the largest and largest thing you can for the money you have, we said, well, no, we're going to train the same thing, but same quality, but smaller, cheaper.

35:10The pressure to give the same quality for less money is very large. In some sense, as a researcher, it almost makes me a little sad. I have a big laugh for these huge models that people say human brain has 100 trillion synapses or orders of magnitude, of course, are not that exactly calculated, but our models don't have 100 trillion parameters yet. So maybe we should reach it. I would certainly love it, but then you need to pay for it. So I think this may be why people kind of think that pre-training has paused because a lot of effort went into training smaller models. Now on the side, people kind of rediscovered how amazing distillation is.

35:51Distillation means you can train a big model and then put the same knowledge from the big model as a teacher to the little model. People knew about distillation. It's a paper for a long time ago. But somehow, at least for OpenAI, I think maybe it was more in Google's DNA when Aureole's there. But people kind of rediscovered how important that is for the economics. But now it also means that, oh, training this huge model is actually good because you distill all the little ones from it. So now maybe there is a bit more of a return to like, it's also a matter, you know, once you realize you have the billion users and you need the GPUs, you need to invest into them.

36:31And of course, everyone sees this. There's a huge investment, but the GPUs are not online yet. So when they come back online, and I think this may play into this, what people call resurgence of pre-training. We both understand that you can distill this amazing big model and there is now enough GPUs to actually train it. So it's resurging. But all of this fundamentally happens on the same scaling curve, right? It's not like we didn't know that you could do this. It's more like the different requirements of different months have changed, sometimes changed the priorities. But I think it's good to step back from it and think of the big picture, which is that pre-training has always worked.

37:15And the beautiful thing is it even stacks with RL. So if you run this thinking RL process on top of a better model, it works even better than if you run it on top of a smaller model. One question that I find fascinating as I hear you speak and the evolution of the modern AI systems has been this combination of LLM plus RL plus a lot of things going on. And it used to be at some point, and maybe that was back in the deep learning days, that people would routinely say that they understood how AI worked at a micro level, like the matrix multiplication aspect, but didn't fully understand once you had everything together, what really, really happened at the end of the day in the model.

38:03And I know there's been tons of work done on interpretability over the last couple of years in particular, but particularly for those very complex systems, is it increasingly clear what the models do or is there some element of black box that persists? I would say both. There is a huge progress in understanding models. Fundamentally, I mean, think of the model that is chat. It talks to a billion people about all kinds of topics. It gets this knowledge from reading all of the internet. Obviously, you cannot identify, like, I cannot understand what's going on in there. I don't know the whole internet.

38:39What we can identify is there was a beautiful paper just, I think, last week from OpenAI about if you tell the model that lots of its weights should be zeros, it should be very sparse, then you can really trace when it's thinking about one particular thing, then you can trace what it's actually doing. So if you say limit ourselves to this and to really study this inside a model, then you can get a lot of understanding. And there's circuits in the models. Antropic had great papers on that. So the understanding of what the models are doing on a higher level has progressed a lot, but then it's still an understanding of what smaller models do, not the biggest ones.

39:19But it's not so much that these patterns don't apply to bigger models. They do. It's just the bigger models just do so many things at the same time that there is some limit to what you can understand. But I think this limit is a bit more fundamental than people think. It's like every very complex system. You can only understand so many things and then you don't, right? Thank you for all this. I'd love now to talk about 5.1 and do a little bit of a deep dive on all the latest stuff that you guys have released in the last couple of weeks, which has been very impressive, in particular as a user. I think that the 5.1 moniker doesn't do justice to the evolution between 5.1 and 5.

40:03It feels like a much larger improvement than the number would indicate from, again, my perspective as a user. Walk us maybe through the evolution of from GPT-4 to 5 to 5.1. What has actually changed? That's a very tough question. I think less than you think. I think it's, no, I mean, from GPT-4 to 5, I think the biggest thing that changed is reasoning, meaning RL and synthetic data. As I told you, the pre-training part in that timeframe was mostly about making things cheaper, not making things better. So, of course, the price has changed dramatically too, right? A thousand times, I think, are some of these orders of magnitude.

40:48The main improvements from four to five is adding reasoning with reinforcement learning and this allowed to generate synthetic data, which also improves the model. So that's the big picture. In addition to that, ChatGPT is now a product used by a lot of people. So the post-training team has learned a tremendous number of lessons and it's added, you know, things clearly experimented, wanted the model to be very nice to you. and turned out to be too nice. Then now when a lot of people use it, you need to be really careful about safety, right? There may be people that are in distress using the model.

41:26The model needs to do something reasonable in these cases. It was not trained for it before. Now it is, and it makes the model much better. But, you know, in the same time, you don't want to refuse to answer any question that has any sign of anything. So as you work on these things, you make the model much better in use, not just for the people in distress, but for everyone who wants questions answered, but the answers to be reasonable. And, you know, there was these things called hallucinations. It's still with us to some extent, but dramatically less than two years ago. Some part of that is because reinforcement learning can now use tools and gather data.

42:06And it also encourages the model to, like, know, you know, verify what it's doing. so that's an emergent thing from this reinforcement learning of reasoning but also you just add data because you realize sometimes the model should say I don't know so you add this to the post-training data you say like well we really need to give it a thought how the model should answer people in various situations. The price to find point one is mostly this kind like it's mostly a post-training improvement. Yes, to double click on this, because it is super interesting. So indeed, as part of 5.1, there's the ability to choose different kind of styles from nerdy to professional.

42:52And that's, I guess, in reaction to the fact that some people were missing the sycophantic aspect of earlier models when chat GPT, when GPT-5 came out. And so adding more tones, that's all post-training stuff. So you tell the model, those are examples of like how you should be responding, which is more like a sort of like super fast tuning kind of a paradigm. Or is that RL like right or wrong with rewards? How does that work? I don't work on post-training and it certainly has a lot of quirks. But I think the main part is indeed RL, where you say, OK, is this response cynical? Is this response like that?

43:35and you say, okay, if you were told to be cynical, this is how you should respond. If you were told to be funny, try this one. So I do think the RL is a big part of it. In between models or different versions of the models, are the releases aligned with pre-training efforts or sometimes you have like one big pre-training effort and like several models that come out based on that? There used to be a time, not that long ago, we have a year distant past where where where the models were did have an alignment with technical stuff right so they would align whether with either with rl runs or pre-training runs that's why you had a beautiful model called 4.0 which was aligned with a pre-training run which was obviously worse than the o3 aligned with an rl run that was the follow-up to a one naturally because you couldn't use the name O2.

44:31But it was slightly better than the O4 Mini because that one was Mini. And, you know, we had this beautiful model picker and people kind of thought this was not the best naming for some whatever reason. So, no, I mean, it was fairly obvious that this was very confusing, right? So now the naming is by capability, right? GPT-5 is a capable model. 5.1 is a more capable model. Mini is the smaller model that's slightly less capable, but faster and cheaper. And the thinking models are the ones that do more research. In that sense, the naming is detached from any technical, in particular, 5.1, maybe just a pre-training, sorry, post-training thing, but maybe 5.2 is the newly pre-trained model or maybe not, but the naming is detached from the technology.

45:24which also gives some, you know, as OpenAI has grown, there is a number of projects, right? There is RL and pre-training and maybe, you know, something just to make slides better or whatnot. And with distillation, you have the ability to put a number of projects into one model. It's kind of nice that you don't need to wait on all of them to complete at the same time. You can try to periodically put this together, actually make sure that as a product, It's nice to the users and good and do this separately from, you know, waiting on the new full pre-training run that takes months and so on. So I feel like even though a little tear in my eye goes for the times where it was that pre-trained model number that was the number.

46:07As it's a product serving a billion users, it's maybe inevitable that you should name it by what the user should expect from it rather than. In 5.1, you have additional granularity in terms of telling the model how long it should think. By default, how does the model decide how long it should think? So the model sees the task. It will decide on its own a little bit how long it should think. But you can give it an additional, it's trained with an additional information that can think even harder and then it will think longer. So you have now the ability to steer that. But I still think it is important to realize.

46:48So this is the fundamental change that came with reasoning models, that using more tokens to think increases your capability. And it increases it, given the computation, way faster than pre-training. So if you give GPT-5 the ability to think for long, it can solve tasks that are, You know, we had these gold medal at Mathematical Olympiad and Computer Science Olympiad. So they're amazing abilities. At the same time, the fundamental training method of reasoning is very limited to science data. So it's not as broad as the pre-training, which I think like pre-training models felt kind of almost uniformly good or bad at things.

47:38I mean, this was still not uniform because it's not like teaching humans, right? But the reasoning models are even more, people call it jagged, right? They have amazing abilities somewhere and then close by, not so much. And that can be very confusing. It's something I always love that. It's weird because you can say the model is amazing at Mathematical Olympiad. But at the same time, I have a math book for, I have a first grader daughter in the first grade. She's five years old. I took one exercise from this math book and none of the frontier models is able to solve it. And you would be able to solve it in 10 seconds.

48:17So that's something to keep in mind. Models are both amazing and they're tasks that they cannot do very well. I can show you this as an example. I think it's quite interesting to keep in mind. Let me start with Gemini 3, just to blame the competitors. Yes, please. So it has, you see, two groups of dots on both sides. And the question is, is the number of dots even or odd? And if you look at it, you see, oh, they're like two identical things. So that would be even. That's what the five-year-old is supposed to learn. But there is one dot that's shared. So now that must be odd. For this simple one, which has like, you know, I don't know, 20 dots or so, Gemini 3 actually does it, right?

48:58It finds out that it's an even number of dots and it says that and that's great. And then you have another puzzle, which is very similar, except now there are two mountains of dots. And there's also one dot shared at the bottom now. And right in context, right after that, you ask, OK, how about this one? And then it does some thinking and it just totally misses that there is a shared dot and it says the number is even. And it's like, in context where you've seen this first example, how would you ever miss that? You know, and here is the same, the exact same prompt for GPT 5.1 thinking. And it also solves the first.

49:37It sees the dot. It says it's odd. And then it sees the mountains. And somehow it doesn't see the dot. And it says it's even. The nice thing is, if you let it think longer, or if you just let it think again, It will see it. So if you use GPT-5 Pro, it takes 15 minutes. So, you know, this is the human five-year-old takes 15 seconds. The GPT-5 Pro will run Python code to extract these dots from an image and then it will count them in a loop. So that's not quite... And why is that? What trips up the model? I think this is mostly multimodal part. The models are just, they're starting. Like you see the first example they managed.

50:26So they've clearly made some progress, but they have not yet learned to do good reasoning in multimodal domains. And they have not yet learned to use one reasoning in context to do the next reasoning. What is written in context is, you know, learning in context happens, but learning from reasoning in context is still not very strong. All of these, though, are things that are very well known. And the models are just not trained enough to do this. It's just something we know we need to add into training. So I think these are things that will generally improve. I do think there is a deeper question whether, so, you know, like multimodal will improve.

51:09This will improve. Like we keep finding these examples. So as the frontier will move, it will certainly move forward. Some things will smoothen, but the question is, will it still be just other things that you don't need to, you know, teach the human like every, you know, okay, now you know how to use a spoon and a fork. But now if the fork has four instead of three ends, then you need to learn a new. That would be a failure of machine learning. You know, I am fascinated by generalization. I think that's the most important topic. I always thought this was the key topic in machine learning in general and in understanding intelligence.

51:47Pre-training is a little different, right? Because it increases the data together with your increase in model size. So it doesn't necessarily increase generalization. It just uses more knowledge. I do believe that reasoning actually increases generalization. But now we train it on such narrow domains that it may still be to see. But I think the big question in all of AI is, is reasoning enough to increase generalization or do you need like more general methods? I think the first step is to make reasoning more general, as we talked before. That's my passion. That's also what I work on. There is still something there, right?

52:20We push the models. They learn. They learn things that are around what we teach them. They still have limitations because they don't live in the physical world, because they're not very good at multimodal, because reasoning is very young and there's a lot of bugs in how we do it yet. But once we fix that, there will be this big question. Is that enough or is there like something other big to make models generalize better? So we don't need to, you know, teach it every particular thing in the training data that it just learns and generalizes. I think that's the most fascinating question. But I also think a good way to approach a question like that is to first solve everything that leads up to it.

52:59You know, you cannot know whether there is a wall or not until you come close to it, because otherwise, you know, we AI is moving very fast. Someone said it's like driving fast in a fog. You never know how far or close you are. So we're moving. We are learning a lot. And does that mean, so that central question of basically learning with very little data, the way a child would, and the fact that a child is able to do things that even the most powerful model cannot do. So this, as you said, to unpack this, making progress on reasoning and showing how far we can get into generalization with reasoning.

53:38And then the separate question is, as you said, whether we need an entire different architecture. And that's where we get into, for example, Yann LeCamp's work. Do you see promising, fundamental architectural changes outside of transformers that have caught your attention and feel like they could be a serious path to explore in the future? I think there is a lot of beautiful work that people are trying out. You know, the ARC challenges inspired one set of people. There are models now that are very small and solve them very well. But with methods that I'm not sure are actually general, you need to see.

54:18Jan Lecun has been pushing for other methods. So I feel like his approach is more towards the multimodal part. But maybe if you solve multimodal right, maybe if you do JEPA, it also helps your other understanding. There is a lot of people pushing fundamental science. It's maybe not so much in the news as the things that push. But whatever you do, it will probably run on some GPU. If you get a trillion dollar of new GPUs, the old GPUs will be much easier to get also. So I think this growth in LLMAI on the more traditional side is also helping people to have an easier time to run more experimental research projects on various things.

55:02So I think there is a lot of exploration, a lot of ideas. It's still a little hard to implement them at a higher scale. The engineering part is the biggest bottleneck. I mean, GPUs are a bottleneck, too, when you scale really up. But implementing something that's larger than one machine, it's an experimental research project, so you don't have a team to do that. I think that's still harder than it should be. But, you know, codecs may get their code. This is the thing where AI researchers have great hope to help themselves and also other researchers. is that if you could just say, hey, Codex, this is the idea, and it's fairly clear what I'm saying, please just implement it so it runs fast on this eight machine setup or 100 machine setup.

55:50That would be amazing. It's not quite capable of doing that yet, but it's capable of doing this more and more. I think that's what OpenAI says. They say we'd like an AI intern by the end of next year. That's how I understand this. Can someone help us? Is part of Codex's, the path for Codex to be able to do some of this, does that revolve around how long it can run? Context behind the question being that, again, like two days ago as we recorded this, you guys released GPT 5.1. Codex Max, described as a frontier-agentic coding model, trained on real-world software engineering tasks, designed for long-running workflows and using compaction to operate across multiple context windows in millions of tokens.

56:39So I'd be interested in unpacking some of this. What does that mean to run for a very long time? Is that an engineering problem or a model problem? And then maybe a word on compaction. So it is both an engineering and model problem. You know, you want to do some engineering tasks, like write a, you have some machine learning idea. You want codecs to implement it for you, test it on some simple thing, find the bugs. So it needs to run this thing. This is not something you would do in an hour, right? It's something you'd spend a week on. So the model needs to spend a considerable amount of time because it needs to run things, wait for the results, then fix them.

57:18The model is not like it's going to come up with the correct code out from thin air, right? It's just like us. It needs to go through the process. And oftentimes in the process, since it was not trained on anything very long in its training, or maybe very few, but certainly nothing that went on for a week, it can get lost. It can start doing loops or doing something weird. That's, of course, not something you want. So we try to train in a way that makes it not happen. But it does. So, you know, how can you make the model actually run a process that requires this larger feedback loop without tripping up?

58:00And the other thing is transformers have this thing called context. So they remember all the things that they have seen in the current run. And that can just exceed the memory available for your run. And the attention matrices are n by n, where n is this length. so they can get huge. So instead of keeping everything, you say, well, I'm going to just ask the model on the side to summarize the most important thing from the past, put it in context and you and forget some part of it, right? So it's a very basic form of forgetting the compaction, right? And that allows you to run for much longer if you do this repeatedly.

58:37But again, you need to train the model to do that. And when you do it, it works to some extent. I don't think it works well enough to replace an AI researcher yet. It made a fair bit of progress. I think another part of progress that's a little understated on the research side, but is very important, is allowing the model to connect to all of these things. So models now use tools like Web Search and Python routinely, but to run on a GPU, to have access to a cluster. It's hard to train models with that because then you need to dedicate for the model to use. And that has security problems. And this thing, like how do models connect with the external world?

59:15It's a fundamentally very hard problem because, you know, when you connect in an unlimited way, you can break things in the real world. We don't like models to break things for us. So that's the part where people work a lot. It overlaps with security, right? You need to have very good security to allow models to go on and train on the things they need to train. One theme that people like me, VCs and founders and startups think about a lot as we see all the progress at OpenAI is as the models keep getting more general with more authentic capabilities, the ability to run for a very long time, you know, going to areas like science and math.

59:56And, you know, recently it was reported that there were some investment bankers hired to help improve the model's capability to do grunt investment banking work. All of that taken into account, is there a world where basically models or maybe just one model does everything? And I don't know if that's AGI, let's not necessarily go into that debate. But what's left for people that build products that sit on top of models? I just showed you a five-year-old exercise that the model doesn't do. I think we need to keep that in mind. So you're saying there's hope. There is hope the next model will do it.

1:00:36That hope, okay. Well, for me, yes. I still think we have some way on the models. Progress has been rapid. So there is good hope there will be less and less of this. But on the other hand, for now, you don't need to do a deep search to find things where you'd really want a human to do that task because the model's not super good. On the other hand, Transformer paper started with translation. I recently went to a translation industry conference. The translation industry has grown considerably since then. It has not shrunk. There's more translations to be done. Translators are paid more. questions, why would you even want a translator if the model's so good in most of the cases?

1:01:19The answer is sometimes, imagine you do a listing for a newspaper, but in a language you don't know. And GPT-5 will almost certainly translate it correctly for you. If it's into Spanish or French or any high resource language, would you still publish it without having a human who speaks that language look at it? Would you publish it if it's a UI of chat GPT that a billion people are going to see. It's a question of trust. Probably right. But if you have a million users, a billion users, maybe you will pay the$50 for someone to just take a look over it before you translate it. So this is in an industry that fundamentally is totally automated, right?

1:01:57There is still the question of trust. And I think that's a question we will grapple with for a long time, there are also just things you want a person to do. Like, I don't think we will have no things to do. But that doesn't mean that some things we do may not dramatically change. And that, you know, that can be very painful for people who do these things. And so this is a serious topic that happy people are engaging with. But I don't think like there will be this global lack of anything for people to do. And maybe as a last question to help us get a sense for what people at the frontier of AI are currently thinking about or working on.

1:02:38Some of the topics that one may see are things like continual learning, world models, robotics, embedded intelligence. So what do you personally find very interesting in addition to what you mentioned upfront multimodal? But what do you personally find really interesting as a research area? Well, I find this general data reinforcement learning is my pet peeve and what I work on, luckily. For example, robotics is probably just an illustration that we are not doing that well in multimodal and that we're not doing that well in general. RL in general reasoning yet. The moment we do really well in multimodal and we manage to generalize reasoning to the physical domains that the robot needs, I think it will see amazing progress.

1:03:31When it does, I have a feeling given that a lot of companies are launching hardware that's kind of teleoperated or glove operated or something. So my suspicion is by the moment we make this progress, which maybe it will be next year, maybe it will be in a few more years. But the hardware will maybe be there by then. And having a robot in the home may be like a big visible change. More visible than, you know, chat. I mean, given how quickly we got used to the self-driving cars in San Francisco, maybe it will be only visible for like the first two days and then be like, yeah, sure, the robot's there.

1:04:12It's always been cleaning since I can remember like the last three months. And it's stunning to me how quickly we get used to these things, right? The self-driving cars in San Francisco are something people got used to so quickly. So maybe this will happen for robots too. Nevertheless, I do think it will be quite dramatic in our perception of the world when it happens. Hardware is hard though, right? Robots may have accidents in the house. You need to be very careful. So maybe it will take longer to deploy them and actually make it a scalable business. We'll see. It is amazing that, you know, we are at this point where we can start thinking like, yes, maybe that will come soon.

1:04:54Lukasz, it's been absolutely wonderful. Thank you so much for spending time with us today. Thank you very much, Matt. Thank you for the invitation. Great to talk to you. Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.