Mojo: A Supercharged Python for AI with Chris Lattner - #634

25:20Absolutely. Are you envisioning that this modular engine is this kind of replacement for CUDA that is multi-hardware capable? Is that the core idea? Yes, I mean, that's one of the value props we provide. So if I zoom out and look at the steps the industry has been going through, so we as an AI industry owe a huge debt of gratitude to CUDA. If you go back to the AlexNet moment, for example, a lot of people talk about it was a confluence of ImageNet and the datasets and things like this. It was a confluence of hardware. And the fact that GPUs enabled an amount of compute that could cause AlexNet to happen.

25:59But a lot of folks forget that Kuda was what enabled some researchers to go write convolution kernels and actually get a machine learning model running on a GPU, which the hardware is definitely not designed for back in the day, right? Today, AI has taken over and it's a little bit different. But back in the day, that initial breakthrough was really in large part thanks to Kuda. And so one of the things that's happened is that as AI has taken over, right, a lot of technology has been built on top of Kuda. It's a very good thing and it's very powerful and flexible and hackable and it's great. But as you say, it's kind of put us into a mode where one vendor has this dominant position and if you're a hardware vendor at even an AMD or some other widely known company that has really impressive hardware to be able to play in this ecosystem.

26:42Now, what's happened and one of the things that led into the thinking that went to modular existing is that there have been a lot of compiler technologies that have been built. For example, there's this XLA compiler that I worked on at Google. There are new compilers every day being announced by different companies where they're saying, I will build a compiler that will make ML go fast, for example, on GPUs. And so several years of work, lots of cool technology, lots of examples of these systems exist, and the names keep changing, but the technology is very powerful. The problem with that is that they have lost one of the things that made CUDA really powerful, which is the programmability.

27:19And so what has happened is the compiler nerds, which I'm a member, so I love the compiler nerds, but those compiler nerds have went and turned AI code generation and things like this into a compiler problem. But that has excluded all the non-compiler people, right? And so if you look at TPUs, for example, TPUs can express everything you can do in this XLA compiler. And so it can do matrix multiplications, convolutions, element-wise ads, et cetera, et cetera, et cetera. But it can't do sparse operations, can't do data operations, can't do pre-processing. And so AI, you're an expert, you know this, AI is not just about matrix multiplication.

27:58It's about data loading, pre-processing, this full parallel compute problem that is part of AI. And so what has been lost over the several years of trying to solve the CUDA lock-in problem is that people have tried to make this a compiler problem. and now you've turned into a different lock-in. But instead of locking into hardware, you're locking most smart people out of the ecosystem. And these compilers haven't been super successful of being compatible with code and things like this, right? And so what Modular is doing is we're saying, okay, again, I love all these people. I've been working on this stuff for a long time myself.

28:33But what we're doing is saying, start from a different perspective. What is our assumption? Our assumption is people don't want to rewrite their code. What that means is you have to have all the operators, all the systems that go into something like TensorFlow or PyTorch need to work. Thousands of operators each, and it's a really messy job, but we handle that job for the world, right? The other thing we say is, okay, PyTorch is really popular in research. TensorFlow is still quite popular in production. What we see out in the industry, again, every shop is a little bit different, but a lot of people have both TensorFlow and PyTorch.

29:03And so they don't want to have this bifurcated stack built on top of these things. They want to actually have one system that they can scale out. And so we make our problem even more complicated by building a unified solution. And so now it's not about 2 ,000 on the TensorFlow side, 2 ,000 on the PyTorch side. It's about 4 ,000, right? And it's actually even worse than that when you bring in some of the other technologies. But now you talk about hardware, right? It's not just about Intel CPUs and NVIDIA GPUs. It's this other access that then does a multiplication to this whole problem and says, okay, well now I have many different, there's probably a hundred or a thousand different kinds of hardware.

29:41And so where traditional teams have built a point solution saying, okay, I'm going to build a fancy compilery thing for one hardware, for one framework and, you know, in one direction along this. And they built one of these often very good tools, but they're very purpose built in one case. You know, we're having sympathy for all the software people that have to deploy because software people, they don't have one piece of hardware. They don't have one model. They don't have one framework. They don't have one product, right? Their products evolve over the course of decades sometimes. And software lives a long time.

30:13And so they need to be able to talk to lots of different generations of this stuff. And so at Modular, what we've done is we've said, okay, well, this is suddenly a very different problem from a technology perspective than building a point solution. And this problem, this I need to solve this massively complicated space where you have hardware on one side, you have the sheer scope of AI on the other space is what drove Mojo to exist because we need a way to make this entire stack accessible, hackable, understandable to people that are not themselves compiler engineers. We need people that know really fancy numerics and sparse algorithms and convolutions and, or people that know their hardware.

30:52We need to know like all these people that are involved in all of this massive technology stack that we've been building to be able to collaborate and work together and build cool stuff at a high velocity, right? And that's where we think that Mojo is really interesting, because as far as I know, nobody's done that. I mean, it's like a completely unique creation in the space, and we hope that will really simplify the world. One of the things we kind of joke about it, that our biggest enemy, you know, the mortal enemy that we struggle with at Modular is actually just complexity, right? In the AI space, there are so many systems, so many technologies, so many layers of stuff that has been built up.

31:29And if you zoom out, coming back to 2016, I thought I was too late to do anything important in AI. What you realize is that AI is still not done, right? The stack you're building on is adolescent. It's in its teenage years. And so what we need is we need to get to that next level where everything actually works way more predictable. It's actually hackable. When you try and experiment as a researcher, the tools don't break out from underneath you. And when you achieve that, we think that the impact of AI can go much further and that many more people can participate. When you talk about the complexity and diversity of underlying components, and then you talk about kind of how the lifespan of software kind of extends over generations of underlying infrastructure, it makes me think of like dependencies and dependency management and packaging and all these things as like huge problems that need to be solved is, does that play into what you're doing at all?

32:25Not directly, but your pattern matching, your neural net there is doing a very good job of pattern matching and seeing what we're talking about here. The packaging problem is often because you have all these incompatible systems that are lashed together. And so if you zoom into Python packaging, I mean, there's a lot of things going on there. I'm not an expert in Python packaging. People I talk to that are, a big part of that is because of the C parts of these Python packages. So you pick our old friend NumPy, for example, right? NumPy has a ton of C code inside of it, as well as the Python API.

32:56Well, packaging that means you're not actually packaging Python, you're packaging C code. C's never had a package manager that's any good, right? And so it's funny, you look at these old problems that we've been struggling with. Well, you get rid of the C code, and suddenly packaging is way simpler, right? And so this is one of the things that Mojo provides, is providing unified language. And more generally, every time you see one of these fissures, like you're talking about the hardware divide. Here we're talking about Python C++. Talk about CUDA versus SICKL versus HIP versus like all these other crazy things that exist in the world.

33:28Like each one of these things is at the bottom of our stack driving complexity up. And so at the end of the day, you'll have a researcher who very reasonably says, hey, I just want to run this model on an AMD GPU. No big deal, right? You should flip a switch, right? But the problem is that at the very bottom, all this stuff is very different and all the cracks go up. And if you take reliability and it's 90 % reliable, and then the next step is 90 % reliable, the next step is 90 % reliable. You start multiplying together all the 0.9s and you get something that's 10 % reliable, right? This is the AI stack that we all depend on.

34:02And you've got this easy problem, which is, well, okay, let me be careful here. You've got this one class of problems that is very challenging, but it's easy to deal with. And that is when you're trying to use all this stuff together and it just doesn't work. Like it doesn't compile or it doesn't run or whatever. But then you have this other problem where it works, but you don't know that it's actually not working. Yeah, exactly. Because of like semantic differences or what have you, it's either not performing well or you're not converging, your results are out of whack. And like you're digging deep into underlying libraries, trying to figure out like, why are your answers like crazy?

34:43Yeah, I give you one example, right? I mean, just go through the lifecycle of deploying a model. So to just make up a scenario, but to just double click on what you're saying, okay, I want to deploy a model. Well, now I need to get it to go through Coromel or one of the many things for deploying to some piece of hardware. Results don't work. Well, now to just plus one you a hundred times. Now you need to know not just PyTorch, not just your model, not just Coromel, but also the translator, also all these things. And you dig in and dig in, dig, dig, dig, dig, dig, dig. And you find out it's handling the edge padding on a convolution slightly differently.

35:15Right. and so now wait a second so like all of these tools were supposed to be making it easy but because they're not all reliable like it's this leaky abstraction now you have to understand all of this complexity right this is what causes it to take three months to deploy a model fundamentally this is something where i think that many folks that are building ai products and they're managing they're the vp of software some technology company right they just see the symptom of why is it take so long to get this model in production? But they don't realize that the tool set, this fundamental technology that all this stuff is built on top of, it's not up to the standards of a software tool set.

35:52No C programmer would tolerate AI tools and their quality. It's just crazy. But again, this is just the maturity of the AI technology space. And by solving that problem, what we want to see is like way more people, way more technology, way more inclusion in the kinds of companies that are able to work with AI and do things. And we think that'll be a really big impact on the world. We've talked about Mojo. We've talked about this inference engine or the engine that we've referred to. In the context of Mojo, you've talked about like 35 ,000 X performance improvements over a standard Python. Do you need the engine to get that level of performance improvement?

36:31Switching, using Mojo, like lock you into using this engine? Like what's the business model there? Do you have licensing issues? I have a bunch of questions kind of coming out here. And they span kind of technical and like business licensing kinds of questions. How does all that work? Great question. So you've identified the right players. There's Mojo, which is a programming language. It's a programming language that's a member of the Python family. It's really useful on, for example, just CPUs, which is the only place that Python plays. And so many people see Mojo as just being a better Python.

37:05Now, we have the engine. The engine itself can stand alone and you can use the engine as a drop-in replacement. It works with TensorFlow PyTorch. It'll make your BERT models go 3X. And you're using it as a drop-in replacement for what exactly? For a traditional TensorFlow implementation. Okay. Before I answer your bigger question, let me dive into that. Yeah. So what the modular engine does is you replace the TensorFlow with our TensorFlow or your PyTorch with our PyTorch or if you're using TorchScript or things like this, and so you just put a new thing in your Docker container. Got it. And what you get from that is massively better performance.

37:42And so, you know, TensorFlow is quite good at production, but we're showing 3 to 5x better performance on, for example, an Intel CPU or an AMD CPU or an ARM-based Graviton server in AWS. And so you think about that and you see 3 to 5x better performance. Well, that's a massive cost savings. 3x less GPUs that you need to... Exactly. That is a massive cost savings. Well, and it's also a massive latency improvement. And so many of our customers love that because then they can turn around and make their models bigger. And so now you can have a better product for your customers. And so you get direct impact on your costs, direct impact on your product.

38:22And this is a huge deal for people. I'm a technology nerd sometimes, right? And I love how it's built, but the impact on products is phenomenal. And the engine is a really big deal for getting production AI to scale. So just kind of continuing down on that line before we click back out, then I would imagine one of the commitments that you need to be making to folks that are thinking about using this thing is how close you're going to stay to the development of that stack, right? Yep, absolutely. Well, so I mean, one of the things also that customers love is that Google and Meta don't actually support TensorFlow or PyTorch.

38:58right these people forget but these are not products right these are open source projects they are hobbies maybe for the megacorps and so you're essentially offering like supported performance optimized version of tensorflow and pytorch absolutely and right but then if i'm going to think about using this i need to know that i'm not going to get left behind like you're gonna you know i'm going to wake up one day and i'm three versions behind the latest thing in tensorflow and it has something that I need in order to make my 500 trillion parameter LLM work. Yep. So, I mean, we're committed to doing that.

39:32So I don't know if this is like a binary question, but yes, we do that. But the thing that the enterprises we talk to that care about their costs, right? Often they want somebody that they can call, right? And if you think about it, right, it's analogous to who wants to run a mail server themselves, right? You can run SendMail or something, right? But nobody in their right mind does that, right? Why do we do this with AI infrastructure, it's because there's no choice. There's been nobody to reach out to, nobody that actually can do this. And the thing that I think many folks forget is that Meta and Google, their technology platform has diverged a lot from what the rest of the industry uses.

40:05So they both have their own chips they build, right, for example, right? And they have their own specific use cases. And so they're not actually focused on making the traditional CPUs, GPUs, and public cloud use case actually really good. That's one of the reasons why we have such high value we can deliver. So yes, this is a product for us. That means we actually support it. That means we invest a huge amount of energy into it. This is one of the reasons why we have such phenomenal results as well. And also to your other question, one of the great things about being a drop-in replacement, from a customer perspective at least, is that it means you can un-drop in.

40:37You can use our technology and if you want to switch back, you can always switch back at any time. And at some point we'll make it back to that broader question, but we've talked about Mojo as being this better Python, but what makes Python usable in AI is not just kind of the core Python. It's all these other things, NumPy and pandas and many other packages. You mentioned, you know, we know they have C at the heart of them. So at some point, there's a significant number of packages that you also have to kind of rewrite that need to be Mojo native, I would think, in order to get the full performance?

41:14Let's dive into compatibility. So Mojo is still a young language. We haven't talked about that, but it's still not, it's not done. And I think it will take another year or so of development before it gets to be like solving all the world's problems that we want to solve, things like this, right? But even today you can import and use arbitrary packages like NumPy, Pandas, TensorFlow, PyTorch, whatever directly into Mojo. And so a really important part of how our stack works is you don't have to rewrite all of your port or touch all of your Python packages. I mean, many people have their own Python code.

41:43It's not just big packages like NumPy, right? And so Mojo talks directly to all those packages. You don't have to write wrappers. It all just works. This is a really big piece of that. Now, if you choose to move your code into the Mojo universe, then you can get the benefits that Mojo provides. And so if you're just talking to an existing package, well, it'll still run at Python speed. It will be fully compatible, but it will also run with the same implementation, this default Python implementation. And so moving your code to Mojo can then unlock these new capabilities, but then you can choose to do that a package at a time or however you'd like to do that.

42:17I guess I'm curious, like how much of the like surface area of AI related packaging have you built? Or am I thinking about this the right way? Like in order to fully provide the performance benefits that you're talking about, did you need to port NumPy over to kind of a Mojo native or to run on MLIR or whatever, at whatever level that makes sense? Did you pandas all these other like, how much did you need to do and how much of that is done like percentage wise relative to what you expect will need to be done to be? Yep, absolutely. Well, so the answer is zero. So our solution is like our solution enables to talk to the entire Python ecosystem out of the box.

43:02So matplotlibscipy, numpy, all that stuff just works. And that, again, come back to being pragmatic and productive. Like, we can't, I'll make fun of you, I'll make fun of me from our last call on the great language debate, right? The problem with any new programming language is that a new programming language has no community, has no package ecosystem, right? And so, again, like myself on that previous call and all the other lovely people there, right? You want to get ML out of Python for whatever reasons is very exciting, but it's not very pragmatic because the entire data science ecosystem is all wrapped around Python.

43:36And Python's also pretty great, right? I mean, I think that that's something that people in other communities like to make fun of Python because of indentation or whatever it is. But Python's beautiful. Subjectively, I'll say it's my opinion. And so what Mojo does is enables you to use literally everything in the Python ecosystem. And then if you want to invest more effort to get more performance, then you can do that, but you don't have to. This is the major value problem. Now, in the case of Modular and why we built Mojo, like business objective of his go make ML really awesome. We care about the matrix multiplications and the convolutions and the core operations that people spend all their time on in AI.

44:14And so we rewrote all of that stuff in Mojo. And so this isn't rewriting Matplotlib, this is like rewriting Intel MKL equivalent, right? Or rewriting the CUDA implementation of these CUDA kernels equivalent, right? And so that's where we've put our energy into because that's what enables unlocking of the hardware, enables unlocking of performance, enables unlocking of usability. We have really exotic, fancy compilery features that enable kernel fusion, automatic kernel fusion and things like this that no normal ML researcher should ever have to know about. They just see, okay, it runs 10x faster in this use case.

44:50Well, that's pretty cool, right? And another thing that I think that folks are struggling with is that take transformers, for example. I mean, you know transformers, I know transformers. We all love transformers. They're eating the world. But one of the problems with this is that because it became so important to so many different use cases, we got all these very hyper-specialized software stacks for transformers. And so these exist at the low levels. So NVIDIA, for example, has a set of kernels called Faster Transformer. These are at the high levels. And so there's all these distribution frameworks for transformers and things like this.

45:22And so you get this very transformer-specialized stack, which again, forces you into this very narrow view of what a transformer is. And it works for the benchmark. But if you're a researcher, You want to go push the boundaries and try slightly different transformers. Or, you know, maybe there's a thing beyond transformers. Like I hear that RNNs are coming back in and maybe FFTs will have their day, right? I mean, there's like all these different theories. And if we can't enable people to do that research, right, we may be missing out on that next big step. And so the specialization that's inherent in things becoming important really cuts against generality.

45:58That's one of the things that we've seen and that we really want to like, again, like if you dramatically reduce the complexity in these stacks, you can make it way more hackable. And that we believe will enable people to invent new things. Yeah. I want to push on this one more time just to make sure, see if I can figure out what I'm, expose any kind of fissures in my understanding here. Is what you're saying that, or is it the case that, you know, in thinking about the relationship between like, NumPy or Pandas and Python, that those libraries that we all use as part of, that are kind of ubiquitous from a machine learning perspective, I can imagine a couple of things.

46:40One, they're sitting on top of the underlying, they delegate enough of what they're doing to the underlying Python that you kind of replacing, fixing that underlying Python gives you some percent of the performance benefit such that you don't need to deal with the upper piece. Who is you here? So are you asking how it works internally? Are you asking how a user uses it? Are you asking when somebody should do something? Which piece of this elephant are you touching? I'm primarily trying to make sure that I understand how it is that you're able to offer the performance improvements that you're boasting without needing to touch any of the libraries that people depend on.

47:32And so I'm kind of asking about internals, but also like how they're used. So I'm imagining like several scenarios. A, for whatever reason, A is like your 35 ,000 number. That's kind of a made up number that doesn't actually rely on any external dependencies. And it's kind of a useless performance boasting metric. That's one possibility. Another possibility is NumPy delegate, you know, these libraries delegate enough of their operations to the underlying Python that you can get the significant performance gains even without touching those things. And hey, if somebody did touch those things, maybe it would be 70 ,000 or whatever.

48:13I can break it down for you if you want. Got it. Okay. Let me break it into a couple of categories. So one is you have unmodified Python that is just imported. So you take map plotlib, just to pick on something that's not performance sensitive. There's no reason to rewrite map plot lib. It's fine, right? And so you just import it. If you import it, the way that runs is that runs with the existing CPython interpreter and Mojo talks to the CPython interpreter. And so that code runs 100 % compatibility. Everything just works great. And this is why the entire ecosystem works, but it's no faster. And so really what you're getting as you're getting at, what are my trade-offs?

48:49What are the levers I'm pulling here? And so full compatibility, but no performance benefit. Those things go together. Another thing you can do, if you go to modular.com, you can see our video and you can see Jeremy giving a demo, Jeremy Howard giving a demo. And there we can see is you say, okay, I just take some Python code I put into Mojo and now it runs, you know, depends on the code, but you know, roughly 10X faster out of the box, maybe 15X, 16X. I mean, there's more we can do to push it further. We just haven't focused on that. And that's running the same code, but in Mojo. And the reason that you get performance is it's compiled instead of interpreted.

49:21It has a new fancy compiler stack, all this stuff under the covers, but it's still running fully dynamic typed code. It's just running dynamically typed code in a better way. And so you can get 10X out of the box. It's pretty good. That's quite nice. Then you start layering in and saying, hey, I want to add types. Okay, well now you're talking about like changing the in-memory representation of this thing to be way more efficient. Well, that's 10X. Now you say, give me threads. Okay, well that's 10X. Okay, now I want to use vectors and new hardware. That's another 10X. And so if you stack all these things up, This is where you get into 35 ,000 times.

49:54And I will agree with you, by the way, that the 35 ,000 number is a cherry-picked number. This is an extreme result on Manelbrot, right, which is a simple algorithm we can explain and people can play with in a notebook and stuff like this. But we have lots of people, just random people on the internet using Mojo that are getting hundreds and thousands of times speedups. And so the 35 ,000 may be cherry-picked, but reasonably expecting getting over 100x is... 100x is pretty big. I consider that to be a pretty big deal. Yeah. Right. And you can look at that as 100x over Python, or you can look at that as saying Python's now 100x more relevant for keeping me out of C.

50:31Both of those sides of that is really cool. Now, anyway, so coming back to your categories. So you tie together. Let me add that third category, or maybe it's more a scenario than a category. It is also, I think, largely the case, but maybe you can validate this for me. You're probably using a lot more of these libraries when you're doing EDA and the early stages of building a model. But then you finally have your model in the form of a graph in TensorFlow or PyTorch. And, you know, at that point, the things that you're relying on are kind of much lower level as opposed to like your pandas and your scipy, all this kind of stuff.

51:11And so your exposure or your need to pull in all these libraries at the kind of the point where you're in the kind of core training loop or inference is less. Yeah, you want to like deploy the model. And so if you zoom into the third part, let's just call, because the things we just talked about are actually completely generic software engineering things, right? We talked about using arbitrary Python package off the shelf. We talked about take Python code, arbitrary Python code in an arbitrary domain and just make it go fast, right? Which is fun, right? But it has nothing to do with AI. Now let's talk about AI.

51:46So AI, third category, super important. And it turns out many of your readers or your watchers want to think about AI, right? And so AI is this really fascinating technology stack that, yeah, you talk to it in Python, but underneath the covers, you have kernel fusing graph compilers and all this accelerators and like all this other cool stuff, right? Yeah. And so this is where the modular engine comes in, right? And Mojo is an implementation detail in the modular engine. And Mojo makes it all super extensible and hackable. But this technology space actually really has nothing, very little to do with syntax or with a programming language.

52:20It's a completely different technology stack that's much more similar to these XLA compilers and the internals of CUDA or the internals of Intel MKL or these kinds of things. So these are all different. But come back to your basic question, several layers up in the stack, which is what is the relationship between Mojo and the modular engine? Because that's also really important. So the modular engine is really focused around high performance, production deployment, go solve problems in AI. And so it's an AI thing. Mojo as a language is actually a new member of the Python family. Yeah. Right. And so for modular, we see the engine as being a product and we see Mojo as a technology.

53:00And both of these things stand alone. So you can use Mojo as just a better Python if that's what you want to do. Or you can use the modular engine as it drops into TensorFlow and PyTorch, and then you just have a better TensorFlow and a better way to deploy your models. But there's a much bigger and hopefully, I believe, much more important in the long term, better together story here, right? Because putting a custom op into TensorFlow or PyTorch is very difficult. We talk about the three-layer problem, right? The Python C++ CUDA. Well, if you want to put a custom CUDA op into PyTorch, you have to write C++.

53:37CUDA is like a C++ thing, right? It's a C++ thing that doesn't have a debugger. A C++ thing with a whole bunch of weird constraints where you might wedge your GPU. And that complexity makes it so people don't do the kind of research that they might otherwise do. And obviously, if you have to hack C++ or even if you just have to rebuild TensorFlow, who in their right mind knows how to do that? I know these people and I love these people, right? But this is monstrous, right? And so the better together here is that if you're an AI person, you're building, deploying models, you're training, you're doing research.

54:08Well, what Mojo inside the modular engine allows you to do is make this whole thing hackable. So you can define custom ops, so you can get kernel fusion, so you can get all this stuff for free. And then when you want to go push boundaries, you can go crack open the box and say, okay, I'm going to write a custom sparse thingy for my domain or a custom summary function that does some fancy domain specific reduction before I send all the data across the wire. and making that possible is, I think, really cool. Interesting. Yeah, I feel like we've covered a lot and there's still a lot to cover, particularly in this dimension of hackability, but we don't have time to cover all that.

54:43To kind of wrap things up, I'd love to have you maybe riff a little bit on future directions, roadmap. What are the big things that you need to attack next to kind of build out this vision? Absolutely. So Modular just came out of stealth. And so we have a nice video on our website at modular.com if you haven't seen it. There's a whole bunch of new drops that we'll be adding to the product over the coming months. And so you can sign for a newsletter on that. The thing I'll say is that Mojo is still quite early. And so it's not ready for production use as a general drop-in Python replacement. But we have an amazing community of people already coming together and we're developing it in the open.

55:22This is, I think, a pretty big deal for something that I hope will be important to a wide range of different use cases. I mean, Python goes everywhere, right? And so I think it's really important that we as a community build and do this together. And Modular is obviously driving this because it's really important to us, but we don't have all the smart people in the world. And so I'd really love for people to join us on our Discord forum and other places where we can interact and build this together. Awesome. You mentioned that Jeremy was a big inspiration. I'm glad he wasn't able to inspire you to ride it around Pearl.

55:54Yeah. Again, he's an incredible person. So he gives a killer demo in our launch showing how to take matrix multiplication. And he doesn't get it 35 ,000 times, but it's 20 ,000 times or something in a notebook, which is pretty cool. That's awesome. He's an incredible person. Don't underestimate Jeremy. Absolutely. Well, Chris, thanks so much for getting us up to speed on what you're working on with Mojo and Modular. Yeah, it's really great to talk to you, Sam. Same. Thank you. all right everyone that's our show for today to learn more about today's guest or the topics mentioned in this interview visit twimla.com of course if you like what you hear on the podcast please subscribe rate and review the show on your favorite podcatcher thanks so much for listening and catch you next time