Breaking Down Modular's $100M Funding Round: The Power of In-House Programming for AI Optimization

0:00So Modular is a Palo Alto based startup and they are aimed essentially optimizing large scale AI systems And so with this new$100 million of funding that they just secured, all of this was essentially led by General Catalyst, which is a massive VC fund. And it also saw participation from Google Ventures, SV Angel, Greylock, and Factory, bringing Modular's total funding now to$130 million because they'd previously done around$30 million. So according to their CEO, Chris Lattner, the funds are going to be used for product expansion, hardware support, and growing Mojo, which is the company's proprietary programming language.

0:41I've seen a bunch of tweets about Mojo. Some people are saying that this is going to be, you know, killing a bunch of other languages. So it's going to be very interesting to see where this actually goes. So the company was co-founded back in 2022 by Lattner and Tim Davis, who are both former Google colleagues. So Modular's mission is to remove the friction and complexity which is often associated with building and maintaining AI systems. So this is what they said, quote, This funding will not be primarily spent on AI compute, but rather improving our core products and scaling to meet our incredible customer demand.

1:15So that was Latner. Both founders believe that the fragmented technical infrastructure in the AI industry is really kind of stemming progress. and they see Modular as a solution to this challenge. So Modular's engine currently in closed preview aims to improve the inference performance of AI models on CPUs and later this year on GPUs, which is going to be really interesting. So the enhancement not only promises to be cost-effective, but also boasts a speed of up to 7.5 times faster for trained models running on native frameworks like Google's TensorFlow flow or metas pytorch and this has compatibility across existing cloud environments and even other ai accelerator engines so modular offers an inviting i think really kind of a solid proposition for developers to to use so alongside its performance boosting engine right because that's a big component of this modular is also developing mojo which i talked about a little bit in the intro but essentially this is a programming language currently in preview that combines the ease of Python with features like caching, adaptive compilation techniques, and metaprogramming.

2:29So according to Lattner, the CEO, the programming languages aim to help defragment their AI technology stack and speed up innovation. So I think the demand for high-performance AI is really just kind of straining existing infrastructure. We're seeing a ton of different companies come out with all sorts of solutions to this. I mean, even the fact that NVIDIA hit a, you know, a trillion dollar market cap at one point because the demand for the chips is so high. I mean, you can look at the stock price of NVIDIA today is absolutely skyrocketing all time highs. And you see really in this whole space, the reason for that is because there is just so much demand for this AI in this infrastructure.

3:13And like they're saying, it kind of is straining the existing infrastructure, right? Because if you want the latest NVIDIA chips, there is a line around the door. You're going to have to fight other AI companies. And I've even seen, you know, it even feels like companies that are able to get their hands on some of these training chips are have a major competitive advantage over others that may have, that might have, you know, perhaps great talent, great data. They just can't get like the infrastructure. They can't get like the chips sometimes. And so I think when you kind of see this situation, you really start to look at, you know, what are the other solutions we can find to solve some of these problems?

3:48And I think this is really where they're trying to plug in. So I think the demand right now is super high. And Microsoft has actually already flagged a hardware shortage that could cause service disruptions. And GPU provider NVIDIA's best performing AI chips are completely sold out until 2024. That's absolutely insane, right? We're in this big AI boom and NVIDIA's chips are sold out. So Latner acknowledges the growing crisis and he says, quote, the compute power needed for today's AI programs is massive and unsustainable under the current model. We're already seeing instances where there is not enough compute capacity to meet demand.

4:27So given this landscape, startups like Deki and OctoML are also really focusing on making training AI models more efficient, but still the adoption of new programming languages in a Python-dominated landscape raises a lot of questions. So as of 2020, 87 % of data scientists were regular Python users. However, Latina remains really optimistic about Mojo, noting its ability to bring together various components of AI applications without sacrificing performance or scalability. So four months after its product keynote, Modular's community has surged over 120 ,000 developers and leading tech companies are already really using the startup's infrastructure.

5:08I think they have, you know, 30 ,000 developers on their waitlist and the company has made a really strong impression in a short period. So the most important enemy of modular, I think, is complexity. That's according to Latner, their CEO. He said, quote, the modular engine and mojo together level the playing field and this is just the start. In a sector, I think, where demand is fast reaching its sustainability limits, right? We're seeing this in all sorts of areas. I think modular is kind of influx and capital could be a real game changer for them and allow the company to kind of expand its suite of solutions to a much broader market, making AI ultimately more accessible, affordable, and, you know, really sustainable for enterprises of all sizes.

5:51So I think from a funding standpoint, the startup is off to a really good start. And now I think we're just going to be keeping our eyes open and watching where they grow and what they do and how their products are received as they actually start launching and getting more mass adoption.