20VC: Are OpenAI and Anthropic Overvalued? The Open-Source AI Reality | How Token Costs Will Fall 10x And Usage Will Explode 100x | The Future Is Not One AGI; It's Millions of Specialised Models with Lin Qiao, Founder and CEO @ Fireworks

22:27Harry Stebbings:Should companies like Harvey and Lagora be building their own model? And actually, if you don't, what happens? So here's one observation I had is software development, especially SaaS space, has been significantly disrupted because of the general intelligence of coding. The application development lifecycle has significantly collapsed in terms of the timeline and the resource needed. In the past, it requires tens of very strong product engineers and PMs to convert from idea to implementation to production scale. Multiple quarters of years of investment. That's a deep mode. And today, one person, a few weeks, can possibly launch their ideas into a product and scale quickly.

23:12This is unprecedented. And that's also created interesting dynamics in redefining where the competition is. because it's really hard just to compete on the idea of application, application by itself, because many people have similar ideas. Now implementation is no longer such a big barrier.

23:31Harry Stebbings:Is that actually true, though, when you're looking at enterprise deployment, enterprise rollout? If you're working with some of the biggest law firms in the world, I mean, the enterprise sales cycle is at least multi-year with relationship build that's very tough. And then you have deployment that's very customized. It's not like 11 labs where you pick it up and go. It's different. And also, I think legal space is particularly challenging because lawyers are usually more conservative. Legal is also not tolerant at all on arrows, right? Because that's why a lawyer gets paid, right? You need to build a very rock-solid case.

24:05If something hallucinates and generates wrong judgment, then you're in trouble. So I do think the legal space is a very interesting space to penetrate. And these companies are both doing a great job. But on the flip side, I do think both companies are owning proprietary knowledge and information, how to build those assistants to do case studies, to go deep in driving legal research and all this, right? So my understanding of legal is so shallow, but there's so many different versions of flavors of cases. So I do think they are in unique position to convert that deep understanding, and they all have data.

24:48It's not just about how defensive their business is. It's about, hey, oftentimes when they build those assistants, there's a harness integrating and deciding, orchestrating which AI tool to use, which tools call into. And this is bespoke. This is customized. And the accuracy of calling those tools and calling into what kind of tools is important. Even that harness need to be co-trained with the model powering it. They're just kind of ample examples of driving that business to excellence by owning their own intelligence of how to do that in the workflow layer. So maybe it's the timing. Coding, for example, I think in coding space, Cursor probably is one of the pioneers starting to tune their model.

25:34And now almost all coding companies tune their own models.

25:36Harry Stebbings:Does that pace of model development slow down? Because every single day it seems like we have a new model with a new capability. And it's like, oh my gosh, Cursor's newest model is amazing. Next, we have Ms. R's newest model is amazing. Gemini's newest model is amazing. In three years' time, will the pace of model development still be so fast and model superiority be so transient, where one day it's one and the next day it's another? So there are a few layers of model advancement. There's base, general IQ advancement. So those will take spec functions. So that's why when they release, there's always major release or minor releases, right?

26:13The major release of spec functions, as you remember beginning of last year, there's a whole this thinking. The thinking process is new. The model just don't spit out answer immediately. The model will think by self and spit out answer is much better that way. So that's one step function. There are many step functions we have seen too, but I see those as every year or every three quarters there's a major leap. But at the same time, build on top of those, the best base models, and I can see the specialization start to accelerate. Because as I said, it's really like a tree, right? There's so many branches and leaves that can possibly hang out on the trunk.

26:53And as the base model quality start to have step function leaps, and there's so much more we can do to specialize. So I do see in the world specialization is going to accelerate much faster than the general intelligence part.

27:08Harry Stebbings:When we think about the general intelligence part, just before we move kind of further into the stack of like multi-model, Sam proffered the 5 % gifting of open AI and others to the administration. Do you think we've reached a stage where model development is so advanced and so important to society that they will in part be government or administration owned? That's very interesting. question. I think there were precedents of that. If we think about the foundation tier of those general intelligence model as fundamentally a base infrastructure for the big economy to operate around, there has been precedents of PG &E owns electricity and gas and so on.

27:51So what I don't want to see is there's only one company owns intelligence. I think that doesn't make sense to me. Because there are different flavors, as I said, there are different flavors of intelligence. There's this general common intelligence that benefits everyone. And then there's a specialized intelligence that actually help us advance in history to think differently, to create a new paradigm of living or new paradigm of doing business and shaping the industry. I don't want

28:28Harry Stebbings:With the many models blooming theory, there's the idea that you will route tasks to different models dependent on what they specialize in. I think so. With that in mind, will you not build your own open router of the world to cater to that? Yes. You can argue they're the best to build it because they deeply understand their use case and they have the evals. So again, my thinking of what is a frontier is not just this one model. The frontier could be your special routing mechanism for your business. And you decompose that based on, hey, in order to fulfill this task, and you need a highly intelligent layer, maybe the most expensive closed models to judge the highest complexity.

29:15And usually people will also be a subagent to solve smaller problems. Then those can go to smaller open models. And those can also further be customized to fit into your special design. So I've seen a lot of people already doing that today. And we also think there's a space to build an automatic routing system that can learn by itself. And that compound with automatic tuning system eventually. We think it should all be automated. And then you can see a self-evolving system based on what flows through your product and your product keeps evolving. Your product is live, right? So you keep deploying and launching new features and to interact with your users.

29:57That just kind of, it will be a totally self-evolving automated system.

30:02Harry Stebbings:Do you think then that routing layer of the stack is valuable? If it can be automated or it can be built on its own, is that a valuable layer to have? I definitely think so. You do think so? I do think so. If it can be automated or companies can build it themselves, why would you need a requestee or an open router? You probably don't. We're not there yet, but I do think this could be an area of innovation. You said Cursor being the front runners in terms of how innovative they've been. I completely agree with you. But I heard, and I really stalk you before shows, but I heard that your CTO, Dima, was embedded at Cursor for months building the RL infrastructure.

30:42Harry Stebbings:Is that how it has to be done? And is that scalable? So what's happening is usually in the early adoption curve of new technology, the early adopters are hackers. A hacker is not in a bad way. It doesn't have a negative connotation. They have deep expertise in certain area and they want to control a lot of things. Versus in the late stage of a new tech adoption curve, it starts to get more accessible to a much bigger cohort user. It doesn't have deep expertise and they need less control. So it always goes into deep control first, usually, and a little control later. So we definitely are aiming towards the later stage as the ultimate time we want to target.

31:27But it's also extremely valuable to understand what is required to get there. So that's why we partner deeply with Cursor. They are the pioneer trying those ideas. They do have researchers from Frontier Labs, and they want to control every single thing. And at the same time, we're also pushing to the boundary. we're doing things that never existed before. We're doing things that never existed before because we push the boundary that is unique to this particular setting. Okay, what is unique is here. Typically, if you think about training, training happens, training is very capital intense, and it usually happens in big companies.

32:01They have a lot of money. They put those money to buy a very expensive training cluster interconnected with each other, super expensive. And then once you have those expensive large fleet, usually you don't need to think too deeply how to be efficient. You just focus on doing your work. Cursor is like us, they're a startup. Both of us are very capital conscious. We want to be efficient while we don't want to slow down the research innovation. So together we figured out a very smart way to drive their training process is they do massive post-training, which is reinforcement learning based. And reinforcement learning, we break that into two pieces.

32:38One is the trainer that is tweaking the weights of the model. and it basically generates a new model version constantly. And that new model will deploy to, we call it RL rollout. It basically is deploying that new version, interact with a synthetic environment, a synthetic coding environment or real coding environment, and then get the reward back to judge if that model version is good or bad. So that's a rough process. And we decouple these two. In the past, in LightH hyperscaler, they run that all together. If you think about you get 10 ,000, 100 ,000 chips all interconnected together through InfiniBand, it's extremely expensive.

33:20It's really hard to find. But then you go really quickly, and we've designed a fully distributed system. We run across five, six data center regions globally and tap into scattered GPUs, and they're able to run massive jobs, our jobs. But the challenge there is we need to sink model weights across all these different regions. And then how hard can that be? It matters because the latency of delay of sending these weights over is going to dictate how fresh the rewards are. And then if it's too stale, then you are too off. So it's a balance. But we innovate a way we can distribute fresh model weights quickly.

34:03It's not too off. So numerically, is still sound, while we are not limited by a very expensive deployment of GPU fleets. So those are the innovations we worked together with Cursor to push the boundary and leading to their recent model launches. We're very proud of them.

34:21Harry Stebbings:Incredible customers have amazing progress they've had with you. It's a very large customer for you. How do you think about the concern of a Cursor churn in the wake of a SpaceX acquisition? Yeah, everyone's concerned. The whole entire industry in terms of application innovations by model in the sense there are few companies that are very successful. They escape velocity, but few of them. So that's the shape of the whole entire industry. And last year, Cursor is one of the few. I would say all model companies are concentrated on Cursor. We concentrate on the same group of app companies. And since then, it does change.

35:01We do have a very healthy, diversified customer base, especially I think last year is the year of coding. I think all major coding companies are on us. And this year is the year of co-work. And co-work is much more diversified by itself than coding because their general purpose co-work, for example, general purpose co-work to help you do all kind of research. You want to ask, hey, what will be the NVIDIA GPU prize two years later? So those are deep research, general purpose deep research. Or there are so many different categories of special purpose co-work. Legal, we just talked about two great legal companies.

35:41Finance, customer support, recruiting, sales, marketing, healthcare. So there's a very broad set of co-work space of innovation app company. They are doing really well. And we have them as our customer base. And then more interestingly, we start to see an uptick of consumer-facing companies are all start to look into JNI technology. They are changing how they are thinking about their traditional business of doing recommendation, for example. And that's very interesting to me because we have obviously worked at a huge recommendation system in the world, Mata. and we are very eager to see how that transform into a new economy for us.

36:25Harry Stebbings:I'm sorry for being naive here. Do people work with just one provider in the inference space like you or do they work with you and with Together or anyone else in the space? I think people are more in tuned to multi-vendor strategy in this space because they don't know what's happening. it feels safe to have multiple providers to balance things out. But we don't view ourselves as an inference provider. Again, we view ourselves as delivering these specialized intelligence where we help companies tune their model. Give you some numbers. Today, we process more than 40 trillion tokens a day. So majority of those tokens are coming from a customized model, not from off-the-shelf models are coming from a customized model.

37:13Harry Stebbings:What will that token count be end of next year? Anywhere ranging from 20 to 100x could be possible. 20 to 100x? Yeah. We're at a very early stage of S-curve of explosion right now. 20 to 100x. If it's 20 to 100x, the idea that we are in a capex bubble is ridiculous and we are desperately needing far more capex than we are ever suggesting for compute. Is that right? That is right. At the same time, I think Jensen has a five-layered AI cake from top-down application, model, infrastructure, chips, energy. We are bottlenecked by the lower part of the AI cake in terms of supply chain. Being energy.

38:00Being energy, being chips in the physical world, how fast we can manufacture. Because in the history, all these industry is not designed for massive scaling. Speaking about 100x scaling, no one was designed for that. I talk with many manufacturers, it's kind of well bottlenecked by small parts. Transistor. The smallest tiny parts that hold off the whole manufacturer line of servers that can deploy to data center and be used to generate tokens.

38:31Harry Stebbings:Do you have to be full, we're going to Jensen's five-layered AI cake. Do you have to then be full stack to win or to reduce dependencies? We've seen OpenAI come out with Jalapeno, terrible name, Anthropic talking to Samsung about building their own chips, DeepSeeker building their own chips, Zuck came out with Meta building their own chips. Do you have to be all of it? It really depends on the company philosophy. To us, agility is everything. And we need to earn the rights of building anything. So focus is everything for us. And we want to focus on where we add the biggest amount of value based on our strength.

39:07We would like to leverage other people's strength to build on top of. So in particular, we want to run everywhere, all possible air chips in the world. We don't want to limit it by how much chips we can bring into our data center, whether we're constructed or rented. But over time, when the business grows very big, right? So I still remember when Matt was young, they don't build everything. And when they're big, they make sense to build. You earn the rights to build for your own giant traffic. And if it saves like five times more cost, then you should go do it, right? But I think at the earliest, that's why I tell you an interesting story in the coding space.

39:47I would say Cursor is the first company they have decided to work with us early on. I still remember when they worked with us, they were single-digit million dollar, very small. It was only two years ago. They grew by a thousand X over two years. But they decided to work with us early on because they recognized they only want to focus on product innovation and later on research. They do not want to focus on, you know, this platform innovation. They know we are putting all our R &D there and they want to find the best partner to win big. I do think that's the right mentality to specialize. And we want to specialize.

40:23We do not want to kind of own the whole entire stack. That's not our goal as a company.

40:28Harry Stebbings:I'm sorry to be harping on that. Why does Jensen skip your layer of the cake? Because he's doing Nemotron with models. Why does he not want to cannibalize your business too? Jensen is not building a cloud either, right? You can say, hey, Jensen probably have all the rights to build an NVIDIA cloud. So he's not building a cloud infrastructure. I think he mentioned that as well. I mean, people asked him that question. And he also mentioned he wants to specialize in what they have the rights to do. Why models? I think it's pure supply chain question. If the US doesn't have a US native open model, it's a problem.

41:06It's a supply chain problem. So he is solely there to solve the supply chain problem. But if there's no supply chain problem because the company like us are providing this specialized intelligence platform layer, then he doesn't need to worry about it. So he just wants to make sure the whole entire five layers of AI cake is flowing. There's no blockage. And if there's a blockage, he's interested in solving those problems.

41:29Harry Stebbings:Mark Benninghoff, one of your investors, I think, in the new round, which obviously this will come out after the round is announced, said that he spends about 3.8 % of developer salaries at Salesforce on Anthropic and Claw Code. And I think it's a useful analogy because if you assume that that is what's spent on crawl code and coding tools, that says one side of the market. But if it's 20%, wow, we're underestimating how big these companies can be. When you think forward a year or two, what percent of developer salaries do you think will spend? Is it less because these tools will get cheaper or is it more because they'll get better and better?

42:11I do think the cost of token will go down drastically.

42:17Harry Stebbings:Because it hasn't so far. It hasn't so far because of supply chain constraint. But we are living in a free economy. So think about whenever there's shortage, price is high. Price is high. High price will invite a lot of people coming in to solve the problem. And it will invite competition. Competition will bring down the cost. And then eventually we're leading to a very economical solution. But actually, that's good for everyone because much more affordable infrastructure will invite more usage. So my prediction is with the decrease of infrastructure, that's where it comes to my prediction of how far next year will look like.

42:54Because the infrastructure costs will go down and usage will explode because of that. So the moment you don't think about that as a problem for you, if it's a utility, you just use it.

43:07Harry Stebbings:How much will token costs come down? Is this like a halving? Is it like, oh, it'll be a hundredth of the cost? So there's different ways to think about it. Not all tokens are equal. I think we should establish a best practice to evaluate the token economy per task. Because different models have different ways of spit out tokens. Some are much more verbose than the other. So you can imagine one model is 2x cheaper than the other, but it's 2x mod variables to solve the same tasks and then they're the same cost. But overall, I think as the model quality improve, I think being precise is going to be part of the optimization.

43:49And so that's one level of optimization is to solve one task, we should need less tokens. And the second is for one token and how to do that is you need to customize the model to solve your problem, especially better. and more precise. That goes into model tuning. And second is for each token spit out from those models and processed by those models, we also specialize in making the unit economists much better through our platform. And third is underlying infrastructure like the GPUs, the surrounding memories, and all this. Today is under stark supply chain constraint, is going to get much better.

44:29Situation will get much better. I don't think probably in the next one year or a year and a half, this situation will not change. But in the long term, two to three years, it should change. And that cost will compress. So overall, I can imagine 10x cost reduction in the next three years. And this 10x cost reduction will drive 100x usage.

44:50Harry Stebbings:You said there about token efficiency and how you enable your customers to be much more efficient. With that efficiency, you do charge more. When I did the research, when it compared to competitors, I got like, together's price king. And I don't mean this disparagingly, but like, they're cheaper. If you want cheap, you go there. And respectfully, if you want better quality product, you go to you. But it is more expensive. Do you think that's a fair assessment and a fair analogy? I think we're probably not comparing apples to apples in the sense that, again, goes back to our business. Majority of our traffic is customized model.

45:26We optimize for quality, Number one, always quality. Quality as in model quality towards your applications, your specific business, your use case, and so on. The second is when we deliver those models in inference, it's also quality. And we care quality so much, we do extreme things. For example, during training time, there's a very hard thing to achieve. It's called zero KLD. It's a little bit technical. The idea here is -

45:53Harry Stebbings:Zero KLD? KLD. KLD is a measure of quality. And what it means is between the training system and the inference system, when model moves over, we have bit equivalence. So as in the numerics are fully the same. We do not lose a bit of accuracy. That's really hard to achieve. But the reason we push that, we deliver that. And the reason we push that is because we know primary business is in model customization and inference of customized model. And we want our customers, every single dollar invest in training, maximize it. And then if cross-training inference boundary is not bid-wise equivalent, they just drop the quality down.

46:40And then it's like you pay your training investment by discounted quality. Why do you do that? So quality first, and quality does bring additional value. And that's why we are not interested in commoditized one-size-fits-all. You know, this off-the-shelf model deployed in the same way for everyone, that kind of business. We'll always customize model, deploy in a unique way for your particular workload.

47:07Harry Stebbings:Two questions. Do you have to have an FDE model to make the customized model efficient? As a matter of fact, we do have an FDE team. It's called Applied Machine Learning Engineering Team. So their primary job is to accelerate this customized deployment. And as a matter of fact, we'll also build an agent to automate a lot of deployments. So given where we are in the stack, a lot of the complexity that we have, we have a margin structure that's a little bit different to like traditional SaaS being 80%. I don't know the margins precisely here, but they traditionally sit in the 30 to 40 % range for where we are.

47:42Harry Stebbings:Is that the new normal for where we are? I don't think that's the new normal. That is a reflection, at least for us, I don't know other companies. For us, it is a reflection of we are in a hyper growth phase. During hyper growth phase, you have the choice, right? You either optimize, to me, margin optimization is a constraint problem, as in, hey, we want to go to 70 % margin, we want to go to 80 % margin, and then we are going to go backwards and impose those constraints to guarantee those margins. And usually constraints slow down innovation. So I'll give you an example. During system development and in a high-velocity system expanding phase, we don't want to overbuild because we're in kind of high experimentation.

48:27We're testing what will stay, what will not stay. Optimization doesn't make any sense. Once we know this is a system that we want to build 100 % and we are going to scale this 1 ,000 times bigger, then we go optimize the heck out of it. I think you might not think about business the same way. We're in the hyper growth. If our focus is only optimized growth margin, we absolutely can do that. But we are sacrificing the speed of growth because we want to go everywhere. We want to go into different geo regions. We want to go into tackle different use cases. We want to constantly create different product lines.

49:05And those are not the time for optimization. That's my opinion.

49:10Harry Stebbings:So we will be able to increase margin without moving into different layers of the stack. not into, we absolutely are not going to move into application layer. Very clear to us. Whether we will move down into build data centers and so on, that could be always be on the table, but the question is timing. Isn't the statement you either die or you live long enough to build your own data centers? As Elon or Zuck now are spending, I think, 10 billion on the latest data center in Canada. Would you like to build data centers? So I have built data centers at meta. Also, lots of innovation possible there.

49:46There's no one-size-fits-all as well. And building a GPU-native data center is also interesting. So it's a trade-off, right? From an operation point of view, it's much better to build a heterogeneous deployment. It's all the same chips, all the same SKU, as big as possible, and run multiple workloads, so it's fungible, right? It's very easy to manage. You build one principle, one process to do maintenance operation. Again, it goes to optimization. But once it's so big, then any optimization is going to drive a lot of economic return. For example, we're talking about NVIDIA recently acquired a company also called Grok with Q.

50:20It's a large SRAM-based ASIC accelerator.

50:26Harry Stebbings:I spoke to Jonathan before this show. Jonathan is excellent. He said what a fan he is of yours. Oh, also fan of his. So, but it's a great combination between a Flops Intense GPU and SRAM Intense A6. Because Flop Intense is really good for first half of LN processing. It's pre-fill. It's called pre-fill. Processing the prompts and so on. And SRAM Intense is really good for generation. That's just the nature of the model architecture. It's great to combine these two instead of running heterogeneously on the same chip. But that requires a very unique system design and deployment into data center.

51:05And it is heterogeneous, actually, before I really mean homogeneous. Design is much better for operation. And this is heterogeneous. And then how to operate this heterogeneous design requires unique innovation in data center deployment and so on.

51:19Harry Stebbings:So data centers aren't commoditized. You can specialize in data center deployment and one data center is better than another. And data center deployment can be done well and badly. Data science is so complicated, right? If you think about the beginning, all the way from construction to power deployment, you have the right power to come in, right? Fiber channel, the right cooling, especially newer chips requires liquid cooling to get all this right and the parts can fall apart and how to replace them. It is all very deep expertise. It's no joke. It's not tomorrow I can be a data center operator.

51:56I cannot.

51:57Harry Stebbings:Is that not where you would bet long on China? with the greatest respect, especially in the US, one of the biggest barriers to data center deployment is policy and is local legal infrastructure that prevents it. In China, you don't have any of that and data center deployment is much, much faster. I think in general, infrastructure, the base physical infrastructure construction in China is going really fast. I literally see some kind of crossover bridge being built within a week. The velocity is very, very high there. and there's a highway close to my home after one year it's not done yet so it's also a crossover so i do think there's a unique strength probably because of the population density and they are specializing in those kind of construction but i do think we also have those specialty people it's just even i heard even electrician is under severe shortage we are under global supply chain constraint here.

52:54Harry Stebbings:What change would moving into the data center layer cause to margins? Would that take it from 30 to 50? Would it be not that meaningful? Like what would that change do to margins? How we calculate gross margin is interesting these days because how long does hardware depreciate has significantly changed. Yeah. In the past it's six years, solid six years, and hardware release is usually three years. That's fast. And now within a year from one vendor alone, we have three SKUs. The newer model usually runs the best on the newest hardware. Model depreciation is also very fast. Every week we are launching a new model.

53:34And then the model is kind of peak in its value before the next model comes out. And the new model likes the newest hardware. And imagine this cadence after two years, which model runs on the two years old hardware? It will be two year old model. Are those models still valuable? So I think that's kind of the real dynamics we're facing right now is the hardware will last for six years still.

54:01Harry Stebbings:But what you're saying, the speed of model development far outstrips the speed of chip and hardware. The speed of model development definitely is the fastest, but even the hardware innovation itself is the fastest. So after three years, if every year there's three hardware skills, after three years, there are nine hardware skills in between. Do you still want to go back to nine generation older hardware running three years old model on that? That's questionable. Maybe there's a war. It's still valuable. But with this pace of innovation, it's questionable. Now with a different depreciation cycle, it changes the dynamics of build versus own, build versus buy.

54:38And again, it goes back to my original thesis of do you optimize for growth or do you optimize for growth margin? It's all about timing.

54:46Harry Stebbings:How do you think about that question? When you're sitting there in an armchair on a Sunday afternoon thinking, hmm, we're optimizing for growth now. When is that time to optimize for gross margin? Well, I would say we want to optimize for both. So here's how I think about it. Optimize for growth requires a lot of business planning, assuming there's a product market fit. Optimize for growth margin is optimize for differentiation. I think I want to avoid over-optimizing for growth margin. But we should optimize for growth margin continuously, as in we should optimize for product differentiation continuously.

55:21There's no question about it. I think we want to continue to optimize towards a healthy growth margin, which allows us to grow really fast. And it's a trade-off, and we don't want to take compromises. The compromise as in we over-optimized gross margin to result in a very slow growth. One possible way to optimize gross margin, we do not grow at all. We just optimize the heck out of it. I know we can kill climb to a high number, but that's absolutely a disaster outcome.

55:49Harry Stebbings:Okay, interesting. If we just said, hey, so gross margin, we're going to take it from 30 % to 10%. Is it a winner take all market where we could eat up everyone else's lunch and then optimize gross margin later? We know it's all probably is not a snapshot in time. It's going to be a long-term situation. We do see a particular industry will oscillate and start to settle with a few good ones. Take legal, for example. I was on a dinner table and interesting. It seems like there were a lot of those companies around two years ago, but now it's pretty much two. I think it's a long game. How do you see the more mature state of your market?

56:31Harry Stebbings:Is it like a cloud market where you have obviously Azure, AWS, GCP, or is it an Uber and a Lyft where one takes 90 % and the others kind of fight for scraps? We're in the adoption curve where a lot more companies, they are in the AI space, start to seriously think about moving to specialized intelligence. To start to seriously think about owning their intelligence is better than renting. Because going back to this optimization, when is the good timing, right? So it's the same question we're answering for ourselves when build versus buy. And our customers also think about build versus buy or build versus rent or own versus rent.

57:09I think AI journey or AI adoption journey has gone further along into a lot of company has meaningful traffic. A lot of company is deploying AI into production. A lot of company is at the phase of scaling. And that's where optimization kicks in. When optimization kicks in, you need to have control to optimize. If you don't have control, you just don't have the range to optimize. And for you to have the control, then you have to build on top of some open model. You have to kind of turn your data into your intelligence. That's pretty much the path that we've seen. So many companies across the industry, they reach the same conclusion that are moving towards this direction.

57:51Harry Stebbings:Speaking of owning your own intelligence versus renting it, that does apply to a national layer. And when we've seen Fable be banned in some cases by the administration briefly for 19 days, especially in Europe, we suddenly went, oh my gosh, we cannot be at the hands of open AI and Anthropik where we can just be banned in our health services, sit on the infrastructure of something that an administration can turn off. Do we see a future of sovereign models where large nations or nation blocks own sovereign models? I definitely see that possibility. I also see if we think about the general intelligence model as the electricity layer, as a power line, every country should own their own power line.

58:33So I think that is a very scary moment is my power line is going to be cut off and all my fundamental day to day is going to not working because I feel so frustrated whenever there is a power outage in my home alone. I feel so frustrated when I cannot access my Wi-Fi. I feel so anxious. I mean, obviously, operating the country is extremely important built on top of this fundamental baseline. And for every single company, it's the same thing. It's not just whether a country should have their unique sovereign independence, but every single company should have their independence. You don't want any single person to cut you off.

59:15That's an extremely scary moment.

59:17Harry Stebbings:Why would you move into the data center space, but you wouldn't move into the chip space? Because I know building a chip is extremely hard. I thought so too. But then how come everyone is seemingly doing it as if it's like just another product? As I said, OpenAI, Anthropic, DeepSeek, Meta. We're building our own chips now. I think Meta has been building their chips for more than five years, way more than five years. And MTIA has been a project since 2018, maybe earlier. Because Meta has been investing in AI for a long time, pre-Gen AI. And they have a huge AR workload to focus on ranking recommendation.

59:55And Meta has been building other hardware as well in the past. So whenever the usage has passed a certain threshold, it makes economic sense for you to build the underlying supply. And then you can specialize towards your workload. And that's another form of specialization is specialize to bake your logic into hardware. And this hardware is purpose-built for your particular workload. And you better make sure this workload doesn't change. because it's really hard. Once the hardware is taped out, it's really hard to go back and change it. It's possible it's very costly. So once your workload's stabilized, once your business's stabilized, it doesn't change too often, then that's the time to consider building a chip.

1:00:33I still see the whole AI world, especially models, customization, is very dynamic. Workload pattern is very dynamic. So think about how much energy in the application space. People are experimenting all kinds of things. You don't know which one is going to take off And they will just take off quickly. And once they take off, which one is going to sustain? And a few ones will sustain. Then that's the time. Oh, now we know this is a pattern. And now we should probably encode this pattern into hardware and bring this hardware into a data center and so on. It's all cascading. And then it's going to cascade down.

1:01:08To me, it's a fernal question. Where are we in the stage of fernal? Maturity fernal, I mean. We're still in the early stage of workload maturity fernal to warrant a chip that will be durable. So now you go back to, oh, we have so many accelerators that are successful. Some are really successful. But remember those ASIC company, they started before JNI. They started from some thesis to optimize some workload. And they all pivot to AI and trying to kind of fit in AI workload. It's almost like you bet before this AI workload emerges, and now it becomes a serendipity question. Are you lucky enough this just work, right?

1:01:49And some really worked. So fundamental design of putting a lot of SRAM on the chip is great for Air model because they are memory hungry. And this really accelerate the execution of inference and so on. So those works. And some doesn't work.

1:02:05Harry Stebbings:What do you see as the greatest bottleneck today? You know, I think it was when I had Jonathan from Grok on the show who said like HBM was the greatest bottleneck. And that's why you've seen the 5X increase in price. What do you see as the greatest bottleneck that people don't talk about enough? I still think we don't have a great system for a very large model. I really believe the fundamental lower-level infrastructure costs will go down. For solving tasks, we should need less tokens. That will increase. So collectively, the costs will significantly reduce. Therefore, we can run the highest intelligence model much more ubiquitously in the future.

1:02:44but we don't have a system designing for that. For example, we don't have a great system designed for 10 trillion parameter models today. That will require very smart engineering co-design from the model to the customization serving platform layer all the way to chip layer. The chip is not the individual chip, but the system. Collection of chips in system and all as a total package. I think there's still a lot of innovation we can do.

1:03:11Harry Stebbings:I think recently you announced that you were at 800 million in ARR. Incredible feat and scaled so fast. What is that at the end of this year? We think we can at least double. By the end of the year. Wow. You know what's so interesting for me as a venture investor? I've been investing for 10 years. We used to be in the day where Slack was the golden child. We're like 1 to 10 million in revenue in 18 months. It was like amazing. And now we have companies like Fireworks, where you scale to 800 million in revenue in a matter of years. And you mentioned Cursor, scaling to billions in revenue in a matter of years.

1:03:46Harry Stebbings:The speed of company revenue growth is just unparalleled. I think it's because there's a fundamental disruption in this technology that is all-empowering. All-empowering in a sense it reach out to every individual one of us to be creative. And it unleashed a lot of creativity that we just don't have access to. And that's why we're seeing this phenomenon of extremely fast growth because of a demand. Final one before we do a quick fire. You hired George Hugh, who was president of Salesforce. He's exceptional. He's one of the most direct, no BS operators I've ever met. But you met him a couple of years before or a year before and you were like, oh, we're not ready for you yet.

1:04:30Harry Stebbings:Why did you say that? and why did you decide now was the time? So a year ago, I think we were probably just 50 people. So today we're at 200 people. We're still not that big. Wow, you're 4 million ahead. At 50 people, I'm more thinking about scaling the product first than scaling massively scale the business. And we talked, and I have huge respect to him. I know he's legendary. He's a legendary operator in Silicon Valley. I just feel like we're too small for him. And I told him that, hey, we're probably too small for you, but I would like to work with you at some capacity. So he helped me actually build out the team, interview a lot of executives.

1:05:09His feedback is always well-balanced, very thoughtful. And we started to work together in that capacity. And to, I think, end of last year, we're growing really fast. And he knows. And we started talking seriously. And that early relationship paid off. He's really cool in the sense that he did a lot of things. a great accomplishment in the past. I find a unique character about him is he's extremely experienced, has high attitude of business vision, but he's also very curious. He doesn't make assumptions. I know it all. I've seen all the movies. It's the same movie. And let me just kind of direct this movie as I did in the past.

1:05:52So he didn't come with that attitude. He knows AI goes in insanely fast pace, learning along the way, but also fully embrace AI. Actually, his team, our GTM team is using all kinds of AI agents. They're sharing skills, so they maximize their productivity. And he knows we have a superlinear demand curve. There is just a certain pace we can build our GTM team. In order for us to catch this curve, we need to build a team, but the team needs to also have increasing productivity to match. So that's a problem he's solving, and I feel very fortunate to work with him. In general, I feel in the AI space, the unique part is people need to have very special traits, almost like contradictory characteristics.

1:06:38For example, very experienced but super curious in the fast learning curve. Or Dima, we talked a little bit earlier. He is brilliant, high intellectual horsepower, but extremely humble. It's a weird combination. And he's almost like cynical in an Eastern European way, but also at the same time very humble.

1:06:56Harry Stebbings:Can I do a quick fire round? Okay, let's do it. Okay. What have you changed your mind on most in the last 12 months? I think how fast we grow, I changed my mind because I have been quite worried about too big a team too early. So that's why when I met George, I told him, we're too small for you because I don't intend to grow very fast in terms of people. I worry about getting slowed down and lose agility and velocity very deeply. But since then, we have been very aggressively using air tools. we have developed our own unique way of hiring certain type of people that we know they will be charging forward with high velocity as extreme sense of ownership, very communicative, and never take no as an answer.

1:07:42So we also learned how to get those people. And now I feel much more comfortable skating really fast.

1:07:47Harry Stebbings:What's your type of people? And I know that sounds weird, but like our type of people is actually really specific. Pretty much only hire immigrants. British people don't work very hard sorry very scientific and rigorous use data for most things i actually think creativity often comes from data and is informed by data and unwaveringly like accountable and ownership like nothing is anyone else's fault it's all my fault even if it's someone else's fault that's a 20 vc person what would you say yours is in weird way it's not competence we want people with the high competence but more importantly the strong indicator whether they will do well in this wave, especially in fireworks, is whether they are really built for taking extreme ownership.

1:08:29Extreme ownership as in we are not putting people anywhere in any boxes, and we're just stacking the box together into a tower. People just automatically claim, hey, this is an end-to-end problem. I'm going to see through the whole thing and work with a bunch of people to make it happen, and I'm going to deliver it no matter what. so those kind of people has the highest longest mileage and their growth curve is amazing also

1:08:56Harry Stebbings:what's your biggest lesson from working with jensen huang on what makes him so special he's everywhere i seriously think he has a clone of like hundreds of jensen somehow for example i sent him an email he will reply in one minute i just don't understand how he's constantly in details. But now I operate a company for four years, I understand why he's doing that. That defines velocity. Because what is leadership? Leadership is just judgment. It's not privilege. It's judgment. You basically have the context. You need to have the right context to make the right judgment for the team. And especially in a high velocity space, if you do not know what's happening, what works, what doesn't work, what are the gaps, you make the wrong call.

1:09:46In a slow-moving space, you can wait for the cascading information, up and down, and make those calls. But in a fast iteration space, you just cannot wait. Because it's guaranteed, there is information loss in transition, layer after layer, people after people. It always happens. And not knowing what exactly is happening, and having the precision of make judgment makes bad leadership. He is demonstrated through his own example, even before this crazy AI thing, he's operating that way. And before I was admiring him, his sheer amount of volume of capability of doing that, now I understand the wisdom behind that because I also operate that way.

1:10:30I need to know what's happening on the ground to make the judgment for the company.

1:10:34Harry Stebbings:What did you wait on in the fireworks journey that you wish you hadn't waited on? Marketing. We talk about it. So we are a little bit nerdy in this way. At the very beginning of our journey, we kind of, we didn't discuss it, but we feel product will speak for itself. At the end, product stands. And we want to devote all our effort and focus on building product, work with the customer, validate product market fit, and go from there. And we didn't spend much time marketing at all. We didn't prioritize educating our customer what's the right direction to think about the trend and the value. But we do think now, I do think it's important.

1:11:18Marketing is not about flubs. It's more about education. It's more about clarity. And we are working on that.

1:11:27Harry Stebbings:What area of AI is under-invested in today, in your mind? You mentioned like cooling or servers. What areas are you under-invested in? AI has the sexy part of this is such innovative, creative technology. And build something on top of it is the focus. But monitoring the ROI, I think the industry starts to kind of pay attention to it. But eventually that's what matters. Not how much spend is, what is the return? And what is the cost? And what is the attribution? In the next couple of years, as AI is getting more and more into production, there will be a lot of focus in getting that clarity and getting that discipline out.

1:12:09The token maxing is just a thing in time, but we're quickly moving to ROI maxing, which is all about running a business.

1:12:17Harry Stebbings:What large customer do you not have that you would most like to have? So we haven't spent too much time in traditional enterprise segment. I think that's just because we were very small. And now as we build out our company, I do think even without us investing, we have customers like Geico, like Capital One, all these companies. So even without us pursuing enterprise, traditional enterprise, they come to us. But I do think that's a very big market. What has to happen before the end of the year that hasn't happened for you to consider it a good year? I'm confident in our capability of driving the business.

1:12:57And to me, this is a year I want to prove we can scale quickly by keeping the same velocity. And that's very important to me. If we reach that point, we reach a proof point, and next year I have a lot more confidence to continue to scale extremely aggressively. I want to make sure we do it right this year.

1:13:13Harry Stebbings:Final one for you. What does no one see about the next three years that you see very clearly happening or not happening? I really see people will own their, every single company will own their own intelligence as a must-have. It's not optional. That's a trend I'm seeing. Because there's an analogy to software is there's a reason why every company build their own software stack. There's no standardized software you just use off the shelf to solve a problem because every single company is solving a unique problem. And they want to build software because they want to have full control. And obviously they will pick and choose which part of the stack they want to build themselves, which part of the stack is common knowledge, there's no point of building.

1:13:56Harry Stebbings:But every single company owns their own software stack. Obviously we're talking about this in the SaaS time, right? So same, I think at a time, every single company should own their own intelligence. Lin, it was Matt that introduced us first. I've had the joy of getting to know you and obviously George. I can't thank you enough for joining me, for coming in person. It is so wonderful to do it in person and you've been fantastic. That's an amazing studio. You asked a lot of interesting questions. I have a lot of fun talking with you. We do a lot of research before, huh? Yes, you did. But before we leave you today, founders face a different set of challenges at every stage of growth.

1:14:33Harry Stebbings:For Sid Shait, co-founder and CEO of Dematrix, JP Morgan delivered the guidance and expertise to help navigate what came next. He credits JP Morgan's high-touch approach with supporting Dematrix as it grew and expanded internationally. Whether you're in the early days or expanding into new markets, JPMorgan helps startups navigate complexity with real confidence, offering personalized guidance and deep sector expertise. Find out how JPMorgan helps founders at jpmorgan.com forward slash grow without limits. JPMorgan is the bank of the innovation economy. While JPMorgan powers your finances, Navan keeps your team moving.

1:15:16Harry Stebbings:Did you know the industry average for booking a business trip is 45 minutes? That's a massive waste of your team's time. Well, with Navan, your employees can book a trip in just seven on average. Navan is the AI-powered travel and expense platform designed for companies that value efficiency. It drives real business impact through high employee adoption and automated policy control. Now, the built-in AI approves in-policy bookings and blocks the rest automatically. This allows finance teams to stop chasing receipts and skip the month-end chaos, and you get this real-time visibility that can save your company up to 15 % on your travel budget.

1:15:52Harry Stebbings:And that's why leaders like Visa, Stripe, Figma, and even Anthropic rely on Navan these days. Go to navan.com forward slash 20VC today to see for yourself, and you'll get a chance to win two business-class flights anywhere in continental US. No purchase necessary, rules apply. Head over to nevan.com forward slash 20vc now. While Nevan keeps your team moving, Base44 helps you build faster. You have the idea, but with most AI tools, you hit a wall. The setup, the config, the gap between what you pictured and what you actually ship. Well, Base44 is where that wall disappears. You describe it? Yeah, Base44 builds it.

1:16:29Harry Stebbings:Apps, websites, AI agents, real working products, built-in minutes, using nothing but plain language. And it's all batteries included, The backend, the database, the authentication, the hosting, the heavy lifting is handled. So you just really stay in the flow. This doesn't just take the busy work off your plate, but it gives you an advantage and pushes you past what you thought you could build alone. So in this market, fast is the baseline. To win, you just have to be first. Base 44 is that edge, the move that skips the troubleshooting and gets you straight to the breakthrough. Build your next thing at base44.com.

1:17:03Harry Stebbings:That's base44.com.