Fireworks AI

0:00As we all know, starting from this year, a lot of companies will build AI into bankruptcy. This concern was never even a topic to discuss in the SaaS era because product market fit and durable business almost means one thing. Because the business operation, the cogs of business operation is quite low. But in the AI era, product market fit would mean your ICP just loves your product. But it doesn't mean you have a doable business because your costs of operating using AI could be much bigger than your income. And then when you scale, you literally scale into bankruptcy. This is a real issue. So we came here to solve both issues.

0:37I'm talking about five to ten times cheaper using. This is Lin Chiao, co-founder and CEO of Fireworks AI. She helped build the go-to platform that helps some of the biggest companies customize and run production AI models at a fraction of the cost. And now she's revealing why generic models destroy your unit economics and how to turn your private data into the ultimate moat. I think everything boils down to data. Product development has been disrupted. Product as is, is not the moat. Because of coding agent, products cannot be easier to copy. So then what is a real moat, right? So every company, they are solving a unique problem in a special way.

1:14Because they're solving a special problem exceptionally well, they will have unique product taste, judgment, and design. And what makes them continue to be unique is the data they have accumulated from the power surface area interacting with their customer. That is their moat. That's the foundational value of why they exist. And those companies will continue to thrive by building their own specialized intelligence. using your private data we can tune a highly intelligent model towards sota quality but with much better unit of economics that means for the same budget you can grow your custom base 10 times bigger there's a framework we use to think about that basically welcome to new economies

2:11Hey, Lin, welcome to New Economies. Thank you so much for joining us today. Thank you for having me. I'm so excited to have you here. Jensen has recently said you're one of the most interesting and most exciting companies to watch. Why did he say that? And why is Fireworks so exciting right now? We started Fireworks about four years ago. We can talk about our background and history, but we have been working in AI for more than 10 years. I think what's interesting is our bet in the future of AI. As we all know, the Frontier Labs, they're betting on general intelligence. They're building this one model that are aiming to solve common tasks very well and offer those models as black box API for all applications to build on top of.

2:59We think differently. We think the world will be kind of much more diversified. And every company, they are solving a unique problem in a special way, exceptionally well. That's the foundational value of why they exist. Those companies will continue to thrive by building their own specialized intelligence. And this specialized intelligence is built on their unique data from their product surface area, capture their product taste, judgment, and design. and those models will continue to grow deeper and deeper into their domain and really be the anchor of their moat. So Fireworks, we execute on the specialized intelligence vision and help all application companies and enterprises who has a huge amount of private data to customize their model, customize model deployment for the highest quality, speed, and cost efficiency.

4:03So that's why I feel like that's extremely exciting for the whole entire industry. And that comment from Jason came up in recent conversation I had with him when we talk about specialized intelligence. You've known Jensen, I think, for a while. What makes Jensen and NVIDIA such an incredible, enduring company, do you think? It has a lot to do with how Jensen operates the company. I think he has a very unique philosophy of company operation, which I share a lot of value. I think to me, leadership is not privilege. Leadership is judgment. And our job as leaders in a company is to deliver the best judgment to move company forward.

4:52Those judgment is reflected in our vision and our operation chops. And in order to get strong and correct vision and operation chops, it requires us to really understand a lot of context everywhere across the company. And before it's hard because without AI, pre-AI, right, pre-gen AI, we are designing the organization, relying on organization hierarchical structure to operate. And there has been more established communication flow and the management structure up and down the chain in the past few decades. But I would say Jensen's operation is completely different from a conventional company operation.

5:43He knows everything across the company. I'm really shocked how much detail he understands from capacity allocation to big conference, all the topics, all the technical details, and all the kind of operation direction where it's going. and because of that, he can make the best judgment for the company. Obviously, that is based on a huge amount of personal capacity of operating that way. It's very unique. It's not for everybody. But I will say with AI, this type of operation will be more and more common across a broader set of companies because information flow is going to be much easier. And we do not need a deeply hierarchical organization to operate.

6:35And as the information flow much smoother, using all sorts of AI tools, and people have access to all sorts of data. And then with good judgment, it enables a lot more leaders. They can operate easily. So this is how I think about our company as well. Our company is pretty flat. We do not like to introduce deep hierarchy because it just slows down the information flow, slows down a lot of decision making, add redundancy in reporting communication. And we as a startup, we thrive in moving extremely fast and making very decisive decisions with quick information collection and access to what's going on and all the context.

7:25So I think that's extremely unique part of NVIDIA. But I think a lot of companies are starting to operate in a similar way. You mentioned that human judgment. You know, I've had a lot of founders and heads of AI from the leading companies on the show recently, who all mentioned this thing around human judgment taste. What does human judgment actually mean when you're running a company of the size of the likes of fireworks, do you think? So we actually are a very small company, given where we are. End of Q1, we're 150 people. End of Q2, we're 300 people. So we're doubling every quarter. We're growing very fast.

8:04But still, compared to where the business is, we're quite small. So one of the kind of philosophy is we really care about the quality of people. I interview everyone before they join Fireworks. and now it's a company scaling much bigger. We do have leaders kind of managing all different areas of recruiting and they are spending a lot of personal time to make sure everyone we hire into a company is the highest quality. And here, quality means different things for a different company. As founders, I really value the fire of founder in the candidates, in every single person joining us because I really believe people choose to join fireworks.

8:49is not because they want to just be an employee in a company. They really want to build a company with me together. And we view them as every single one of them as a pillar in the company growth, in the company operation, in where we're pushing the boundaries, in where we bring innovation, creative ideas and seeing through things end to end. So every single one of us in the company are taking a huge amount of responsibility. And one of the key tenets of our culture is extreme ownership. So extreme ownership is actually a military term for Navy sail. In the Army, it's extremely hierarchical. And no one crossed the boundary.

9:37You stick in your line. And that's how Army operate. But in a battlefield, it changes because it's life and death situation. and the thing here is it's important for the Navy SEAL leaders to watch each other's back. And here at Boundary, it doesn't mean anything. If it's a life and death situation, you just go ahead and take over responsibility and save a life, right? And make the call and watch each other's back. So there's a lot of similarity to how startup operate is no problem is anyone else's problem. we are actually power a company growing to the next phase and we really value what really matters is is seeing is kind of calling out hey we have a gap here and not just calling it out and actually seeing it through and fix this problem it doesn't matter who wrote the code it doesn't matter who are the primary person it's kind of more important to solve the problem than just wait and think about someone else is going to solve the problem.

10:44And this is not unique to startups. I think I learned a lot from the company I worked last, in the previous gig, from Meta. I learned a term from Meta, how to kind of think about ownership. And I've seen tremendous amount of results operating with extreme ownership. But that's kind of one of the key things we value from people joining us. I totally hear you on the ownership and also getting involved in the hiring, but also respectively, there is only so many people you can spend time with and hire until the organization becomes so big. At one point, you stop interviewing. Is there 450 people, 600 people?

11:28Because you can only do so much. First of all, I started a company with six other co-founders. So we're a very big funding team. And they are, every single one of them our pillar operating company right now. And along the way, I brought in great leaders across the industry and they are building a pillar, a bigger pillar of the company under their realm. So for example, recently I brought in George Hu. He was a legendary operator in Silicon Valley. He worked with Mark Benioff at Salesforce, a Salesforce COO and COO at Trilio as well. So he is driving, like building the whole entire GTM engine for fireworks.

12:13And that part is growing extremely fast. And he is managing all the hiring for the GTM side. So I don't need to spend too much time there. And recently I brought in Dwork, Roger Gopal. So he is also a very experienced technical cooperator that has been working on almost all stacks in AI. And so he came in and took over a lot of deep hiring work on the edge side. So I will continue to hire researchers, our leadership team, and the product team in various different areas. So I think obviously it doesn't make sense for the entire company to report to me. So I do have very strong leaders I'm working very closely with to expand the company.

13:00And those people are also carrying the same value and the cultural norm that will help us build a consistent company over time. You mentioned earlier around you at Meta. Why did you want to join such a large company? And then later, obviously some years later, go and find, found your own company. You know, building a company is so hard. You know, when you're working at a big corporate, you have a nice comfy lifestyle, you know, a good salary. And then you want to go into the trenches and actually go and build. Why do you want to go and build your own company? I didn't join Meta for the stability of a big company.

13:40I joined Meta to learn. At that time, it's still called Facebook. That's 2015. I want to join a company. I can learn how to build a startup. I want to look at a company that is at a later stage, but still with a startup culture, for me to absorb what this managing and high people talent density look like. So that's my motivation to join Meta, and I did learn a lot. I would say at that time, it is still in hyper-growth mode. Everything is quite chaotic. It is not that structured. I think when I joined, there's already 2 ,000 people. but the sense of ownership was off the chart. I was shocked. I completely don't have the expectation of anything surprising me.

14:34I know I'm going to go there and learn, but I'm like, maybe it's not that different from my prior companies, but I'm going to go there and see. But I was shocked. The most important thing I learned is the sense of ownership. Everyone fell deeply in love with the company. They really treat the company as their own. And that just unleashed superpower within everyone. So I think these carry on into fireworks is I deeply believe everyone who choose to join fireworks at our current stage has a fire of a founder in their heart. they are not just adventurers they're not just seeking for challenges or end opportunities they are ambitious to manage uncertainty and pave find a walk an unpaved path to excellence so that's audacity that's ambition that is courage but also it requires perseverance experience for us to kind of get to continue to innovate and penetrate through blockers and always find it.

16:00I talked to the team. I describe our spirit like water. It doesn't matter how many rocks are there along the way. We always flow through around the rock and keep moving forward. Nothing will block us. And so that's the spirit where I believe every single one of us, like joining this company, joining Fireworks, is caring. And that's the part I deeply appreciate and feel very exciting waking up every morning, working with them, solving all sorts of problems. So that's my personal style is I'm not a leader that just sits on my seat, receiving reports, giving, just giving high level instructions and directions and the chill.

16:47I'm an operator. My job is solving all sorts of problems within the company. And I solve problems very quickly. And there are a lot of interesting things we can talk about as a founder. And I'm pretty sure this audience has a lot of founders that is solving all kinds of challenges. Happy to kind of pick an area you're interested in diving into. I love that company philosophy around we're all in this together and whatever the challenge is, we're going to get through this and we're going to absolutely win and hopefully own it right. But I also think building a company is all about the founders. You have six fellow co-founders, which I think is a pretty incredible, but also quite unusual because most companies only have maybe two, three co-founders at most.

17:32What is Fireworks all about? We should touch upon that. and the second part of the question is around how do you manage six incredible co-founding relationships at the same time? Yeah, so I would say I'm extremely lucky to be able to start this company with six of them and each one of them are incredibly brilliant and at the same time we are different. We're not all the same. So it's not like we're echo chamber of each other. That's why we get along. The interesting thing is we all have different perspectives. And one thing that really bound us together is we're extremely intellectually honest.

18:19So that defines us is we're brutally intellectually honest with each other. and we rationalize company's strategic decision or operational steps through logic. And because we're all engineering background, it's very easy. So that's our common language. But also at the same time, it's very important to have different perspectives so we have a much broader range of design space to rationalize and for us to make the best decision. And oftentimes when we're sitting in a room, we do a lot of pre-modern. We do a lot of post-modern. We do a lot of reflection together. We constantly discuss. So one of the challenge, I would say, the biggest contrast between how we operate at hyperscale, we all came from hyperscale.

19:12So that's kind of one interesting thing. I can talk more about the funny conversation I had with my first investor. We all came from hyperscale. The biggest difference is at Hyperscaler, there's abundance of data, abundance of product information that you can make extremely precise calculation and decision of the outcome of a particular path. Operating a startup, we have the opposite problem. We do not have abundance of the data. Because if it's that obvious, if we have a lot of data to analyze, anyone can do it. There's not a gig for startups. So startup is basically pave a path where there's no path.

19:59And obviously there's no data. So we have to operate a lot with strategic hypothesis and assumptions to build a map in front of us. And that's where we are brutally honest with each other. and critique different ideas and make a decision. So usually we go very broad and we shrink down. We cut all the branches and shrink down. And we are not fixated with a particular decision we have made. There's no ego. And when new data comes in, we will validate the assumption we made and be honest with each other whether we made the right decision or whether we should continue to kind of iterate and adjust and tweak the direction we're heading towards.

20:47So that feedback loop and that trust in holding each other accountable and truth-seeking, is the connective tissue bonding us together? Is the deep respect we have to each other to operate together for four years and we are going very strong into the next 10 years? So, yeah, I think that's a very unique setting. I understand the question you're asking because when I raised the first round talking with Eric Richwar, he's the partner from Benchmark. When we first met, he was surprised. I have so many co-founders. He asked me, are you sure you're going to start a company with seven of you? That's unusual.

21:36So obviously the subtext is, you know, usually those founders don't get along and we'll have all kind of drama. And there's so many cases that companies split because of founder dynamics. But I think we are just an unusual collection of people. We are each other's strength. And we back into each other and create a much bigger force than every individual one of us. How do you manage conflicts or disagreements when there are seven of you? You know, if you and I are on a team and we've got five others, I love something, you don't love something. How do you come to a common ground where you then all finally agree?

22:20Usually when there are different opinions, it's not black and white. It is all about trade-offs. So we would like to put different opinions on the board and analyzing what are the underlying trade-offs we're talking about to leading to these different opinions. And the trade-off has to be based on what is more important for the company to grow, what is more important for the product to be more loved by our customers, which customer segment is more important. and those kind of goes back to the foundation of product decisions. And once we dissect, bisect that, then it's very easy to rationalize it.

23:05So that's the beauty of working with a group of people who are very, very rational. And we basically put everything on the table. So those trade-off analysis usually leading to, hey, we don't have data for certain assumptions. And then it becomes simple. Not making a decision is not an option. not making a decision is a bad decision. So I will make a decision. When there's no clear data to a hypothesis, we can support one another. Then I'm going to pick one based on my intuition and the taste and move forward. So that's kind of, but we will constantly reflect. So that's kind of the extremely strong muscle of the team.

23:51we'll constantly reflect and decide whether we should think differently. So, yeah, so we have a lot of kind of stories along the way that we have changed the initial decision and pick one which we'll understand better. So I think a lot of startup founders would have to go through similar exercise as they seek bigger and bigger goals. as they grow the company. I think that nature shouldn't be a foreign concept to all the founders. If you're enjoying today's episode, hit subscribe at the bottom of your screen. The guests on the show are getting bigger and bigger and we need your help to grow this channel.

24:36So if you're enjoying today's guests or any previous episodes, hit subscribe and I promise you it'll be worth it. Now back to today's episode. So let's talk about fireworks. we're going through this incredible inflection point. I think 25 was all about the year of code. I think this year is all about the year of co-work. We should talk around, you know, what does Fireworks actually do? And what is the current state of the company? Yeah, so last year is absolutely, so it's interesting. We are observing the whole entire industry building on top of our platform. So that's one of the most fun parts of my job is to see the creativity and innovation going off the chart from all different applications.

25:28So you're right. We can see the whole market. And last year is the year of coding. Coding is the biggest application, most successful segment last year. And now all coding companies are running on fireworks. We started joining with Cursor. We started to work with Cursor when they were 2 million AR. Now it's a thousand times over the past three years. I'm so thrilled being in their journey of hyper growth. I think coding is a fundamental building block of all sorts of software development. Right. And it is a foundational skill that AI brought. a foundation skill that AI brought in to automate. And based on coding, the next step is knowledge worker, because knowledge worker use a lot of digital assets.

26:24And those digital assets are all sorts of software to help them being productive. And because coding liberates the software development, and then the next step is the software used by knowledge workers are going through the next generation of innovation. And many of those knowledge workspace, another interesting thing is coding is more kind of universal or more consolidated, but knowledge work is very diversified. Every profession has a unique depth of their own domain. If you look at dentists, how many different dental professionals are there? So it's kind of different categories. It's an enormous number.

27:12So you can think about knowledge worker as a very bushy tree, where coding is the trunk, right? So it's just kind of software engineers. Most likely there are very few categories of differences, but knowledge work is a very broad bushy tree. And then each of the tip of the bushy tree is, you can imagine one AI native startups. They are trying to kind of automate. They are trying to kind of build smarter agents, assistants to make a particular professional domain so much more productive, so much more creative and innovative. So naturally, this year, we start to see huge diversification of all sorts of application that comes into picture all the way from legal.

28:05But by the way, I start to learn legal has so many things. Legal has, you know, defender side, defense, offense, and legal research somewhere in the middle. So it's very broad. Finance is a very broad space. Recruiting, all sorts of recruiting, marketing, sales, customer support, cybersecurity, manufacturing, you name it, right? So supply chain management. There's so many, all sorts of agents being built. And one interesting thing is it's not just the startups. there is significant movement from the enterprise side as well you can think about many of those enterprise they were startups a decade ago two decades ago few decades ago and they're the winner from from their the competition at that time and they deeply understand this wave is a defining wave for their future.

29:01They are all top down from board level, investing in AI to compete with the rising startups because they have seen those movies happening all the time. If you are not staying on top of your gig, then this is time to change the landscape. So the startups and enterprise, they're all moving. But then they face two big existential challenges. One challenge is lack of control of their IP. The other challenge is control of their cost. So control IP is because of multiple megatrend. One is product development has been significantly disrupted software product development. Because of CodingAgent, it cannot be easier.

29:57In the past, product itself is a moat because it required creative idea and ability to implement it with top-notch product engineers and PMs over multiple quarters. That is considered to be fast and the scale in production. Now with CodingAgent, without knowing how to write a single line of code, is a good visionary person with great ideas can launch their product within a few weeks and reach huge scale. And that collapsing of resource required and timeline required is completely shifting the competitive dynamics in the product space. In addition, products cannot be easier to copy because even if people don't have original idea, it's now kind of through the coding agency very easy to replicate an idea.

30:51So then you will say, imagine a product built on top of, which is very easy to replicate, built on top of a black box API, that combo, that mode is very, very thin. So then what is a real mode, right? So, and where is the IP? That goes back to first principle. we think about in the past 100 years, a recent 100 year. Capitalism thrives because, I don't know how many, we have more than a million companies across the whole globe, right? And those companies thrive because they solve a unique problem in a special way exceptionally well. And the diversification of specialty is the value. And that's kind of the, that's the foundation of how our current economy works.

31:44And I deeply believe in that. I do not believe in the future there will be one company across everything. I do not believe in that. Well, we can argue philosophically, there's these two future, one company versus millions or maybe tens of even, you know, a lot more diversified specialty companies. I believe in the other side, not one company. So let's continue this train of thought. Then each company, because they're solving a special problem exceptionally well, they will have unique product taste, judgment, and design. Unique for anyone else. And now product development becomes so easy. And their uniqueness may not be that unique.

32:31And what makes them continue to be unique is the data. they have accumulated from the power surface area interacting with the customer. And that's where the customer's preferences, judgment, and feedback is being collected. That is their moat. And today, those are the private data that will never get shared with any other company because this is their alpha. But interestingly, if we take a big step back, that alpha, that extremely valuable data is not being activated to create any intelligence. We believe the future of the intelligence lies inside of those deep pockets of product development. And that's what we call specialized intelligence.

33:23And we believe every product team, app, enterprise companies, they should own that critical supply chain of intelligence of their own, not depending on other companies because then your motive is going to be very, very thin. So that goes back to control of their IP. The control of their IP is to turn their private data into their customized model and continuously evolve the model together with a product. Second is control of cost. As we all know, starting from this year, it is a well-established sentiment across the industry. this AI is going to, you know, a lot of companies will build AI into bankruptcy.

34:04Okay. So this is a very interesting phenomenon as it was never existed. This concern was never even a topic to discuss in the SaaS era because product market fit and durable business almost means one thing. Once you hit a product market fit in the SaaS era, you just scale as fast as possible. because the business operation, the cogs of business operation is quite low. It is built on top of commoditized CPU and storage. And then probably the biggest cog is people, right? So during such era, it's one concept, but in the AI era, these are two concepts. Product market fit would mean your ICP just loves your product and they're willing, they want to use it day to day, day and the night and pay money to you and they want to keep going.

35:02But it doesn't mean you have a doable business because your costs of operating using AI could be much bigger than your income. And then when you scale, you literally scale into bankruptcy. This is a real problem for startups because startups have to raise money, use their equity. And once they burn all of their money, and they cannot grow their business fast enough, they cannot raise the next round, then they're pretty much dead. But this is an even bigger problem with enterprises who are the winner from the past decade or two. As they grow a startup, they won the war of consumer, pro-sumer developers.

35:47They have a huge amount of traffic running on their existing product. And when they roll out AI features, they're going to roll out AI features to all their customers and they cannot afford that. Their CFO is literally going to look at their cost forecasting and say, I'm sorry, I cannot approve this because we just cannot, it will destroy our book in the coming quarter or even multiple quarters down the road. So this is a real issue. It's the control of cost. So we came here to solve both issues through specialized intelligence. using your private data we can we can tune a highly intelligent model towards sota quality but with much better unit of economics here i'm talking about five to ten times cheaper in the cost side and and that means you for the same budget you can grow your custom base 10 times bigger or with the same size, you can reduce the cost 10 times lower.

36:52So this is while you don't send your offer out, you keep that deep mold within the company yourself. So I think it resonates with every company I talk to, with every company I talk to. The only question they have is how much it sounds like it's an investment. How should we do that? We don't have talent or I'm not sure how to think about this budget. And I am happy to talk more about that because there's a framework we use to think about that. Yeah, let's talk about that. What does that framework look like? Oh, so basically for the company, they want to own their own intelligence. They should think about that as their investment, right?

37:32And this customization training is not one time. You don't train a model once and use it forever because your product keeps evolving and improving. And the baseline model quality also gets better and better pretty fast. We launch new model every week, literally. So you want to continuously train this model as a continuous investment. But think about this investment under the umbrella of total cost of ownership. and think about you investing 10 % to 20 % of your overall AI spend into training and the remaining 80 % to 90 % as your investment in inference. Because of training that we're helping you customize your own model and your overall inference costs will reduce by 5 to 10 times and your overall total cost of ownership will still get significantly reduced even with the training investment by four to eight times.

38:34So that's how we rationalize this. And it's a very sound framework we've been using with many, many other companies. If we think about the opportunity today, the number of companies who are now trying to build their own models is only going to get bigger and bigger as the trust with these closed models potentially becomes a bit opaque. For the likes of Cursor, Harvey, Lagora, who are coming to you potentially and saying, hey, we would love to build our own open model, what does that process actually look like? And how long does it take to get a first version deployed? So because we are in an early technology adoption curve, the customer profile has a range.

39:21The range are the following in kind of a rough three buckets. One bucket is the likes of Cursor. They have deep researchers in-house, and they want to control every single knob to tweak and get the best quality. So we give them a very low-level API so they can change anything as they want and be productive. And oftentimes, they also operate with a very large fleet. And there we design our training system for GPU poor. GPU poor doesn't mean it requires less GPU to train. Oftentimes, the training job could start with tens of thousands of GPUs. It is a large fleet, but expecting these tens of thousands of GPUs to be in one cluster, RDMA interconnected, high-speed interconnect for big training run, is very expensive.

40:20Very expensive cluster. Also very hard to find. Good luck finding those clusters. And we designed our system to be able to run through a fully tiered disaggregated, regional, smaller cluster. We can assemble into a big virtual cluster to drive those big training runs. And that requires a lot of deep technical design because once it's a very large distribution system, then we introduce a lot of synchronization and kind of weight transformation that will introduce delay. And during training time, introduce delay means there's a tradeoff with quality, but we have innovation to really control and keep pushing the quality to the best.

41:07So this system can be used for all different customers, but we offer the lowest level of abstraction to the power users. And the second category are the many companies, existing companies, if they have more than 500 people, usually they have a machine learning team. And those teams are there even before JNI to drive machine learning algorithms or deep learning, even deep learning training job, but those are a much smaller model. their transition to manage JNN models. And they also want to control, have a lot of control over the training job but they don't want to have all the knobs. That's going to be too overwhelming.

41:54So we give them middle level abstraction that is our training SDK and they can use the training SDK to still control loss functions, control training algorithms, control how they want to drive, set up various different hyperparameters and monitor how things are going and decide how they want to explore and do experimentation. So, and the other category of customers is they do not have anyone yet in the company who can run training. So we will just, we will send our researchers to work with them, to pair with them and help them set up the initial run and bring that to a really good stage, along the way, we will educate and teach them how to do it themselves.

42:43Because we believe a strong partnership is to kind of really make them self-sufficient along the way as we help them solve the problem. And we believe, and those are extremely, extremely smart founders or product engineers, they pick up really quickly. So I think it's going to work because this is an area, a lot of cutting-edge companies, they can learn and do it themselves. So that's our third mode. And across the board, those different modes represent different level of product abstraction and different level of engagement for us. But the beauty is they stack on each other. We don't build different systems for different modes.

43:28It's all kind of built on top of each other and it's kind of fully streamlined. mind. So I'm very happy with that design. I think that that is a reflection of where the industry are and we always want to meet the customer where they are. I imagine there's an additional motor. If we see what's happened in the last few weeks, few months, we've had a bunch of security hacks. Maybe there's going to be a bunch of national security concerns at some point. And as a company who is helping all of these companies build their own AI models, I imagine one of the biggest questions these founders have to you is, how do we trust you?

44:09And how do we make sure our data and security is fully protected? This is really deeply rooted in our company operation. First of all, we really respect privacy. So our company operates with ZDR. ZDR means zero data retention for both training and inference. So that means data in and out gone. So if a customer self-serve training, we don't even see anything. The platform, they just directly use our platform and do their job. And we also deeply understand a training process, a lot of data. So you can imagine model is the new form of database. It absorbs the data and the knowledge can be used in many different forms.

44:58So we have various different secured data access implemented, including customer can bring their own bucket of storage where they have for control. And we can redefine that storage with their approved authentication. It can be accessed through deep encryption. They can roll the encryption key. So anytime we will not have control of when and how we get access. they also need to manage it's not just data security working with us so internally different roles from their company will have different type of data access some will not have access to those data some will have read only access some will have write access and we give them role based access control that they can manage data themselves so those are areas we really care about and that is the area also build deep trust between us and our customer.

45:58You know, we've talked about modes. So when all of these companies have good taste, good judgment, they have their own AI models, which is what we've been talking about, and they have security, what is going to be the next mode companies should be thinking about to stay ahead and to stay relevant? I think everything boils down to data. So here's my extremely simplistic view. I think product is a vehicle for deeper customer understanding. So here, I think product as is is not the moat. Product as is is a vehicle to start customer engagement. And that customer feedback and deeper understanding is the moat.

46:48So once you get the wheel rolling, then you keep rolling faster and faster. And then that understanding to turn into flying wheel, right? That flying wheel is better product design. It's better intelligence to power the product. So I do see a trend that's happening in the industry right now is all this innovative product company, they are evolving to be a hybrid. the product managers or product engineers, they're also researchers training models. And the researchers also carry, they naturally think about product. These two are becoming one thing. So for example, whether you train a model or not, every product company, if they use AI, they need to write evaluation, right?

47:38Evaluation is judgment. And evaluation is needed if you want to switch models. Just kind of use API as is. You want to switch from GPT models, from cloud models to all sorts of open models or your own tune models. Even for you to make that judgment call, which models to switch to, should you catch up with a new release or not, you need those evals. And typically people think about researchers creating evals, but that's actually product judgment. It's a product team create evals. And once you have evals, then you can start the training and you can start the prompt tuning. You can start the context tuning.

48:21You can start a lot of things. But we see the companies that move the fastest are the companies who invest in evolving people to be hybrid. Or they hire people who can be the hybrid. And we start to see a convergence of product people also doing research. So that's kind of a new trend. But I'm happily surprised. I've seen so many people who are capable of doing both. That is amazing. When I think about the huge opportunity for Fireworks, you're providing all of these incredible tech companies with their own infrastructure and with all of their models. But then when you think about the GPUs and data centers, is this potentially a market where Fireworks would enter at some point?

49:14I think I will refer back to one thing that Jensen mentioned. I don't remember the exact framing, but the idea is to do the minimal thing that we are best at doing. So I would say, I don't think Fireworks will be the best building data centers. We can. We can do anything if needed. But we are really, I think we're the best building specialized intelligence platform. And we want to double down on that. So in that regard, we would love to partner with companies who are the best building data centers, who are the best operating GPU fleet. And there could be an intersection point as we work a lot with hyperscalers.

50:00We work a lot with new clouds. we start to kind of give each other feedback. We start to kind of influence their roadmap. We start to be more engaged in helping them design things and design kind of very robust operational cadence of thinking through how to put pieces together. So we start to get more involved because we want them to thrive. As our partner, we want to teach them, like from our vintage point of view, what does good quality of service look like? And if they're successful, we'll be successful. So we were more thinking about a co-development mode where we double down on the strength from both sides and really provide a multiplicative outcome.

50:43So that's our current thinking. And I really respect a lot of very creative, innovative founders building new clouds these days. And they're really outstanding ones. You know, when we also look at the opportunity between closed and open models, if these enterprise customers are preferring to partner with the likes of fireworks with open models versus the likes of anthropic and open AI with closed models, where does the value lie with these closed models if companies preferring to partner with people like you on the open model side? I think it will coexist. So first of all, I deeply admire these two companies, OpenAI and Anthropik.

51:30They are brilliant companies, but we are executing on two different strategies. They're executing on the general intelligence strategy. We execute on specialized intelligence strategy. I think both strategies will coexist into the future. but I do not believe the world will be dominated by a few models, a few generalized intelligence models. I think the world will be millions of specialized intelligence models, continuous updating and improving, owned by every single company and every single use cases. So again, I think the generalized intelligence will more evolve into become a utility and it will be the baseline to power the future economics.

52:14But we are building into the essence of capitalism where diversification of specialization is the back rock of the next phase of economics built out. And we want to give everyone the tool to own their own intelligence. You know, one of the best things about doing a podcast show is A, you get to speak to the most incredible AI leaders, but you also get to learn what a week is actually like with them. So one of the questions that we love to ask on our show, if I was to join you as chief of staff for a week, what would that week look like and what would we be doing together? Actually, this is not just about joining a chief of staff.

52:57For all the leaders joining me, initial few weeks, they're going to shadow me. So basically, they're going to go to the meetings I go to, and here's how I spend time. I spend time meeting customers. I spend time recruiting. I spend time solving specific problems. For example, I will solve problems of, hey, here's a recruiting bottleneck, and here's how we're going to restructuring the flow of recruiting and approach of recruiting. So we got to kind of be efficient and still tap into the highest talent. They're going to spend time with me to drive capacity. There are a lot of capacity prioritization discussion and capacity procurement to future planning growth discussion.

53:48They will join me on product discussions in terms of product parity, people who we're going to have to accelerate. What part of product, part of reflection, is this experiment successful or not? Are we going to double down or are we going to shut it down? It will be a wide range of strategic, operational, even tactic things. A lot of problem solving we're going to do together. You know, it would be silly no longer to ask, but how do you actually use AI personally on a daily basis? So first of all, my whole entire company is AI-peeled. And the whole entire company actually built on top of our own stack.

54:32So we have a product called Nexus. Nexus is an intelligent router. It's multiple models. We train also ourselves. It can route across different models. Our own model, the model we hosted in the Frontier Labs model to solve all sorts of problems from software engineering to GTN to finance, even legal. So that actually significantly increased the productivity across the whole entire team. And I myself also use that. And that is kind of one of my way to stay very close to every day what's happening. Because those air tools are very good at summarizing and distill the higher bits of what's going on.

55:22But more importantly, I would like to observe where are the bottlenecks. So remember, we all operate, I want the whole entire company to operate like water. Our job is flow around rocks. roadblocks. And my job is follow the company around the biggest roadblocks and keep pushing forward. So understanding where's the biggest roadblocks, and I'm pretty effective in unblocked through various different resources and vehicles and getting a very close tap to what is going on and it's very important. So that is very helpful. I think through AI tools, I got a very good understanding of what's happening across the company.

56:06And my leaders, my team across the board, they're also using their tools to kind of make themselves very productive. And the productivity shows, I think, you're processing 40 trillion tokens a day. If we were to come back in 16, 17 months, at the end of next year, how many tokens do you think you'll be processing a day? Oh, that's really hard. I think, so let's move forward a year. I think there are multiple factors. One is the demand is definitely, my prediction is the demand is going to go up. Two is the token efficiency will go up too. As in model preserving the same intelligence is going to be smaller.

56:49And the model to deliver a high quality result requires less thinking token. So the token volume to reach a good quality result is also going to reduce. So there are multiple compounding effect. I believe in a year, reaching 100 times more token is possible. So that's kind of where things are. Obviously, there are a lot of assumptions I make here, but that's through my observation of how the AI space is moving. It's moving so quickly. I'm super excited for Fireworks. What you guys have just done in the last three, three and a half years, I think is incredible. and hopefully you can come back in 12 months and share the latest, but really appreciate you for coming on the show and thank you so much.

57:36Thank you for having me.