Cosmic AI: Exploring Celestial's $100M Boost for AI Model Performance with Photonic Fabric

0:00What can 160 years of experience teach you about the future? When it comes to protecting what matters, Pacific Life provides life insurance, retirement income, and employee benefits for people and businesses building a more confident tomorrow. Strategies rooted in strength and backed by experience. Ask a financial professional how Pacific Life can help you today. Pacific Life Insurance Company, Omaha, Nebraska, and in New York. Pacific Life and Annuity, Phoenix, Arizona. Celestial AI has just raised a whopping$100 million for a very cool use case. So today on the podcast, we're going to be talking about what they are spending that$100 million on, what they do, and why I think this is such an important use case for the future.

0:43So the first thing to know is that essentially they're creating this cutting-edge technology that speeds up data transfer. and this massive funding round is all about the company's technology known as photonic fabric platform so essentially it is something that has been very interesting to a lot of big investors has caught their attention they had investors like a IAG capital partners coke disruptive technologies tesmax xor innovation fund samsung porsche the engine fund a lot of other big investors have been super interested in this and actually this isn't the first obviously money that they've raised this will bring up to 165 million dollars that they've raised total since the very early stages of funding till now so in basic terms celestial ai is working on a solution for a problem known as the memory wall challenge so this problem essentially comes up when you're trying to work with advanced ai models like you know gpt4 for example which power things like advanced chatbots and recommendation systems.

1:49So obviously something that is very prevalent in the industry today. So these models need a ton of memory capacity and speed, but this is actually very hard to achieve because of how computer memory and computing work together. So the current method of transferring data is electrical interconnects, and they have a lot of limitations. For example, you know, like lower bandwidth, higher delay times, and huge power usage, right? We've heard this a lot when people are training these AI models. It uses an insane amount of power. So these problems can slow down the advancements. And essentially, when you're trying to scale your AI technology, all of these things slow them down a lot.

2:31And so that's actually where Celestial AI comes in and the problem they're trying to solve. So Celestial AI has partnered with some top tier tech companies to develop their photonic fabric, which is essentially a special way of transferring data that's designed for large-scale computing and memory tasks. So Celestial AI says that this tech breaks down the limitations of how memory can be distributed in data centers, and basically it's enabling a lot faster and a lot more efficient computing. And I would say the main problem moving forward, according to Celestial AI CEO Dave Lovosky, is the memory capacity and data management for large language models and recommendation engines.

3:11So Photonic Fabric's advantage is that it can directly deliver data anywhere in the silicon die, which is like you can think of the silicon die essentially as like the brain of a computer, and where current solutions can only deliver data to the edge of the die. So they're delivering data to the edge of the die, and then it's getting pulled in, and there's a lot of lost, there's a lot of lost, speed, it has latency issues and whatnot. So this new photonic fabric, it can deliver the data directly to the center of the die or anywhere else on this silicon die. So this new technology has reportedly overcome one of the really, I mean, you could call it like a quote unquote beachfront problems, but essentially it's offering much higher bandwidth and lower delay times.

3:56And this lets the platform provide super fast links between different computing parts. So the recent funding has also got the attention of Broadcom, which is a leading tech company, which is now working with Celestial Eye on developing prototypes based on this tech. So their goal with these prototypes is to have them ready for customers within the next year and a half. So, you know, within the next 18 months or so. So as the amount of data being moved around data centers grows, this isn't, you know, this is obviously a trend we're seeing continue to grow. These AI models are just getting more popular.

4:29More companies are doing this kind of technology and kind of building these AI models out. And as this continues to grow, the data transfer speeds also need to rise. But as these speeds go up, current methods of data transfer, including, you know, electrical interconnects, they start having problems like signal loss and limited bandwidth that can't keep up with the data growth. So really this restricts the overall speed and just efficiency of the system. So with the photonic fabric's low latency or delay time in data transmission, more servers can be connected with traditional electrical interconnects.

5:03So the same low latency also allows latency-sensitive applications to use remote memory, which wasn't possible with older methods. So by using this new Photonic fabric, companies can actually better use their memory and computing resources without a hit on power latency and performance. So poor use of server memory can lead to massive waste of money. I've looked into this a lot. I've talked a lot with my different CTOs for different startups. but essentially if you don't have your your servers configured correctly there's all of these different arguments you have to make between the different types of servers and how you set that up but it can cost you a lot of money if you don't have those set up correctly so by making better use of your memory celestial ai's new tech can actually help to reduce a lot of the waste and improve memory utilization they believe that their new essentially the new product can deliver data from anywhere on the silicone to the point of computing.

5:59And the bandwidth provided is significantly greater. And the delay time is significantly lower than competing solutions. So I think this is really ideal for working with large AI models that can range from, you know, 100 billion to a trillion plus parameters, which is actually a trend I think we're seeing in the future and in this space with, you know, GPT-4 being rumored at being over a trillion parameters and Facebook themselves, or I guess Meta, has talked about a lot of different AI models they're building and how many parameters are in those. And they're saying, you know, they're orders of magnitude bigger than even GPT-4, allegedly is what Meta is saying, which would be, you know, trillions and trillions of parameters.

6:38So very, very large. By enabling each processor to store and process larger chunks of data, the photonic fabric reduces the number of processors needed and provides the fastest link between processors, allowing them to process much faster and really just increases the speed overall. I think it's going to be really interesting to continue to watch this space and see if new innovations like this out of Celestial AI make a really big impact because as they're making a lot of these improvements, they're essentially allowing us to build bigger, more powerful, more power consumptive models, which is going to be really, really big for some of the big use cases that people have in the future.

7:19So I think this is going to be really interesting. I think this is going to push the industry forward in a lot of ways that people aren't predicting right now. We're going to see some pretty massive improvements in AI in the coming years and even perhaps, you know, months because of this new technology that's coming to the forefront.