An Interview with Danyal Akarca, Co-founder at Callosum & Suraj Bramhavar, Programme Director at Aria

0:00Suraj Bramhavar:Yeah, great. Maybe, thanks for having me, by the way, and maybe to start, it probably will help if I give a little bit of background for those who are unfamiliar with ARIA and the Scale and Compute Program historically. So ARIA was kind of designed to unlock these scientific breakthroughs that wouldn't happen otherwise, and it's a government-funded science agency. Um, and the scaling compute program in particular, which I was running, uh, or I'm running, I, we started about two and a half years ago with just with the general notion that, you know, there's this existence proof that intelligent systems can operate much, much more efficiently than the way most modern AI hardware operates.

0:58Suraj Bramhavar:And the vast majority of industrial resources are being spent marching down a well-known set of train tracks. So we launched this program called Scaling Compute two years ago maybe, and maybe over two years ago, that its primary purpose was to unlock alternative pathways. We wanted to give people an excuse to explore alternatives. And we felt that there was a vast space of alternatives that could be explored. And we found 11 compelling projects along with a whole bunch of seed projects that we thought were interesting. And we launched that. I think we kicked off work like a year and a half ago.

1:48Suraj Bramhavar:But in the time since, it's become very, very clear that at the idea stage, there's actually a ton of private capital and risk appetite to unlock a lot of these alternatives. And we've seen just in the last year or so, countless companies popping up with hundreds of millions in funding for AI hardware. and some of the ideas have even kind of longer time horizons and riskier ideas than some of the things that ARIA funded. And so one of the things that we kind of asked ourselves is, what is there in the ecosystem that with a 50 million pound program funded by the government, what is the thing that we could be best suited to unlock given the appetite in the private markets.

2:49Suraj Bramhavar:And the thing that became clear was that, you know, it's not necessarily just getting the ideas out from the idea stage into a prototype. That's one step of the process. There is still this massive gap between that and actually reaching global scale of implementation. And so we're targeting this next phase, which is called the Scaling Inference Lab, to attack this exact problem. That's on the institutional side. And then on the technological side, the idea is based on the fact that inference going forward is most likely going to be... Right now, the industry, the way it operates, it's mostly kind of scaling up homogenous, closed, very heavily engineered compute systems, right?

3:47Suraj Bramhavar:You hear all the talk about nuclear power plants and, you know, gigawatt data centers and all this stuff. And really what they're saying is like, we have a known thing that works. we need to figure out how to scale it up as much as possible that means like a lot of energy liquid cooling a lot of very expensive gear in those racks and it's usually the same gear that's stamped out over and over and over again but we're pretty sure that the future is not going to look like that those data centers are going to be filled with more heterogeneous equipment and different models are going to run on different hardware.

4:29Suraj Bramhavar:And what is missing in the ecosystem is a playground, a place for all of the different ideas to come and be tested. And so ARIA is kind of moving forward to create that testbed and trying to design it in such a way that invites as many of those ideas at the prototype stage to come into the fray as possible. And that philosophy aligns almost exactly with the philosophy of Daniel's company. And so Daniel came into the fray as an ARIA seed creator two years ago. And over time, he's kind of independently evolved in this other direction, which just happens to match exactly with where we think things are going.

5:23Suraj Bramhavar:And so it made sense to kind of partner pretty closely with his organization.

5:28Danyal Akarca:Before we go to you, Dan, I just want to like recap, clarify. So Serge, what you're saying is you're building this new lab where you're going to be essentially testing and connecting new hardware elements and aspects with the software that kind of plugs into the hardware. And the big bet that you're making is that the hardware stack that we have now, that as you say, is being scaled and rolled out everywhere, isn't the stack that you isn't the right one you think to be scaled globally to a thousand x you're making you and the team are making the bet that actually the hardware is going to change we're going to have different hardware needs for different bits of software and you're building this lab as a testing ground to match up different bits of hardware different bits of software

6:08Suraj Bramhavar:to see what happens yeah yeah and i i would say i wouldn't even call it a bet so much as to say that I think it's inevitable that this will happen. I mean, if you look through computer systems history, this has happened over and over and over again. You get these waves of technology where everything consolidates around very heavily engineered, closed computing systems. And then you get waves where that starts to get broken apart and the computing systems get more heterogeneous. There's space for more component providers. The interfaces get opened up. The costs drop precipitously as a result. And so this happened in the mainframe era.

6:57Suraj Bramhavar:It happened in the early internet era. It happened with the personal computer revolution. We think it's just going to happen again with AI systems. And we're positioning ourselves to kind of accelerate that evolution.

7:13Danyal Akarca:and right now you think we're sort of at that that time or that stage where we're leaving that first wave behind of the sort of like broad stroke single system to this much more open

7:24Suraj Bramhavar:and diverse range of different systems that's right yeah got it all right well done and and that and that pairs with the the technological gap uh that we know exists between like how much power and cost these things currently require yeah uh to to what we know there are alternatives

7:44Danyal Akarca:which is your which is your point why it's inevitable right because the current the current system the current setup is unable to scale at the level and speed that we needed to to make these things work um all right so i think now dan like you you tell us how you fit into this you've coming out of stealth today what are you building and where does this fit into the border picture. Yeah, totally. Thanks for having me, Seb. So just kind of piggybacking on what Seraj said there, it's important to realize here that it's not just kind of, you know, we can say that we'll go into a world of heterogeneous compute because this is the natural evolution, obviously, of how computing systems have been in the past, kind of from more kind of generalized architectures like CPUs to GPUs and then GPUs to what comes next.

8:23Danyal Akarca:So there's an inevitable trend. But it's important, I think, to maybe take a step back and realize why this is not only going to happen on the supply side, on the computing side, but what are the bottlenecks on the demand side? Why is this just an inevitably better paradigm to go into? Why is it that heterogeneous scale is better than a modular scale? And the reason of this is that the world is inherently very heterogeneous. The problems that we need AI to solve are not naturally solved by just one single type of intelligence. If you think about kind of your work, all of the really hard problems that we actually need AI to be able to solve, none of them typically are kind of single intelligence problems in putting it really simply.

9:05Danyal Akarca:They're typically very kind of complex, multi-step, quite uncertain in terms of the type of decisions that you have to be able to make. And so the world is very heterogeneous. And as a result, it's very amenable to many different types of intelligences. But it isn't just one. This is kind of a little bit different to what we thought probably around 2022 or so. So like around 2022 to 2023, I think there was a general belief that kind of a single model scaled up sufficiently would do really well. And it did. It did unbelievably well. Kind of the homogenous era was great for being able to build what I'd call kind of minimally viable intelligences rather than MVP and MVI, kind of a sufficiently good intelligence to solve many problems.

9:43Danyal Akarca:But now we're kind of going into the era where to solve the really hardest problems well at cost and sufficiently quick. So kind of cost, speed and performance metrics. We really are inevitably moving to a heterogeneous kind of multi-agent system or multi-agent intelligence paradigm where the benefits aren't naturally just coming from making single models better. That's always useful, but it's actually many different models. And in doing so, this is naturally amenable to many different hardware pieces that can accelerate different parts of that. So it's a little bit more of a shift in paradigm from what we call homogenous scale to heterogeneous scale.

10:15Danyal Akarca:And this is just because heterogeneity is ultimately a better way to build your computing systems to scale. And so at Colossum, and we've done lots of work before today in showing this to be the case. And so we've now released some of our earliest discoveries on this. When you actually treat problems as a system problem rather than a model problem, there's actually a huge surface area for optimization. So many of the capability gains that we'll get of the future, we expect to be from kind of systems of models and systems of intelligences interacting across mixed hardware. And we've proven this and shown lots of this and we're super excited for kind of what's to come.

10:51Danyal Akarca:And so Colossum is the company that effectively is building the infrastructure to make this a reality. And so working with Suraj, you can think of us as being the kind of software piece in this where we really are optimizing across that heterogeneity of the hardware. we have to kind of rethink a lot of ways in which we build AI systems and our software to work with heterogeneity. We'll probably get into that in a moment. And so super, super excited to kind of be working closely with Surajinaria and kind of, yeah, which would be the hardware platform ultimately to bring kind of the super exciting inference technologies and networking technologies to be able to bring them to market as well to actualize that vision.

11:29Danyal Akarca:Yeah, so I read in the press release that some of your systems are delivering like double as much accuracy, seven times fast performance, four times lower cost, I think it was. Can you give it a bit of flavor? What does that mean and how does that actually work? How is your system enabling to get benefits from different bits of hardware? Just give a beginner's explanation of how that's possible. Totally. So I use an example of, say, real-world problems. So real-world problems that aren't just, say, a chat interface. So real-world problems such as if you take a photo of your kitchen table and you want to say, I want that box of cereal over there.

12:06Danyal Akarca:I would like that kind of cereal, but I don't, I want it to taste similarly, but I don't want it to have as high sugar content. Can you please automatically purchase that for me? And then I'll have it tomorrow, please. Thank you. So that kind of problem. So I'd call that a more real world problem. It's quite a toy one. That's quite fun. But the point is, is that no single model alone is kind of, can do this kind of optimally. You know, we can actually make inroads in being able to do those problems, but because they're multi-step and multi-turn, there's many different, there's a lot of, how do I put it, reasoning alone does not actually solve that problem.

12:36Danyal Akarca:So despite us thinking a lot about reasoning and agents, it's super important, but reasoning alone actually can't solve those problems. The models actually have to interact with the world to solve them. So you can think of this as actually many, many, many different steps in the chain to actually solve those problems. And as a result, you can make an individual model better. So you'll get a lot of returns from that. That has diminishing returns. You know, we have to put in exponentially more money into the compute to make an individual model better. But where we see kind of the next bottleneck of performance, cost and speed is actually how you build those systems of agents.

13:08Danyal Akarca:So this is all the way from how you bring them together, how you allow them to communicate with each other. So there's lots of things that we've been doing on this and the communication between the models. And then it's how do you enable each of those models to be run across multiple different chips that are specialized for the task that they're actually doing. So this is a story of kind of specialized models, many types. It could be that they're the same model class. So it could be that they could all be Gemini, for example, but different versions, or a combination of open source and closed source, which is what we actually see happening.

13:41Danyal Akarca:And each of those processes can be kind of run on custom hardware that is ideal for this. And so you have different trade-offs of the different hardwares that are there. So what I'm basically saying is that the kind of surface area of what you can do is so vast that you can actually make it economically and economically viable to be able to implement these systems in practice. Now, we still have a lot to do. So, for example, we aren't kind of getting it to this point yet where we think that it's going to be easily adopted for the hardest problems right now. And we need to make hard renovations.

14:14Danyal Akarca:We need kind of new chips that are particularly good at certain processes. And we're seeing lots of great work and investment in this. But this is where we can get these cost, speed and performance gains. is really at the system level rather than just the individual model. Interesting. So you think about, in my head, I'm thinking about the way that it currently works is you might ask a model something, the model is predicated or reliant on a single bit of hardware. And I guess what your system is saying is we can speak to or connect a dozen different models that are all connected to different bits of hardware based on the specific use cases.

14:43Danyal Akarca:And so it opens up, I guess, the breadth of options and the breadth of combinations across software and hardware to create the optimum outcome. Exactly. And so the way I'd call this is a co-evolution process. There's a term in the technical term would be co-design. So people typically say hardware, software, co-design. This is where we know that the magic always happens when the algorithms and the hardware meet. So always the reason why transformers worked at all in the first place was because transformers played really well with GPUs and the highly paralysable and we could scale them up. And that just teaches us that all the magic happens kind of at the intersection of the algorithms and the hardware.

15:18Danyal Akarca:And so what I'm all I'm saying really is, is that we want to speed that up. So it's much more of an evolutionary process and it's much more dynamic. And we shouldn't just kind of build a single chip necessarily, although we can do that single chip for a single model. That's great. But it could be a little bit more of a dynamic process. And so it's this kind of co-evolution of both. There's a kind of a little segue on this is actually this is, I think Serge kind of alluded to it, an existence proof of intelligence. This is kind of how the brain has evolved as well. So the brain really evolved really without a clear distinction between the software and the hardware.

15:48Danyal Akarca:kind of our brains kind of exist both as the hardware and the software and inherently they're kind of co-evolved themselves and so you can think of Colossum really as doing the same type of process philosophically but for our digital silicon that we have today and also readying ourselves for silicon of the future as well which is super exciting. Got it and it looks like a lot of the bet that you are making is on the similarities or the parallels between the natural world and this world of AI and kind of hardware that we're building. I think it's also worth mentioned, I don't think we've actually mentioned it, that you're coming out of stealth today, having raised just over$10 million, which is amazing.

16:23Danyal Akarca:I think the round's been led by Plural. Plural is an amazing VC and their whole thesis is backing GDP altering companies. And it seems like that's definitely what you're building. So congratulations on that. How does it feel to get it done and get it out there? Thanks, Seb. Yeah. So yeah, great. Super, super happy. I mean, obviously we've been thinking about this, me and my co-founder Yasha, and we're working on these ideas for so long. I mean, we met during our PhDs at Cambridge, kind of at the dawn, really, of the scaling era of AI, and over kind of a lot of our work and thinking about this and meeting Suraj and really our ideas and thoughts maturing, we really kind of developed a real sense of urgency around the ambition of the mission that's required in order to, you know, what we're saying really, as Suraj alluded to, is that the real trajectory of computing itself can really be redefined here.

17:12Danyal Akarca:We're in a scenario in which we're not having to, you know, cloud systems and we're built for an era that wasn't actually multi-agent intelligence. And not only that kind of paradigm of multi-agent intelligence, but everything that comes after this period. And so, you know, we've been working super, super hard. We've published lots of our results that are super exciting. It will be releasing. And it's great to kind of be out there and to be able to more easily meet people and kind of show them what we're doing rather than kind of keeping it secrets and well not secret but kind of we've just been working so hard that we haven't actually released anything so now we're super excited to do that amazing and you're obviously sorry go ahead

17:48Suraj Bramhavar:yeah i may just jump on this to to cheerlead a little bit for for daniel uh and get some historical context so in every other era of computing there has always been uh a kind of systems level company that didn't necessarily, like, wasn't necessarily fully vertically integrated all the way down to the compute chip. So if you think about mainframe systems or personal computers or networking systems, inevitably there always emerge these companies that are kind of focused at the system level and not necessarily tied to building their own particular chips. And right now, if you look at AI systems, there are a bunch of systems companies, but they're all very, very tied to their own particular stack or their own particular chips, almost all the way up and down.

18:49Suraj Bramhavar:And when we talk about Daniel's company becoming this like trillion dollar company which i think they have the potential to become like this is the role in which i i would believe like this coming from like a totally impartial uh perspective i think that they could play in the future and it's a it's a very different narrative to the one

19:15Danyal Akarca:i think that we had maybe a couple of years ago which was the almost like a winner takes all market that we would probably see one dominant model come out of top one dominant chip provider maybe it was like OpenAI and NVIDIA. And I guess the bet that you're making now is that actually we're going to have a wide range of models, a wide range of hardware providers, but actually a lot of the value can be captured in the system or the software that sits between it all and connects it all up.

19:38Suraj Bramhavar:I think so, yeah.

19:40Danyal Akarca:Interesting. And look, we're obviously going to be working and have been working closely with Dan and Colosum. Can you talk a bit more about maybe another company or some other companies that maybe you've already selected or you're going to be working with on this program?

19:54Suraj Bramhavar:I'm not 100 % sure we're ready to talk about some of the other things. I will say, so Daniel's company in particular, we are very well aligned in what our end objectives are. And we're still working on ironing out the details of the other partners that we have. So I'm not 100 % sure we're ready to share. Suffice to say that the way it's structured is that we intend to build a cluster, a full AI system every six months in this lab, right? And each one of these is going to be structured as almost like a pilot project. so we intend to build functioning systems on six month cadences and pilot technology from a particular one or two companies per system and so we're mapping out this roadmap now and we have like two or three early partner selected but in short order we'll be able to share who they are hopefully and

21:03Danyal Akarca:what would you hope to see at the end of that six-month period.

21:08Suraj Bramhavar:Ultimately, what we're trying to show is that this, so there's two pieces to this. One is the institutional kind of innovation, right? It's a nonprofit. It's a neutral playing ground. We're trying to demonstrate that that neutral field is attractive to a wide variety of people, including big corporations who want to come see the results that spit out of this, or contribute their own technologies and startup companies who have their own kind of technology theses that they want to see proven. So there's an institutional component of that, just seeing that the kind of audience can be expanded, the audience of companies that we're engaged with.

21:57Suraj Bramhavar:Then there's a technological component, which is very clear. It's like, we're going to build these systems on a six-month cadence. We're going to pick particular technologies that we think are going to insert into our roadmap at that cadence. And we're going to want to show that the cost of these systems can drop iteratively at a rate that is much faster than what you are seeing in industry. And we will very clearly evaluate that. We'll know in a couple of years whether that's happening

22:28Danyal Akarca:or it's not interesting and across the ecosystem i think especially here in europe you know last year it felt like there was a lot of money going in a lot of excitement about ai companies on the application layer and it feels like already at the start of this year we're seeing a real change and a shift towards more money going towards like deep tech but like you know like frontier labs we've seen ineffable intelligence we've seen olex chip companies are you seeing in your position at ARIA, are you seeing more and more companies like that coming through from Europe? You know, there's like solving really hard problems at the deep level.

23:04Suraj Bramhavar:Yeah, I think it's great. All these funding announcements are amazing. I think there's probably more of them coming. And I will celebrate all of it. And in fact, like this pivot of ARIA's activity is designed in response to a lot of these announcements, right? We're noticing that there's capital available both on the algorithm side with companies like Ineffable Intelligence and on the hardware side with companies like Olex. And that's awesome. Now, the question is, how do we harness that? Like, there's a connective tissue that's still required between the hardware companies and the the algorithms companies and as daniel mentioned like the magic really happens when you can kind of co-design those two things and if you look at all of these people there's still a big gap like if you go to the companies receiving all of the funding for neo labs like frontier ai labs most likely they're all still using a homogenous compute stack.