Can Cosine really build the UK’s first sovereign AI model?

0:00Mike Butcher:Welcome to Path Founders with me, Mike Butcher. We cover the entrepreneurs behind the code, the capital that backs them and the consequences for the tech industry. Squeezed between the gigantic AI race being fought between the US, China and to a lesser extent, the EU, United Kingdom, once a powerhouse country inside the EU, faces a difficult choice. Prestrating itself before frontier AI companies from the US is not an edifying prospect, and thus it wants desperately to reduce its dependence on foreign AI infrastructure. But building sovereign AI means a little more than whacking an open source AI model onto some servers in the British Isles.

0:45Mike Butcher:UK startup Cosign is now setting out to train its AI, Lumen Sovereign, and has won the UK government's backing to do it. It's being developed from the ground up on the Isambard AI, one of Europe's most powerful supercomputers, which is very much built for this kind of training, which is based in Bristol, using compute power awarded by the UK government's $500 million sovereign AI programme. Cosign has also assembled a coalition of backers and customers who will help define the model security, governance and operational requirements. These include BT, Babcock, Lloyd's Banking Group, HSBC, LSEG, NatWest, BA Systems, PwC, Thales and other major British companies and institutions.

1:36Mike Butcher:That's quite a list, isn't it? Lumen Sovereign's initial applications include cybersecurity, defense, financial crime investigations, healthcare and administration and legal. Well, lots of legal stuff, I dare say. So joining me now is co-founder of Cosign, Yang Li, and he's the COO as well. Yang, thanks very much for joining Path Founders. Thanks for having me today, Mike. Well, first of all, perhaps you could give us a little bit of a background about how this project came about. Were you approached by the government and these institutions or did you do the approaching initially? Sure. So we approached the government.

2:20So we heard about the government's sovereign AI initiative. And, you know, as a small startup, we are always looking for more compute. You know, the hyperscalers have lots of compute, but it's very expensive. and oftentimes if you're using a NeoCloud provider, you don't get access to it because there's always in demand. And so we applied and we had to obviously go through multiple rounds of applications and discussions, but ultimately we were backed with it by the government sovereign AI program with essentially three things. But capital as part of a right to first refusal for our next round. Also compute.

2:57So we've been, as you mentioned, access to the Isambard supercomputer computer to train our next sovereign model and also contacts, right? So navigating the annals of the public sector and also helping with this new coalition that we've built and announced today.

3:12Mike Butcher:Right, I see. So you did a little bit of approaching, but was that a while ago or was that just after the announcement? No, it was a little while ago. It's probably three months ago, three, four months ago. It was when we began the process. I see. So when the whole Sovereign AI project was being kicked off before the public launch, I presume. Yes, absolutely. And in fact, we didn't get a chance to talk, but we were both at the launch event, you know, when we were talking to Liz Kendall. Right, I see. Okay, well, what exactly is Cosign going to be building with Lumen Sovereign? Can you sort of unpack that a bit for us?

3:52Sure. So we're building a frontier LLM. It's a mixture of experts model of around about 130, 1.35 trillion parameters. And what that puts us is within range. It's not quite the largest model that you have access to publicly at the moment, but puts us within range and frontier. But I guess what our USP is, is twofold, is that like Anthropic, we provide both the model that we have trained ourselves, but also the product layer. So for coding tools, we have a CLI, we have a desktop app, we have a web app for people to access and do more general work. So we provide both. But our USP here is that we can deploy the entire stack on-premises, fully air-gap.

4:36So there's no data going in or out. So for a very sensitive organization like BAE, what they can do is then they deploy it onto their own infrastructure. and then they can actually do the inference and continuous training and improvement on the model in-house without data ever going to a third party, whether that's a hyperscaler or an American provider. I see.

4:58Mike Butcher:So it's really going to be quite different from, say, the cloud AI providers that are out there at the moment. I think on functionality, it's going to be very similar. But I think for people who are concerned in terms of compliance, in terms of cost, you know, it's on-prem sort of AI models are becoming more in focus. Right. I think it's anyone who's been following the news can see the not quite panic, the concern about token costs accelerating. Right. Uber came out this month to say, you know, they blew their entire year's budget in a single quarter. The good thing about having an on-prem AI is that once you've invested the capital cost of installing GPUs, you're capped.

5:42It's a cost that can be amortized. And actually, at that point, the more you use the models, actually the cost per token decreases because it's a fixed cost. Secondly, obviously, the compliance. There are sectors or there are companies that will not feel comfortable with either using a cloud version or through sending their data to an API to be processed in a black box and then returned. You have data sovereignty issues for certain jurisdictions. And lastly, a lot of customers are just increasingly don't want to use open source models from a foreign country. They want to use models that they can actually trust in their own jurisdiction.

6:24Mike Butcher:Right. And does this mean that you're sort of more on a par with, say, Mistral in France, which is providing some on-premise solutions? Yeah, I think we are very similar in terms of our ethos. We obviously do things a little bit differently. We focus primarily in terms of building for specific use cases, which is why we are working with this new coalition, I think 15, don't quote me on this, UK PLCs. And for us, that's really important to say we want to build a model that is not just generally capable. So I would hypothesize that our models probably would never be good at writing poetry or generating video game assets.

7:08That's not what our customers want. They want assurance by design. They want transforming legacy code bases. They want to build edge computing smaller models so that in field it's very effective. And so I think for us, it's very much more focused in that sense.

7:26Mike Butcher:um and presumably all of these companies and institutions that you've brought together under this umbrella they'll be contributing the data is is the data going to be sort of shared is any of the any of the data at all going to be shared across uh these institutions to improve the model or is it very going to be very siloed across these companies i i think it's a so So if they share the data with us, it will be shared to train our model. But I think it depends on the organization and the level of comfort. So we are still working organization one by one to understand exactly what types of databases are shareable.

8:03Of course, I'm sure you understand defense are a little bit more concerned than certain other regulated industries. And so for us, it's case by case. But ultimately, the data that is going to be used will be for the benefit of the overall model.

8:16Mike Butcher:Right. Well, obviously, there's some clear advantages for things like defense where, you know, the data sovereignty is very, very much a front of mind. But we've also heard that models like Mythos from Anthropic are very much at the cutting edge of cybersecurity. Is there a danger that your solutions will be somewhat behind against the front when set against the frontier, other frontier models? Sure. I think as a startup, there's always a fear of a bigger player encroaching. I think there's a couple of reasons that allow me to sleep easy or easier. I have three children and so I'm not getting much sleep.

8:59Better, number one, it's the way that we deploy. So I think fundamentally, you know, there's nothing stopping Anthropik or OpenAI from deploying, you know, fully on-prem. They can send their weights, the model weights into it. They can send their IP into organization, but so far they've chosen not to and it's a business decision. So the way the customer profile that we're going after is somewhat a bit of an advantage for us. I think secondly, it's also just hugely expensive. I think when I read a report that say Cloudflare released that they were using Mythos to scan from bugs, it cost them a million US dollars per scan.

9:42That's on a daily basis to continue and actually more efficient, specialized models that you own, that you can customize and continues to have a good relationship with to own. In my view, sovereignty is not just on a national level. Sovereignty is on a corporate level. You are in some way, I always define to our customers to say, don't think of it as sovereignty, think of it as control. Control over your own destiny, control of what data goes in, control of how data is processed and ultimately control over cost. Now, if you wish to scale any of these up or down or take more risk, that is within your grasp rather than outsourcing that and feeding into a black box.

10:21Mike Butcher:Well, I mean, it all sounds very patriotic that you're doing this for the UK. Is there not a danger for your company, though, that you get locked into the shores of the British Isles and it's going to have an effect on your ability to expand internationally. So yeah, I think we obviously are very proud to be British. All of our employees are in the UK. We set up here and we wanted to continue to build here. We think we can be a very large company, but it hasn't stopped us so far. In the last couple of months, we are talking to Swedish banks, we're talking to Polish cloud providers, we're talking to public service departments across the European Union.

11:06We're about to take a trip to the Middle East, where there are Middle Eastern governments inviting us to talk about building a sovereign solution for them, and whether we can fork the current UK solution, or if necessary, build one from scratch for them as well.

11:21Mike Butcher:Right, I see. So effectively, you're going out into the market saying, we're going to help you with your sovereign AI capability outside of the big US models, for instance, and you won't have to use Chinese models, presumably as well. Is that right? Yes. I think across the European environment or market, we can just use one model. I think the conversations we've been having, we don't need to retrain across Europe. But for further afield, definitely for the Middle East, the appetite is how do we build our own sovereign solution, you know, and have that, you know, resilience and also that sort of ability to control your own destiny.

12:00Mike Butcher:Well, it's a very interesting project. But while I've got you, I'd love your opinion on the debate around Palantir and whether or not you think that, you know, given that the UK is making this investment into you with Sovereign AI Fund, and these British institutions are going to be under your project, your new umbrella to create these sovereign AI capabilities. Do you think that perhaps you should maybe advise the government to ditch Palantir and get them out and take over from there? I think my view, obviously this is just my personal opinion, is I don't think Palantir is a bad choice for delivering a solution.

12:41I think what is important is to look at what power is their solution. What models are they using? How do they process that data? What do they do with that data down the line? Do they have unlimited access to it? Can they do what they want with it? I think that part of it during the procurement process, and ultimately the government handing out a contract is the devil is in the details there. Palantir is a fantastic software organization. They have proven that they can deploy software at scale and provide scalable solutions. But ultimately, it's, are they using a model you can trust for AI? Do they have assurances about the data, how it's processed, how it's used, you know, and also there is a safeguard in terms of will they build something off the know-how and the workflows that they learn from, for example, the NHS, right?

13:33I think that those controls are probably where I am most concerned about and the details are where I'm concerned about, not as whether Palantir should be. No, we have no issue about giving contracts to Atos and Talis and Tata Consulting and HCL. A lot of them, not all of them are UK sovereign GSIs. And if you view it in that lens, which I do, there's no problem fundamentally with Palantir. It's actually how do we maintain that control and resilience in the long run for ourselves.

14:08Mike Butcher:Sure, although some of the companies you just mentioned aren't nearly so heavily associated with the military applications. Yeah, I think it's not directly, but I think that there's a large swave of subcontractors that are definitely within the military or defense space. Well, that argument I'm sure will run and run. Last question, you're going to be using the Isambard AI compute, which I think is with the University of West England. Is that right? Yeah. So, I mean, that's obviously UK supercomputer. Do you think that's going to be sufficient for your needs? I think for this next model that we're training, Lumen Sovereign, yes, it's fit for purpose and it's adequate.

14:55I think in the future, we definitely need to scale our computers as well as we start training larger models, models with more active parameters, and also more further customized, differentiated models as well.

15:09Mike Butcher:Well, thank you very much for unpacking that announcement for us. Yang Li, co-founder and CEO of Cosign AI. Thanks for coming on PathFounders. Thanks, Mike, for having me.