The U.S. leads AI. Why other countries want a backup plan

0:00Hello and welcome to The Tech Download. I'm Arjun Kharpal and on today's episode my guest talks about sovereign AI. The US has been carrying all of the weight of technology for the past quarter century. AI cyber threat. These models are the most potent cyber weapon that has ever been. And China's big progress. Do you think Western policymakers still underestimating the technical capabilities of China?

0:32Hello, I'm CNBC's senior technology correspondent, Arjun Karpal. My guest today is Aidan Gomez, CEO of Cohere, which is an enterprise-focused AI startup. But before he was a CEO, Gomez was one of the co-authors of the 2017 research paper, Attention is All You Need, better known as the Transformer paper. The key breakthrough was teaching AI how to pay attention to the most relevant pieces of information in a sentence or document at the same time, rather than processing information sequentially. That made models vastly more powerful, more scalable, and ultimately paved the way for the generative AI boom we see today.

1:10If modern AI has a founding document, many would argue it's that paper.

1:18Aiden, look, enterprise adoption is going to be very key to this entire AI story in terms of scaling, in terms of revenue generation, all different parts as well. And there's a lot of considerations from the enterprise when it comes to security of AI systems cost. But sovereignty is a word we continue to hear over and over again. I've heard multiple definitions of sovereignty as well. What does this mean in practice for you guys at Cohear? So for us, the way that we look at it is sovereignty is control. So control over the data in your system. You need to be able to know where it resides, ensure that it's not being exploited or used to train models for your competitors or for others.

1:59The second piece is control over the infrastructure. So where does the infrastructure actually live? Which entity controls that infrastructure? can foreign governments force that entity to give your your data to them or grant them backdoor access to the systems that are running on that infra and then the third piece is really this question of like can you get switched off can you have this system compromised in some way that sabotages whatever it powers. Those three things together encompass control and sovereignty over AI or really any software that's running some critical process within your economy.

2:46Is it less about that kind of longer supply chain and more about kind of some of the things you were mentioning there? It's up and down the stack. So the reality is like you can't do everything yourself, right? So you're going to buy parts of this stack. The question is, what does your supply chain for that layer of the stack look like? If you have a single line of dependence, if you buy all of your technology from one country or one entity, then you have a single point of failure. You've lost sovereignty because one player can just shut you off. If you have a diversified, resilient supply chain, then if one switches you off, you can just flip to another supplier.

3:26And so I think it's about diversifying supply chains to give yourself the maximum leverage. You can say no to things, you can choose a different direction, because no one party has all that leverage over you. The overarching theme, it appears, is control. And you mentioned sort of control of your data and making sure that your information isn't being used to train models. There was this interview that went viral with Alex Karp of Palantir on CNBC over the last few months. He alluded to the ideas that enterprises who are working with frontier labs are effectively handing over intellectual property and data to those frontier labs.

4:08Is that a genuine concern? Absolutely. I think it's more than just the frontier labs. it's anytime you're using this stuff via an API, anytime you're sending your data to a third party for them to process it and run AI on your behalf, you're vulnerabilizing yourself to this. They can see your data. And what's interesting is like, even if they give you the disclaimer, we're not going to train on it. What they can do is take your data and then create lookalike data and train on that and so it's a very dangerous thing like the whole model of cohere's product is the ability to deploy privately within your own infrastructure whether that's like in your vpc on a cloud or uh on-prem or even air-gapped right like in a disconnected submarine a kilometer under the surface of the ocean um that gives you hard guarantees that only you can access these systems You have full control.

5:05Cohere can't see in. No one can see in. No one can switch it off. No one can exfiltrate your data. No one can even observe it to create lookalike data. It's like fully, fully protected. And I think that's the threat of these APIs is that they don't provide really any control. Overnight, your access to those systems can just be switched off because it's a third-party controlled apparatus. And so what does that look like in practice for companies? Do they need to effectively build their own data centers at this point? It's an option, yeah. And it's an important option. It's a core part of digital sovereignty is controlling your own infrastructure.

5:45And I really do believe you should not just rely on hyperscalers from another country. You should have a domestic champion. That's something that I think every country can do. Most countries have telecom providers that build the critical infrastructure of telecommunications, the internet, etc. Most countries have electricity companies that build the grid, build that infrastructure, build power generation. The same is necessarily true of the digital world. You're going to need a data center provider. You're going to need a sovereign cloud, truly like national sovereign cloud and for cohere what we provide is the flexibility to deploy literally anywhere like on your sovereign cloud or if an individual company wants to build their own data center we're happy to deploy there that the important thing is that the economy the the companies within your economy your government they can choose where it goes instead of being forced to access via some third-party api so i mean because building data centers is an expensive endeavor.

6:51So that's not going to be an option for many companies, right? So that's where you're saying that actually, you know, countries or regions who are sort of aligned need their own data center builders within their own countries rather than sort of potentially foreign, mainly US-based at this point, hyperscaler kind of providers. Exactly. Yeah. You're going to have to build that capability within the nation. That's something that is like critical infrastructure to the functioning of your government, your economy, things like your financial system, that needs to be fully under your control. Water treatment, energy, the grid, all this stuff is like, if this goes down, if this gets shut off, your economy stops.

7:35Or if the grid gets attacked, people can't flush their toilets, like the water stops running, you know? So it's a national security issue. and people haven't taken it seriously. Over the past quarter century, there's been this, as like the digital revolution took off and we entered into this totally new phase of technology, we did not prioritize sovereignty. We all were very complacent and that's put us in this very vulnerable position. And so now there needs to be this effort to basically go and build up that capability again, strengthen our posture because the posture as it stands is quite weak.

8:16It's best in critical industries, like the ones that I was mentioning, like water treatment, financial services, telecommunications. It's best there, but it's still not great. There's still tons of reliance on foreign technology. And that's okay, like I said, if it's diversified, if you have multiple suppliers from multiple different places. But the issue is it's not right now. It's very concentrated. Do you feel like it's being taken seriously now, one? And two, are you seeing progress being made? Because again, I look at somewhere like Europe and what we've seen is fascinating in that there has been this sort of resurgence of startup activity and scale up activity where you're seeing startups grow very, very quickly and position themselves as kind of European champions.

9:04You look at a company like Mistral on the sort of AI model level, a company you acquired, Aleph Alpha as well. On the sort of NeoCloud side, there are companies emerging as well. But there is still a very, very strong position in the market for big US champions as well. There definitely is a level of attention being given to sovereignty in tech that I haven't seen before. Past couple of years have really woken everyone up. It feels like we're in a very rare moment in history, and this sort of thing only happens, I don't know, once in every few generations, something like that, where both a huge geopolitical change hits that we need to face and tackle, and an enormous technological shift happens at the same time.

9:58in those moments sovereignty is in many ways like the only thing that matters because leverage is going to be exploited there's all this change people are looking for opportunities to take to exploit leverage to gain ground and they'll exploit every lever that they have so the amount of prep work that you did going into that phase of making yourself stronger and more robust more resilient to those types of tactics, the better position you're in to come out the other side of all this change, strong and leading, and having gained ground rather than lost it. So I think people see, they're fully aware of these two big changes.

10:41It's like the only thing that dominates are headlines. And so there's now starting to be very strong momentum behind sovereign technology. and we're trying to contribute, right? Like I think the past 25 years have been, I grew up, you know, in Canada and Canada has played a huge part to the development of technology. Some of the best computer scientists in the world, you know, Jeffrey Hinton, Joshua Benjo, they won the Nobel Prize. And they basically like invented AI as we understand it today. And many of the technologies before, Blackberry and mobile innovation, that type of thing. And none of it succeeded.

11:25And I think the same is true of other countries as well. In Europe, all around the world, these developed democracies. The past quarter century, we've seen technology be developed somewhere else, be commercialized somewhere else. And we contribute to the innovation side of it. We invent it. Like we come up with the ideas and then we just end up buying that IP back from someone else. I think that's a really sad truth for us to face and one that we have to fight against. If we care about our sovereignty, if we care about existing as countries, we have to develop these capabilities ourselves. like social media, consumer electronics.

12:11These are the most potent cultural, geopolitical, economic institutions that exist on the face of the planet right now. And we have none of them. We have no players. And so we have to change that. I think it is one of the most significant threats to our national security that exists. What do you think the read-through is then to what is effectively a very U.S. dominant position in infrastructure around AI, in the models around AI? I'm glad it's a democracy and not the alternative, which would be an authoritarian regime in the lead. But what I would say is that we need to ensure democracies contribute at each layer of the stack capabilities to augment and diversify and ensure democratic leadership across the stack.

13:06because, yeah, it is too concentrated. Like, the US has been carrying all of the weight of technology for the past quarter century, and it's time that we stepped up, Canada, Europe, Australia, etc. It's time we stepped up to build our own champions and ensure that democracy has many very capable participants and contributors playing for it.

13:33Executive Decisions is the new podcast from CNBC. where I ask powerful leaders about their decisions that changed everything. I'm Steve Sedgwick, and here's the CEO of Siemens Energy, Christian Brueck. Business leaders should not stay quiet in a world which is super complex. We sit in a privileged position, and we have to use that to ensure that society remains prosperous, stable, successful. That's Executive Decisions with me, Steve Sedgwick. Get it wherever you're listening to this. The debate Aidan Gomez is describing comes down to a simple question. If AI becomes essential to a business or a country, who is ultimately in control?

14:12Today, many businesses use advanced AI through what's called an API or application programming interface. Put simply, it is the connection that allows one piece of software to send a request to an AI model run by another company, then receive an answer back. It is convenient. A business does not need to build or host the model itself, but it can also create dependence on the provider's infrastructure, terms of service, pricing and availability. And of course, depending on the setup, sensitive company data may be processed outside the organization's own systems. But the dependency piece of the equation is one of the most concerning for companies.

14:47If an organization, for example, relies too heavily on one country, one company or even a single model, what happens if access is suddenly denied? That concern is no longer theoretical. In June, Anthropic said it had received a US export control directive requiring it to suspend foreign nationals access to its fable-fied and mythos-fied models, including foreign national employees. I think this episode showed how access to a powerful AI system could be restricted by a country where its developer is based. That helps explain the push for AI sovereignty from countries and companies outside of the United States.

15:23The goal is not complete self-sufficiency. modern AI still relies on global chips, cloud infrastructure and big supply chains. It is about having alternatives, greater control over data and infrastructure and fewer single points of failure if access to a critical technology is ever restricted.

15:45Just want to get your take on going back to the enterprise as well. You speak to so many enterprise leaders. what are they telling you privately, I guess, about adoption of AI? Is it real? Is there strong desire for it? And I guess, what are some of the challenges that they're facing right now in adopting the technology? A lot of it is about loss of control, fear of risk, right? Like people using the technology in ways that they shouldn't or that the technology isn't ready for yet, the vulnerabilization of their data, their IP, a lot of these are like the core constraints. The governance of these systems is the thing that they're most concerned of.

16:30Governance in the sense of knowing where the data goes, ensuring that it's behaving in the way you want it to behave and it's not exposing the business to risk by making mistakes that a human wouldn't make. At the same time, the demand is actually pretty insatiable right like the employees want to use this technology they're consuming tons of it they're using it every single day like the adoption rates are all shockingly strong you know there's lots of negative press about AI and data centers that type of thing but when it comes to at least the companies that we work with and what we see inside their employee bases they use it every day they love it it's like a it's an incredible tool it's extremely, extremely useful.

17:16The blockers or the barriers to more adoption are mostly about protecting against these risks. Is the cost issue still an issue? It's been a big conversation this year with many enterprises talking about the fact that it's very expensive to run and use these frontier models in particular. How big a conversation is that right now? The big issue is that you have this kind of uncapped cost. You can have one employee who starts a job that consumes your entire budget for a year in a week. And this has happened. This is like something that is happening pretty regularly. And you've seen sort of uneconomic behavior like companies setting up internal leaderboards for who's using the most tokens, aka spending the most money.

18:07And that gets gamed, obviously, in really unproductive ways and lots of money gets burned. So you see that. And what Cohere has done on our deployment model is because we deploy privately, we have a fixed cost structure. So you know exactly a priori, the maximum you're going to pay. you're never going to have some sort of like shock uh around what you're you're being charged now if your instance that you're running on if your your system is overloaded because people want to use it more and you haven't allocated enough compute to support it we'll tell you and you can choose then at that point to pay more and expand the the deployment to support the the usage growth uh or you cannot you know like that that is fully within your control um but this is something that is like a concrete benefit aside from security on the cost side for these private deployments fully under your control can we talk about then like how you're approaching model development our audience will know from the start of this podcast aiden you were you know one of the authors of a very key paper the transformer paper in 2017 that was it underpins basically modern day AI right now and all the products that we use.

19:21The Frontier Labs and labs over in China and elsewhere appear to be engaged in a kind of arms race, right? They're throwing a lot of money at training incredibly expensive models so that they can be at the very, very leading edge. Those are the kind of models that are very expensive to run and that we've spoken about now. do you think that's a fundamentally flawed approach at this point in time for for what the world needs and and how are you approaching it differently when we talk to our customers they don't just buy one model right like they use our models they use other models there's usually like an ecosystem that they uh they adopt and consume and what we see is that they tend not to use the most expensive like the frontier category of model instead they use like a middle uh scale model way more cost effective does the job well enough that's like uh what they need to do what they they want to do um for our own model development we try to support exactly that pattern which is instead of going for like these ornate extremely expensive like behemoth models um we go for something that is right size to market so we try to find like the optimal point where the model is big enough that it is capable, can do the things that you want it to do at good enough accuracy to be valuable, high ROI, but small enough that it's actually efficient and scalable and can be deployed across all of your employees, hundreds of thousands, millions of employees.

20:56So that's the balance that we're trying to strike between those two priorities. I think increasingly it's proving to be the right strategy especially as some of the exuberance in the market is coming down and these cost controls are being implemented everyone is calling us asking for you know can we flip to your model that only fits on two gpus it's way more scalable I can deploy it to more people I finally have cost controls and I can understand what I'm going to be paying this year for AI because my CFO is freaking out. We just blew a year's budget in a week. And I think that's the point. And does anyone care about the models?

21:38They just want to see the result. Ultimately, what it's going to come down to when enterprises are looking at adopting this technology. Does the model matter? I would say yes, in two ways. The first is that they care a lot about the performance of the model, of course. Can it actually do the job at what accuracy but the second thing that is less spoken about is do you trust the model um so do you trust the model developer uh will this model behave the way i want it to in these deployments that are increasingly sensitive like before when they were like just augmentative when they were you know help me draft an email but still the human's like able to read that email and check it and make edits and then press send more and more you're asking these models, refactor my entire code base, right?

22:31And there's no way someone is reviewing like 100 ,000 lines of code by themselves to check each line that that model wrote. And if you don't trust that model, and that model very subtly introduced vulnerabilities, right? Like if it was built by a company that had the intent to introduce vulnerabilities into critical software, it would be very easy to code that in and have it subtly depend on, you know, depend on software that they know there's an exploit in. So instead of using the typical, like more trusted one, I'm going to use this one that's, you know, kind of less known. I know I have a vulnerability in that, but I'm always going to use that whenever I want to do this type of thing.

23:21That is super easy to do. And that is a huge supply chain risk for all of software. And so there is a, you are making a leap of faith that you trust this model developer, because increasingly that model is going to be doing so much of your infrastructure development, so much of your work inside the company. And if you can't trust them, it's like not being able to trust an employee. If that employee is like an operative of a competitor or a foreign nation, that's a huge threat and you need to be able to mitigate that risk we have good ways of doing that for human employees we need to make sure that as we adopt models we take that same sort of critical posture towards them there's a large portion of open weight models that are coming out of china with very very good performance um you know matching at some parts some of what the frontier labs out in the US are doing.

24:17We've seen open weight models out of Europe as well. Where does that fit into this? So whether the weights are open or closed, if the model was created in a way that intends to introduce exploits into software or intends to sabotage operations selectively, you can't tell in the open weights or closed weights version and once you deploy it it'll do that um so it doesn't really matter whether it's open or closed weights i think the thing that open weights does provide is you can deploy it on your infrastructure and so you can have more guarantees about privacy and control in that setting but if you don't trust the model developer it doesn't matter so then that was my point if even if you could run that in your own environment your own infrastructure that still doesn't in your view take away the the risks associated with?

25:15It's the same model. You're like, yes, it's running inside your infrastructure, etc. You have less data visibility risk because you're not sending your data to someone else, to some third party. But the risk of that model misbehaving or doing something to sabotage you completely still exists. It's just inside your infrastructure now, which some could argue is even more risky. One of the arguments is that the rise of open-weight models and the strength of these models is leading to effectively a commoditization of the model layer and poses a threat to some of the frontier labs as well in terms of these are cheaper, they're available to be used, etc.

25:58And developers can build on them, all that kind of thing. Where do you sit on these kind of arguments and the, I guess, popularity of some of these open-weight models that have been seen particularly out of China? The commoditization narrative around models, I think it's a bit misleading. there's like a tiny there's like 10 companies on the planet that built these models it's a tiny pool of organizations that have this capability coheres one opening ianthropic the chinese model providers um that's a very small group that know how to do this and all of us are posting record profits and growth and so it's not it's not a commodity it's an extremely rare skill set and capability that a few have developed and that is going to power an industrial revolution.

26:50And so the numbers disprove that. Just the scale of revenue growth, profit margins are improving over time, which you wouldn't expect in a commoditized race. You would expect them to be decreasing. I think just all the evidence points in the opposite direction. I just want to talk about China for a minute because we've mentioned China a few times. Very strong technology coming out of that market as well, particularly on the open weight side of the equation as well. In terms of performance capabilities, how does it compare to what you guys are doing, what the US frontier labs are doing? The lead is evaporating very quickly.

27:33There was a lot of talk about the Chinese are just copying. They're just distilling, they're cheating it's you know a lot of these tropes were were applied and and that was definitely true to an extent like they definitely i'm i'm sure uh they were distilling and um you know that i don't discount that however um they have developed an exceptional capability independent of distillation and it's proven by the fact that the latest models that are coming out actually on some benchmarks, on some axes, capabilities, beat the best American models. And you can't copy or distill to better. You can close the gap and reduce the gap by copying, but you can't outperform.

28:19And so I think we, just decisively at this point, China is catching up and in an accelerating way. I think recently the GLM model that was released was serving 10 trillion tokens a day, I believe, purely on Chinese silicon. So even beneath the model layer on the chip and infra stack, they're catching up in a, frankly, extraordinary way. There's always been this narrative, hasn't there, around China for such a long time that all they do is sort of copy technology, even when they come out with new things. it's always a this kind of thing that's just been there all the time this sort of stereotypical but we are seeing and we have seen in so many different areas innovation happening i think what's interesting is you mentioned there that you know the glm model running on chinese semiconductors and and you know in data centers is fascinating because there's been so much work over the last few years that's been going on on the semiconductor scene in china across the whole ecosystem memory through to like the accelerators and elsewhere not just the last few years yeah like the one point like the the lesson from china i think for democracies is that it has been a decades-long commitment um to uh industrial policy to develop these capabilities and there were so many questions about would it succeed you know it's always going to be worse they can only copy, et cetera, et cetera.

29:55And now look at their electric cars, right? Like look at their phones, look at their like models and chips, I'm sure soon. If you have decades long continuity of strategy and commitment to fund industrial policy, to develop capabilities, it will succeed. Do you think Western policymakers, maybe not so much the companies, because I think they've recognize the competition, maybe Western policymakers in particular, still underestimating the technical capabilities of China? Absolutely. Many, many. Yeah, I think we are, it takes a while for people's mindset to change and to recognize a shift. We're still reckoning with the fact that China is no longer just copying, but is actually innovating outright.

30:51And if we believe that democracy is the system of governance that we want to pursue, if we believe that people should have a say in who governs them, we believe in the people in this way of life, we need to strengthen the democratic bloc. We need to ensure that multiple members of the G7, for example, are contributing across all these critical areas of technology, rather than everybody just buying from one. That is a weak and unsustainable and vulnerable posture for democracies to be in. What role, because there'll be many countries that have already restricted Chinese technology or refuse to use Chinese technology, but there'll be many, many countries around the world that are openly adopting Chinese technology across the board, whether it's in EVs or smartphones, telecommunications, whatever that might be um what role do you think the chinese ai industry labs models companies have to play um going forward where will they find success or not um i think there will be a lot of you know unfortunately i think there will be a lot of dependence and adoption uh on chinese technology in the global South, I hope that what we'll see is through this push for sovereign technology, we will see champions emerge across all these different layers of the stack, whether it's like EVs, chips, drones, robotics emerge across democracies.