Can We Trust Silicon Valley With Superintelligence? — With Nick Clegg

29:11The other one is engagement with sort of expensive engagement, because I would have thought at the moment what they're just racing to do. And they're clearly they're clearly burning a lot of money in order in pursuit of this objective is just to is just to expand and get people using these products. And that, in a sense, is a bit of a playbook from from Mark Zuckerberg. I mean, you know, Mark, it's one of his most enduring principles, which is build technology, which people find engaging. You'll work out a way later to monetize that. I mean, how many years was it that WhatsApp barely generated a penny of revenue?

29:45And so maybe that's also something that the new AI hyperscalers are. That's the page they're taking out of the meta playbook. It's just that the ROI for these companies has to be so much better. So much better. Ads alone cannot pay for this. No, no, exactly. No, no, but that's the, I was about to say$10 million question. It's the multiple trillion dollar question. Exactly. No one seems to have the answer to that. And we're clearly in this rather odd position where the infrastructure investment dwarfs anything that happened in the run up to the dot-com boom. So it's not enough to say, oh, well, you know, this has happened before.

30:28you know, yeah, sure, there was a market correction. People went bust. A lot of bunch of companies disappeared. But we also had this wonderful infrastructure that we then repurposed for other things. But this is just off the scale compared to that. I mean, I'm sure if you tot up the amount of hundreds of billions that were spent on telecoms infrastructure by some of those, you know, telcos compared to the trillion. I think this year alone will exceed that. We got 300 billion plus in CapEx from big tech this year and$1 trillion committed to OpenAI this year. Yeah, exactly. Exactly. And no one's been explained to me, but I'm not a financial analyst, but no one's been explained to me how you recoup that money.

31:06So clearly, at some point, something's got a right size. Someone's going to lose a bunch of money. There's going to be a correction. I kind of think that the folk are in the driving seat here, whether it's the new hyperscalers, Anthropic, OpenEye, notably amongst them, or the established players, Google, Meta, Amazon, and so on, Microsoft. I just kind of think they're locked in a thing where it says, yeah, we don't know where this is going to go. But we know one thing for sure. If we don't compete, we're sure to lose. So we don't know whether we're going to win. We don't know what the shakeout is going to be.

31:43But the surest way to lose is just not to throw as much money as your next competitor. So they are in a bit of a kind of, you know, spend whatever it takes mania. And there was a sort of manic feel about the whole thing. That's obvious. Right. And so what they do, they've told us what they basically need to pay the money back. Mark Zuckerberg's talked about developing superintelligence. Sam Altman's talked about superintelligence and artificial general intelligence. Why does superintelligence equal? Yeah. Why is that a pot of gold necessarily? Because I think this is, if you're able to build. Oh, I see.

32:20If you're able to hold on to it and no one else can do it. Well, if you're able to build technology that's smarter than every person, I mean, if you think about what's economically valuable, that would be the most economically valuable thing ever created. If you can hoard it. Correct. This is a bit I've never quite understood. And I may be completely wrong here, but I'd often know what you think. But I've never quite fully understood why that would be a hoardable asset that only one company has, keeps under lock and key, and everybody else is then thwarted. It seems to me much, much more likely that it's going to be a more diverse and dispersed technology than that.

33:05There's a lot of early evidence that some of the most useful and commercial models are quite specific ones, are small specific ones. I just, I don't know, until someone, until I at least, and this is my rather sort of primitive way of looking at these things, until I have a bit more information to be able to visualize exactly what these slightly hand wavy turns like AGI and superintelligence really mean, I find it super difficult to understand why the assumption is that there is a winner-takes-all logic would prevail. Maybe it will, but in a world where, just this week, wasn't it? I've forgotten the, what's the Hong Kong-based company that's just produced another particularly good open-source AI model for agentic and coding purposes, Mini...

33:53Manus? Is that what it is? Yeah. M2 or something. But anyway... Kimi K2? Yeah, maybe. Anyway, we're in a world where it's now become a conventional wisdom that the world's largest autocracy is churning out the world's most advanced open source AI models. I see Chesky the other day saying that Airbnb rely very heavily on DeepSeat for their own work and so on. I don't know. That seems to me to be an indicator of just how versatile and disperse the technologies rather than how much it can be hoarded in a winner-takes-all knockout blow by one of the labs. But I accept, of course, A, I may be entirely wrong.

34:39But secondly, that if that is the case, if the race to superintelligence is this eureka moment where one entity wins, everybody else is left in the dust, then of course that is of such immense commercial value. You can hardly put a figure on it. I get it. I get it. But it does seem to me to imply a bunch of fairly heroic assumptions. Okay, so that is a great lead into this question that I wanted to ask you, which I might have to alter now. I wanted to ask you whether we can trust Silicon Valley with superintelligence. Let's say they do build this all-knowing AI. Okay, so let's do that. And then I want to ask you whether they can even control it itself because of what you just said.

35:19So let's start with, can we trust them, superintelligence?

35:25Well, I said, of course not, because in a sense, my knee-jerk response is, as I said earlier, these are technology companies. You shouldn't trust technology companies. It's not even trust. I use a less loaded way of putting it. It's like, don't look to a technology company to sort out the moral, societal, political, ethical trade-offs by which, you know, which are entailed in the way in which millions, billions of human beings interact with technology. They're technologists. They are hard-driving, highly competitive, highly commercial technologists. So to that extent, no, that's not their role.

36:06It's not their expertise. And it's one of the reasons why this fashion at the moment, certainly in Silicon Valley and DC, this ultra libertarian thing of any kind of constraint, any kind of regulation is unacceptable, is so foolish because it's like they don't have all the answers and nor should you ever expect them to have all of the answers. so so that's on the on the one that's the sort of easy bit the bit i find um just harder to answer is um i kind of just don't yet really know where is what is the moment we work walk through the looking glass and super intelligence has happened some people say it's when they deliver they you know when these systems deliver uh or develop a certain level of autonomy and an ability to self-improve.

36:54Others claim that in the end, actually, there's no way that they can fully escape the rather clunky, probabilistic underlying architecture upon which they're built. And they're always going to come up, spit out some slightly hallucinatory outcomes. They can't be fully precise all the time, and that they're never going to entirely escape the chains of human command. I just, you know, I'm very interested that there are folk like my dear friend, Jan LeCun, and others who've been pretty consistent. And sure, they get criticized and mocked for it. But if you ask yourself, who's been most astute in the kind of commentary of the big trends of this technology over the last three or four years, I would have thought it's fair to say that people like him who have claimed right from the outset, yeah, this is really powerful technology, it's really versatile.

37:47But it's not the only or maybe not even the best route to human style machines, which really can self improve and develop their own autonomy, their own, dare I say it sort of inverted commas conscience. If that's true, and in fact, this alternative paradigm, which people now talk about world models and so on is where the future lies, then we might look back at all this kind of super intelligence AGI hype and say, wow, that really was kind of hand-wavy stuff to recruit AI data scientists in Silicon Valley more than based on something which was fully realizable. So I tend to always slightly look the other way when I hear a lot of hype, because I think the hype just becomes, it has an intellectually paralyzing effect.

38:38I find it very difficult to think clearly when you hear people throw around these kind of really, you know, these lofty terms when I don't know what they mean. They don't seem to themselves have any consensus about what it means and where there are very serious folks saying, look, the paradigmatic limits of the LLM based technology is going to act as a persistent constraint on getting there in the first place. So I really just sort of feel, I just feel the jury is so out now. But I realize, look, there are people in Meta and elsewhere, colleagues of mine who I like and admire enormously. So, you know, it's a failure of imagination on the part of people like me.

39:17No, no, this is around the corner. And the scaling laws still prevail. And the relationship between how much you put in and how much you get out is still holds. And it's still surprising us. Yeah. We listen to Jan here often. He's been on three times. I think he has a really good perspective. But let's get back to the control part. Yeah. I mean, assuming... You know, let's throw out the jargon. Yeah. These systems are becoming much more powerful. They're open enough. You know, the companies have moved towards closed systems, but they're open enough that something that happens in China can get copied right into the US and US into China.

39:53Totally. So is there a concern on your end that like, no matter how much governments or companies might try to control this technology. It's not very controllable. Well, clearly that is one very real possibility if the predictions are even half true about how this technology might develop its own logic, its own sense of motivations, its own sense of survival. You know, you've seen these recent reports from some lab. Unbelievable stuff, right? They will manipulate evaluators just to preserve their values. To preserve their values or to stop themselves being eliminated. And to be extinguished. Yeah, of course.

40:28I mean, they're hacking, they are playing chess games and then hacking the program, the chess program to change the rules so they can win. Oh, wow. No, I hadn't seen that one. Right, right, right. I was referring to the thing I read. When I see this stuff, I'm like, that's so cool. And also just a little bit scary. Yes, it is. It is. It is definitely what suggests a survival instinct, which is a very, very kind of profound thought that that they develop a kind of animal like survival instinct for themselves. and I don't want to just simply dismiss that but it seems to me at the moment these are fragmentary indications they're little sort of flashes of sort of fragmentary evidence that maybe these systems will develop fully full-blown autonomy in the sort of sense that we as humans would understand it seems to me that there's a long, long, long, long way to go and at the moment these are being driven by such powerful systems and so much compute capacity and so much data, is it really plausible to assume that it's just one more heave, one more layer of data centers, yet more improvements at inference as well as training levels will deliver?

41:43Maybe, maybe. I have to say I'm intuitively a little skeptical only because I ask myself, what are the motives for the folk saying what they say? Right, no, I'm not thinking that it's like an escape scenario. I'm just like, if you want this technology to follow a certain set of values or to be used in a certain set of way, it's even less and less like that's possible because of what you're saying. But it makes it a little more reason. It makes it a little more important. And this is probably the bit where I can speak with greater authority than I can about the claim and counterclaims between different paradigmatic AI models.

42:14That's not my expertise. That's why, in the end, politics does need to insert itself. And that's why this peculiar phase we're in where DC and Silicon Valley have kind of, which always sort of regarded themselves with great sort of skepticism and kept each other at arm's length, have fallen into this sort of cloying embrace with each other and all the tech bros, you know, in and out of the White House like nobody's business. and married to this very kind of belligerent America first agenda, you know, to hell with the rest of the world. We're in the lead. We're going to assert our lead ever more forcefully and screw anyone who says anything else.

42:56I would be very surprised if that is a workable strategy for the US. And at some point, I think there's going to be quite a big sort of falling of the scales from the eyes when they realize you can't beat, you can't beat you certainly can't beat China like that and I suspect in the long run you can't beat India either and who knows about other other places particularly as they start adopting Chinese open source models more and more widely in other parts of the world and then using them for ever more ingenious purposes so I think at the moment we're in this rather odd phase where we're making where it seems to me the US political and tech elite are united in an assertion which seems to me to be self-evidently flawed, which is this is not going to lead to permanent, enduring American supremacy.

43:48And when that becomes more obvious, there'll need to be a big course correction. And politics, in one way or another, in my view, which is what I advocate in the final third of my book, politics ideally in a coordinated fashion between the world's major technodemocracies, the US, India, and Europe, in that descending order of importance, will, I think, at some point need to rediscover the merits of multilateral action, however unfashionable that is to say in the current environment in the US. Okay, I want to talk about that a little bit more, and we'll do that right after this. The countdown is on and holiday shopping is officially here.

44:26Uncommon Goods takes the stress out of gifting. With thousands of unique, high-quality finds, you won't see anywhere else. The most meaningful gifts get scooped up fast, And now's the perfect time to cross names off your list. Uncommon Goods looks for products that are high quality, unique, and often handmade. Many are crafted by independent artists and small businesses, making every gift feel meaningful. And I'm feeling that way exactly with my Yosemite ski hat that I've been wearing as the temperature gets colder. When you shop at Uncommon Goods, you're supporting artists and small independent businesses.

45:00Many of their handcrafted products are made in small batches. So shop now before they sell out this holiday season. So don't wait. Cross those names off your list before the rush. To get 15 % off your next gift, go to uncommongoods.com slash big tech. That's uncommongoods.com slash big tech for 15 % off. Uncommon Goods, we're all out of the ordinary. Shape the future of enterprise AI with agency. A-G-N-T-C-Y. Now an open source Linux Foundation project, Agency is leading the way in establishing trusted identity and access management for the Internet of Agents, the collaboration layer that ensures AI agents can securely discover, connect, and work across any framework.

45:47With agency, your organization gains open, standardized tools and seamless integration, including robust identity management to be able to identify, authenticate and interact across any platform. Empowering you to deploy multi-agent systems with confidence, join industry leaders like Cisco, Dell Technologies, Google Cloud, Oracle, Red Hat and 75 plus supporting companies to set the standard for secure, scalable AI infrastructure. Is your enterprise ready for the future of Vigentic AI? Visit agency.org to explore use cases now. That's A-G-N-T-C-Y dot O-R-G. And we're back here with Nick Clegg. You should check out his new book, How to Save the Internet, the Threat to Global Connection in the Age of AI and Political Conflict.

46:37I've read it through, as you can see if you're watching on video. Nick, I want to talk to you a little bit briefly before we get into Silicon Valley's attachment with the Trump administration. how Silicon Valley buys influence in Washington. There was a terrific story, actually a great leak of a memo from Brad Smith, the president of Microsoft. I don't know if you remember this. I think Microsoft employees were asking him. It basically made its way around the press a couple of years ago. The Microsoft employees were asking him why Microsoft donates to PACs and causes. And he gave what I think is like the most interesting and honest answer.

47:15I'll just read a bit of it. he says about the money. I can tell you it plays an important role, not because the checks are big, but because the way the political process works. Politicians in the United States have events. They have weekend retreats. You have to write a check and then you get invited to participate. So if you work in the government affairs team in the United States, you spend your weekends going to these events. You spend your evenings going to these dinners. And the reason you go is because the PAC writes a check. But out of that ongoing effort, a relationship evolves and emerges and solidifies and can tell you as somebody who sometimes is picking up the phone, basically, to get people to answer.

47:55Is that how it works? Yeah. I think it works. It's the way it works in the US. Yeah. Yeah. It absolutely doesn't work like that in the same way elsewhere. But the US political system is so moneyed in a way that's, I think, almost without precedent anywhere in the democratic world, certainly that I know about. And in a sense, you know, what's happened under Trump too is that that has just become, that transactionalism has just become way more overt. You know, you don't like the idea of bringing a gold bar to the person. Yeah, you know, exactly. Gold bars, paying for a gold laminated ballroom, all the rest of it.

48:27It's kind of, it's kind of like, it's become all like, most like a sort of pastiche satire of a highly transactional cash, money, sorry, financially based, you know, set of relationships, which I think all the big corporates are involved in. And they do it openly, they do it legally, they do it lawfully, there's nothing illicit going on. But in a culture where elected politicians are literally non-stop fundraising, and that's all they're doing, this has clearly become an established way, exactly as as Brad, who I like and admire enormously, as Brad explained, you're not buying a decision. You're buying an entry ticket into an event, which then get, you know, et cetera, et cetera, et cetera.

49:16Are the retreats fun? I didn't go myself. There were much better placed folk in my meta team who had much better relationships. You know, I'm a non-American. And my role was, I had oversight when I was at Meta for the US operation, but was quite globally focused. How do, if we talked about in the beginning of this conversation, sort of the need for there to be a check on some of the companies within Silicon Valley, if they're pursuing this powerful technology. It doesn't seem to me like there can be if this is the way the system works. No, I mean, it doesn't mean that the US political system is incapable of regulating or putting constraints on companies that, whose lobbying teams still make these contributions to go to the golfing weekend here or the dinner there and so on and so forth.

50:09So I don't think the evidence is that the US political system is just rendered completely inert by the way in which corporate contributions through PACs and so on are made. it just seems to be the way in which the relationships are conducted and established in the first place. And the US body politic has regulated everything from the pensions industry to the banking industry to arms and oil, even though all of those companies and the companies in all of those sectors are doing exactly the same thing, trying to go to the same golf retreat, going to the same fishing weekend or whatever it is. So I don't think it is the case that that means that thoughtful political action in legislative form is made impossible.

51:05Though I totally understand why people say or think, like, you know, your average punter is going, wow, is that the way it works? But it is the way it works in the US. It's hardly a secret. And it's totally open there. There's nothing illicit about it. That's the crucial thing. Was it ever weird for you that these politicians who I'm sure were like holding these retreats and weekends with your team members were then up on the hearing stage and giving the tech business? I'm the worst person to ask because I come from a much smaller but older democracy called the UK by comparison. And this kind of behavior, this is just totally alien to our, to any European.

51:41I think your wife is German. She is. a little bit like that. It doesn't mean that there aren't money issues in the way in which politicians are funded. I just find it so hypocritical that they would take the money and then call this tech CEO in for the hearing and lambast them to show that they're tough on big tech. But wouldn't you prefer them at least to do that than then not lambast them? I mean, I think if anything... I prefer them to be effective, I guess, is my... Yeah, no, no, no. I'm with you. And personally, as I say, as a non-American, I'm just stunned at the sort of the extraordinary amount of money that sloshes around an American democracy.

52:20But it always has. It's kind of the way it is. And the kind of European approach of having any kind of state subsidy or anything for ads or for any kind of politics is considered to be, for perfectly good reasons perhaps, is considered to be deeply, deeply sort of suspect here in the US. We do it in New York actually. In New York, we have matching funds. So if you rate every dollar you raise up until a certain amount, you get like$7 or$8. Okay. Well, I defer to you on that. But that may be the case. But given it is the way it is, and certainly in my job at Meta, I dealt with the way the world as it was, not as I might ideally want it to be if I could architect it from its foundations.

53:00You'd want, surely, politicians to do exactly the thing you've just said, which is hypocritical, is they take the money and invite person X from Silicon Valley Y to the fishing weekend or the golf retreat, you want them, surely, then still to be able to get up on their hind legs and excoriate those companies and apply pressure to them. But they've done nothing to big tech. No, no, I know. And that's what I'm saying. Of course, you don't want them to, like, say, free pass. But to me, the point is... The thing that's really surprising is why... The system itself is broken if that's what's going to happen because we see the effect.

53:35And I'm not here saying we need to have big tech, Like, you know, massive big tech regulation. Obviously, that could be misguided as well. It's just that when you look at the way the system works, it's crazy to me. So my experience, just to do something very unfashionable, which is to stand up for the political class and to stand up for the, you know, the really, in many ways, really thoughtful, good people who I saw were in the business of trying to represent these companies in D.C. and my team and elsewhere. They're not shady people. These are really decent people trying to just do a good, decent day's job.

54:09But here's the thing. I often found that the reasons why there was no consensus across the aisle on issues had less to do with who's gone on their fishing retreat or their golf weekend, but more to do with deep differences between Republicans, Democrats on state preemption, for instance. That was something which constantly bedeviled and stopped progress. Now, you might say that was an alibi for people not to take action. But I was always very struck, you know, because I would ask myself, why on earth does the U.S. of all countries not have a federal privacy law? It's like, you know, this is, you would have thought they'd be entirely in line with the constitutional principles of this country.

54:50And I think it often came down to very deeply held views about the relative roles and responsibilities of the federal government and states and so on. The thing I am surprised about, I have to say, is why there hasn't been, despite all the energy that is generated on both sides of the aisle on this, why there hasn't been more progress at federal level on legislating to make sure that kids and teens are protected in a way that other users, you know, which is quite special compared to other users of social media and other online experiences. That does seem, and it doesn't surprise me at all that other states, California is only the latest example, perhaps the most significant one, are now taking matters into their own hands because that's a reasonable thing to do if nothing happens in DC.

55:39Totally. All right, I have one last one for you. We have like three minutes left, but I think this is important. So at Trump's inauguration, we saw Mark Zuckerberg, Tim Cook, Sundar Pichai, Elon Musk. I'm probably missing one. Bezos was there. Sergey Brin. Sergey. So the US tech elite have tied themselves to the Trump administration in a way that we haven't really seen before. I mean, the tech, they loved Obama. I don't think they were like this close. So eventually, politics goes in cycles. The Trump's way of doing things may not be the way that the U.S. wants to do things forever or the globe wants to do things forever.

56:20What do you think will end up, what do you think will be the end result for these tech leaders from tying themselves so closely to someone who has gone out globally and, you know, taken this America first approach. Yeah. So I think it's short term and long term. In short term, it works for them because they're all, you know, they're all driven by FOMO and fear or a mixture of both. You know, one person's beating a path to a moral log, oh my gosh, I better get on the next plane to do it myself. They all worry that someone else is going to somehow, in this highly transactional environment, this very capricious transactional environment where the Trump administration looks from a distance like a sort of form of institutionalized gangster capitalism where favors are done to favored companies and individuals and others are kneecapped, whether it's countries.

57:14We just saw over the last 24 hours, Canada experiencing 10 % increase in tariff because of an ad. I mean, he'll even do it to friends too tim cook has been on the receiving end and he's again we've talked about what did the gold bar get him i don't know it's not irrational for them to say wow this is so random and there are so many sort of random drive-by shootings that are going on metaphorically i'm speaking um you know by the by the political class now we've just better try and kind of all do what everybody else is doing turn up at these dinners turn up at these events and hopefully we won't get singled out particularly in an environment where to our earlier conversation they're all spending so much money on what is to them almost a commercially existential race with each other.

57:56Any disadvantage or any advantage garnered by one of your competitors because of their relationship with this administration could be commercially of great... So it's not illogical for them all to do the same thing in the way that they are. I think the longer term problem is that it just erodes an immense amount of trust across the political spectrum in these companies, or at least the leadership of these companies. Because if you're a Democrat, you're going, wow, I remember sitting at a dinner with tech leader X and tech leader Y saying, oh, they were great progressives, and now look at them.

58:35But also, honestly, I also think from the Republicans, they're going to go, I know that this person said X or Y a few years ago. And globally, by the way. You talked about Canada. How is it going to look? Yeah, and globally, I mean, you know, you could imagine what it looks like if you're in Delhi, Brussels. But I think many people understand that business leaders have to duck and weave, particularly in an environment where ducking and weaving seems to be about the only option available to you in this very capricious kind of governing paradigm that you see in the Trump II administration. I think in the long run, though, of course, it poses difficulties for them.

59:13It's trust eroding in a big way, because what happens if there's a Democrat president? Well, they suddenly turn around and turn up at the White House and say, actually, we agree with everything you've always believed. You can't do that. So at some point, in my view, as I say, I understand why they're doing everything they're doing. But at some point in the long run, all of these industry leaders, if they want to continue to see their businesses prosper for decades to come, have to find some way that they don't go into this, I think, slightly demeaning whiplash of kind of, you know, herd-like behavior, limpet-like, herd-like behavior, if I'm not mixing my metaphors, attaching themselves to one administration and attaching themselves to another.

59:52At some point, I hope that a certain kind of distance will be restored between Silicon Valley and DC. I don't think it's, I often say to people, about the only worst thing in a developed capitalist economy than having major companies and governments at each other's throats is having them in each other's pockets. It's much better if there's a certain wary, respectful distance between the two. I also kind of think technological innovation just does better when it's not too tied up with the weird vagaries of politics. And I suspect Silicon Valley will relearn that. Well, we could definitely do another full episode on this topic, on what the values of Silicon Valley actually are.

1:00:32And I hope we get a chance to do that. Me too. Nick, it's been great to see you again. You're always welcome on the show. So the book, folks, again, is How to Save the Internet, the Threat to Global Connection in the Age of AI and Political Conflict. All right. That'll do it for us here. And we'll see you next time on Big Technology Podcast.

1:01:04Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points and miles. Now, imagine a credit card with rewards that can grow in value. With the Gemini credit card, you can earn Bitcoin or one of over 50 other cryptos instantly with no annual fee. Every swipe at the store or gas pump earns you instant rewards deposited straight to your account. Plus, sign up now for a$200 Bitcoin bonus to kickstart your rewards. Visit Gemini.com slash card today. Check out the link in the description for more information on rates.

1:01:40Again, if you're looking to invest in Bitcoin but don't know where to start, the Gemini credit card makes it easy. The Gemini credit card is issued by WebBank. In order to qualify for the$200 crypto intro bonus, you must spend$3 ,000 in your first 90 days. Some exclusions apply to instant rewards in which rewards are deposited when the transaction posts. This content is not investment advice and trading crypto involves risk. The Gemini credit card cannot be used to make gambling-related purchases.