A White House Accord with AI’s Leaders 9/30/26

0:00When did work become so much work? The meeting about the meeting, the hundreds of files to find one insight, setting aside the things you want to do for the things that pop up. Your workday's gone. But what if the insight surfaced itself? Or you could ship the deck without the distractions. Gemini Enterprise helps you get that done. It's AI that knows your business with agents that take stuff off your plate. Make work less work with Gemini Enterprise from Google Cloud.

0:55or find Schwab Market Update wherever you get your podcasts.

1:01Hi, I'm CNBC producer Katie Kramer. Today on Squawk Pod, an artificial intelligence, safety, and more packed between the leaders of some of the biggest technology companies in the country. It's been officially changed to superintelligence or SI. And the president. Is this a cord that everybody signed? Is it binding in any way? I think it's morally binding. But can a massive industry regulate itself? I'm dooming today. I'm feeling dooming. Voluntarily. It all seems to involve displacing humans. I'm back to that. The CEOs go to the White House. There's no conflict between innovation, technology and safety.

1:44I think the technology has very real risks. Author Sebastian Malaby joins us on the leaders behind this deal. The industry is accelerating and building the systems faster than they can think about the safety protocols. And someone who was in the room, Director of National Intelligence Jay Clayton, who says this is a national security issue. It's not just the frontier labs who yesterday recognized that we are responsible for what our products do. It's also the what I would say is the providers to customers who implement those models. Those big questions today. Plus, can we feel good about this?

2:24They have the warmest, fuzziest feeling right now. The president deserves credit for that. Oh, my God. It's Wednesday, September 30th, 2026. Squawk Pod begins right now. Good morning, everybody. Welcome to Squawk Box right here on CNBC. We are live from the Nasdaq market site in Times Square. I'm Becky Quick, along with Joe Kernan and Andrew Ross Sorkin. We got a flurry of news from OpenAI. Will they rename themselves OpenSI, as in superintelligence? An issue to be discussed. But following yesterday's developer conference, CBC confirming reports the company is seeking to raise at least$30 billion from investors.

3:01That would be a new round of funding after the company pushed back on plans for an IPO. It's seeking a valuation of around$1.4 trillion as part of that funding effort. Here's what CEO Sam Allman told CNBC's Kate Rooney about the IPO plans yesterday. I don't have a particular timeline in mind. Once we have run a few safety cases, once we feel like we understand how to contend with this next level of AI and do it safely, I think we could. CFO Sarah Fryer telling CNBC the company is, quote, very well capitalized, but is in a fast paced business and must keep investing. She said the company will go public when the time is right for its business.

3:44We're very well capitalized, right? In Q1, we raised that$122 billion. That said, we're in a very fast-paced business right now, right? When you grow 100 % in your business, you quarter over quarter, you have to keep investing there. Separately, the company announced several new products that it's developed for today. That includes plug-in extensions, a new private intelligence product that gives users stronger control over their data, and a new premium ultra-fast speed tier. It also introduced an AI agent called Dots that have their own cloud computer, can connect to more than 4 ,000 apps and can tap you on the shoulder, if you will.

4:22For reminders, users can message their Dots within ChatGPT. You can also do it from Slacks and Teams in the future, in the future, I should say, via text. So this product will be somewhat similar to Muse. And for those folks who've used a product called Instinct, which was just valued at$10 billion earlier this week with an employee base of only 14 people. Meantime, the product announcements come amid renewed concerns about safety. The New York Times reporting yesterday that months before OpenAI's agents went rogue, two employees raised an alarm with top executives at the company and were ignored.

4:57Here's what Sam Altman said yesterday about AI safety. AI has to be built for people. It has to be good for people. part of that is it has to be safe. It has to be always under human control. It has to do what people want. And you want to be able to use it and be confident that nothing bad is going to happen. As the models have gotten more capable, the safety cases that we now use have to be held to a higher and higher standard. I don't know if they did a search, but Dots was already taken. By the candy? It means something very specific to me. What does it mean to you? The candy. And I'm dooming today.

5:33You're a doomer. I'm feeling dooming. You're a doomer? I'm just feeling dooming. After all the executives get together in a room and say they're going to fix it. I'll tell you why. Because, I mean, we're expecting so much from this technology. And I think about all the things it's going to do. And it all seems to involve displacing humans. That's my latest thing. You know, I'm back to that. You're back to that. I'm back to that. Isn't all the productivity? It just seems like it's going to do everything. And it's going to be amazing. obviously but doesn't it aren't there a lot of i can't believe that's what you're worried about i'm worried about it again i'm worried about because if it if the payoff is as big as we think it's going to be like and it needs to be the industrial and it needs to be to make these valuations these valuations i just don't see how that happens without human sort of displacement Yes.

6:26And then I'm worried about IPOs like you now, if that window shuts. That's something I'm more worried about than I am about the displacement. And then the last thing I'm worried about. Look at you being a worrier. Oh, I can now relax. That's right. Let's share. Let's share. No, the latest thing is what you said. You're right. Very stealthy, the way it's going down. When it goes down fast, it's like, whoa. And a lot of times that's enough to shake out the weak hands. The markets. Yeah, yeah. When it's stealthy like this, you can stay complacent. And then all of a sudden you look around, it's like, well, wait a minute.

7:04The S &P was only down 1%. Well, wait a minute. It was only down 2%. Wait a minute. And then suddenly it's now 5 or 6. It's been down 10 days in a row. Yeah. And then a lot of times if there's a lot to worry about, I'm like, that's good. And now it's like we're almost getting used to the 520 on the 10-year. We're kind of getting used to 90 on oil. Now people are talking about the 30-year potentially at 6%, that that's not out of the question. We haven't been there in 30 years. 8 % mortgages. Yeah. And look, we have seen those numbers before, not in a very long time. The economy is built a little differently today than it was then, and it's the rate of the rise that has people concerned, too.

7:44Right. Let's hope it's good that I'm feeling doomy. Yes. Maybe it's good. I mean, I'm happy. I'm a happy person. OK, I tweeted out I had four animals waiting for me when I and they were they were so excited. So, I mean, that's nice. Life is good. I come in here. It's not like that. Oh, Andrew and I are here. We're so glad to see you. And I told you that all that all probably a politically incorrect joke. But I told you the old joke about, you know, lock your wife in a trunk, lock your dog in a trunk. Drive around for four hours, see who's happy to see you when you open the trunk. The dog will forgive.

8:20You are looking for unconditional love. I'm looking for unconditional love. I get it. I have it. That's why I'm happy. You want unconditional love? That's why I'm happy. That's why I'm happy. By four. And then you're, I don't know, the fur balls almost. Let's talk about what Andrew said. They were there, man. They were there. And do you really think they're going to do the SI thing, Sorkin? Do you really? Do I think that OpenAI is going to change their name? Everybody. Because he's like putting it out an executive order to change it to SI. I have a hard time believing that they're going to change their name and their logo and their everything.

8:56What made you say that? He was being sarcastic. I was just saying if there's an executive order that says it's SI and you noticed Elon at one point stopped himself during the press conference yesterday, he said AI. I said, no, I mean SI, sir. Well, I think it is worth highlighting the positive benefits of AI, which will... SI, pardon me. Thank you. Super intelligence. I mean, everybody was in this. Well, that's what I thought you were intimating, that the coddling of a mercurial president by all of these CEOs may lead to... So he's got some ideas that are out of the mainstream a lot, the president, but he's very sure of what he wants to do and he's very powerful.

9:43So I don't know. You know what I mean? What would you do if you would you say? It's not bad branding. You're saying if I was OpenAI? If you needed the president to like you and be on your side like these guys, would you change the way? Would you change the way, no, not just the lingo, but would your whole behavior, would your whole behavior be more amenable to making him happy? Yeah, more deferential. Don't you think that they're all trying to do that in their own way? That's what I mean. But if you were there, because. Are you saying me? I'm saying you. I'd be hard for you. I would be like Dario.

10:23Which he's sort of like taking his castor oil, but. But but he sort of like does a thing and sort of says, Mr. President, I'm trying and it's all. Yeah. Yeah. Probably. It might be interesting. Did you see the seating chart for where they all sat? Yes. Dario was far away. Yes. Well, suppose he wasn't invited. And then he had dinner with the last time. Two of them were two of them were there, though. From from Anthropic. Yeah. Right. To me, the man that was missing. Nobody from Apple. Nobody from Apple. What does that say about where Apple is in the AI race? No, Tim Cook. And Tim is obviously very close to the president.

11:03But the idea that he wasn't there or John Ternus wasn't there. Tim was at the state dinner a couple of nights before that. No, but I'm saying in terms of a conversation around AI. Are they considered as part of the big names? Are they even considered as part of the game? Maybe they were invited and didn't go. Well, in case you haven't figured out, there were some of the world's, they want me to read this, like we already talked about, one of the world's most powerful, most of the world's most powerful tech executives in AI gathered at the White House yesterday where they signed a voluntary agreement to adopt safety policies to include risk reviews, auditing and third party evaluation.

11:39That's something that people have been talking about on the show. They also agreed to adopt President Trump's rebranding of AI as super intelligence, super duper. I believe they're going to be used for the good. And when they're not, we're going to be able to nab them. But there's going to be a tremendous self-policing aspect. We're also thinking about forming a committee of sorts where we put maybe 10 people on that committee. It could be from that group so that we can so the committee can watch over. the whole enterprise. And several of the tech executives spoke outside the White House, including Anthropics, Dario Amadei and NVIDIA's Jensen Wang.

12:22Whoever wins AI wins. I think that's very important. But I think the technology has very real risks. And, you know, the mechanism, how we address those risks is still under discussion. But we all need to work. We all win and we can win safely. If we do this right, we work with the president and everyone here, we can win safely. There's no conflict between innovation, technology, and safety. We are going to have to create new technologies to advance the capabilities of AI, but we're also creating new technologies to advance the safety of AI. All of this technology is being developed by the industry today.

13:01He said something relatively innocuous. I would have said something, tried to say something relatively innocuous. Would you have said super intelligence in that innocuous statement? We can do this the hard way, Andrew, or we can do this the easy way. It's like, I mean, it's your choice. Right? Sure. Isn't that sort of what is happening? The easy way might be easier. Hence why it's called easy. As a separate matter, I actually think this is just... It's all good. I actually, I'm not under... I want to be clear. I am actually not critical of the president in this instance at all. Because I actually think it was great to bring everybody together.

13:41It was great that they actually had a conversation. A lot of these people don't even want to be in a room together. Have we switched doomers? And the other piece of this is what they signed, while not binding or particularly hard to accomplish, because effectively most of them are doing what's already documented anyway, the idea that they've actually put their name to it, I think is all to the good. It's all to the good. And by the way, it was done instantaneously versus how long it would take to actually set up regulations, which might be coming anyway. We can never we just have a hard time occupying the same.

14:14Before I was saying I'm ready to roll the dice with Trump. Go for it. We've got to beat China. Now I'm do me a little bit. And you're kind of like glad they were all there. And you think a moral obligation not to kill the humanity is going to be sticking. I'm not suggesting it's going to stick. I'm suggesting it's a step in the right direction. By the way, while I'm in the praise game, I'll give you something that actually forget about all politics. So yesterday, after we did our show together, I spent much of the day still in D.C. at the America.gov announcement. And actually, Jensen and Musk came later and a lot of them were coming back and forth speaking.

15:03What they did with America.gov yesterday was unbelievable. This is the Jebbia project. Right. And it was, I mean, what they did with that website, if that website is able to do all the things that they say it's going to be able to do. That Oz was talking about. Which is basically the front face for 27 ,000. For 29 ,000 websites. The ability to get a passport, the ability to get your Medicare. I mean, it was like being at a product, like being at an Apple product announcement. And when they went through what they did and the president was there in the morning and walked through what they were doing.

15:38And Marco Rubio was there. Scott Bessent was there. The whole cabinet was there. It was one of those moments where you actually thought the government was working for you. Really. And Jebby deserves an extraordinary amount of praise for it. That's a lot. Again, yeah, the pot, whatever your pot doesn't matter. When people talk about what government is supposed to do, this product, if it actually, again, there's a lot of things that have to come in 2027. Some of it they were launching yesterday. Some of it they were showing what's going to come. If it does all the things it's supposed to do, that's great.

16:12I have the warmest, fuzziest feeling right now. And by the way, the president deserves credit for that. And he deserves credit for putting Jebby in that role. And Jebby and his team put it all together. Now I'm getting hot flashes. Wow. So this is amazing. I'm just saying. And it's just as surprising to me, too, that that. For the government to ever act like Apple, as you just described, it is that's revolutionary. And maybe that's what A.I. is going to give to all of us, maybe. And even the government could be dragged kicking and screaming. Well, look, Andrew's already said, I don't know if you were here that day, but the the the interface for the app for the Trump accounts.

16:55is really easy to use. And I've used it too. And that was also a JVIA project. And interesting. It's like using a real app. And here, I give the president more credit. One of the things that... Can we save it so that we can space it out? One of the things that they all talked about is that when they were given the mandate to put all of this together, because you could consider the president a bit of a host, someone who likes to host things, someone who thinks of himself almost as a, They used the word concierge service, like a hotel. That was really sort of the thought process behind how they actually put this stuff together.

17:32And it showed. It really showed. So there. You need to read this.

17:42Tease will be next. Coming up on Squawk Pod, the AI inflection point and the personalities involved. Why some are up and some are down. Author Sebastian Malaby on the White House gathering and the role of anthropics leader Dario Imude, who has been advocating that we hit pause, slow down and just figure out what the heck is going on. The administration said that's woke. That's ridiculous. We want to accelerate. We want to beat China. And so Dario was frozen out of the White House circle. Fall flavors are waiting for you at Whole Foods Market. Cozy picks all through the store, like maple leaf cookies from 365 brand.

18:21Swing by the bakery for pumpkin cheesecake, here for a limited time, and pumpkin turmeric bread freshly baked every single day. Sip the season with 365 brand cinnamon spice coffee and pumpkin spice creamer while you still can. Spice up fall at Whole Foods Market. With the Discover Cashback card, it's payback time when you earn cash back on everyday purchases. Activate and earn 5 % cash back at different categories each quarter on up to$1 ,500 in purchases. That's 5 % cash back at different places each quarter, like grocery stores, on gas, and at restaurants. It pays to Discover. Terms apply. See discover.com slash 5 for details.

19:05This episode is brought to you by Schwab Market Update, an original podcast from Charles Schwab. Join host Keith Lansford for this information-packed daily market preview delivered in 10 minutes or less, including projected stock updates, monetary policy decisions, and key results and statistics that may impact your trading. Download the latest episode and subscribe at schwab.com slash marketupdatepodcast or find Schwab Market Update wherever you get your podcasts.

19:37This is Squawk Pod from CNBC. Up and Andrew, cue. You're watching Squawk Box right here on CNBC. I'm Andrew Osorkin along with Joe Kernan and Becky Quick. President Trump meeting with tech CEOs focusing on AI risk and regulation. Joining us right now to talk about everything we've been hearing about AI is Sebastian Malaby. He is the Paul Volcker Senior Fellow for International Economics with the Council on Foreign Relations. He's also the author of The Infinity Machine. That's a biography of AI pioneer Demis Hassabis of Google's Deep Mind. And, Sebastian, thank you for being with us this morning.

20:14So this has been the topic du jour for years at this point, but especially the last couple of weeks. Things have been in focus since these Frontier Lab CEOs came out and said that they were worried about what was happening. What did you think of yesterday's meeting just in terms of everyone being there, the accord that's being signed and the idea of the industry regulating itself? I think the participant to focus on is Dari Amadei of Anthropic, because he was so much on the outs with the administration. You know, he was the man who wanted regulation. The administration said, that's woke. That's ridiculous.

20:47We want to accelerate. We want to beat China. And so Dari was frozen out of the White House circle. all of a sudden he gets a call last week from the president saying, why aren't you coming tonight to the state dinner with the Chinese? And Dario Amadei hasn't been invited. The president didn't know. And the president thought, well, you should come anyway, even if you were off the list, even if my team left you off the list. And Dario is unable to get there physically in time. So then the president says, well, what about dinner on Sunday? So Dario goes for dinner in the White House and they kind of break the ice a bit.

21:22It seems to go well. Then he sticks around for Tuesday, has lunch yesterday with the president and the other tech leaders and the House speaker. And so this person who was supposed to be, you know, the anti-White House figure in AI has suddenly come into the fold. And, you know, that's, I think, more important than the statement that came out saying companies will regulate themselves, blah, blah, blah. Important why? What does it mean, not just from an optics perspective? What does it mean in reality? Well, there's been this big division in the AI world between the ones who want to accelerate.

21:53That's the administration. That's Jensen Huang of NVIDIA. That's Meta, which believes in open-weight models that aren't controlled. That's some figures like David Sachs, who was in the White House now back in Silicon Valley. These were the accelerationists. And then there were the very cautious people. That means Demis Asabis and Google DeepMind. That means anthropic, Dario Amadei, and somewhat also open AI, being cautious. And I feel like these two camps may be coming together a little bit, which is good news, because sociologically, it's been very hard to get coordination among the different labs.

22:30And unless you coordinate safety measures, you know, if some people are cautious and they go a bit slower, then the other guys are not cautious and they go faster, it doesn't really get you any more safety. Right. You lose on that front. There has been a lot of supposition about why Dario Mote, why Sam Altman, why all of a sudden they've raised these concerns, why they're more cautious. And the pushback you've gotten from David Sachs and others in the administration is that, hey, they were just coming looking for liability protection. And they want to slow down and throttle everybody else now that they're ahead to keep them from being able to catch up.

23:08What's your take on that? Well, look, that's total nonsense. It's also hypocritical. I mean, some people on that side of the debate, especially I think Jensen Huang of NVIDIA, are accusing Anthropic and the others of talking their own book and wanting liability protection. The truth of the matter is Jensen Huang wants to sell more, you know, GPU chips and get the profits from that. So he wants the AI models to be all open weight, not regulated. That means they're going to compete with each other. is going to have no margin for the model layer, and all of the profits then accrue to the chips. And that's why Jensen Huang is saying that.

23:45I mean, to be fair, NVIDIA, Jensen Huang, they have a$30 billion stake in OpenAI. And they're very tight. Right. But it's a$5 trillion company. I don't think that's really driving. Yeah, right. I mean, you know, I think, but back on Dariama Day and those who want to be cautious, the idea that this is some plot, I mean, he's been talking about safety for 10 years, way before it was even a commercial thing to be doing AI. He's always been focused on the risk. And the risk is real. You don't have to just take it from him. You can look at academics who are experts in this. You can look at the consensus across several different labs that this is dangerous.

24:20You can look at the UK AI Safety Institute, which is the most respected of the government regulators. All of these people are saying there is risk. So it's not an invention. So when you hear the safety researcher community say, look, the way these things are built unto themselves create risk. And then you hear Jensen, who was sitting where Joe was sitting earlier this week, saying, look, this is an engineering problem that can be solved. He didn't say easily, but he said it effectively can be solved. In fact, they're producing a product in part to try to solve it and to put some guardrails around it from an open source software basis.