Democratized superintelligence is coming: The world needs to get ready

0:03Roberta, great to see you. Lucia, always great to see you. So it's been a challenging couple of weeks for AI in the media. Society is asking some tough questions about super intelligent AI and its influence on how we work and how we will work in the future. Very far into the future, we hope, right? Hopefully, hopefully. So today's episode, super timely. We have Brett Taylor, who is actually the chairman of OpenAI in conversation with Eric Kutcher, our North America chair. Brett is also the co-founder and CEO of Sierra, which helps companies build sophisticated AI agents. Yeah. And you know, Lucia, what I found really interesting about this conversation is that Brett is both very optimistic about what AI can do, but also incredibly realistic about the concerns associated with AI development.

1:05And in this conversation, we hear about all of it. We hear about cybersecurity, frontier labs versus open-weight models, and the topic that everyone can't get enough of, and that's data centers. It's a great discussion. Let's get into it.

1:28I am joined today by a man who has many titles. He's a friend, but he's also the founder and CEO of Sierra AI and also chairman of OpenAI, if I remember that correctly. That's right. Amazing. So it's great to have you here with me, Brett Taylor. Just truly wonderful and can't wait to have this conversation. Thanks for having me. I was at a dinner last night and I was asked about this moment and I said, this is the most exciting moment that any of us will get to live through and lead through. And you are at the center of it. And so I just love to hear a little bit. How do you think about the moment we're in, the things that excite you most, and maybe a little bit of the things that might even frighten you a little bit?

2:11Yeah, I think we're in a very significant moment in technology, this current generation of artificial intelligence. It fundamentally, first, I mean, just in really simple terms, it's changed the way we interact with technology. But more than anything, because these models support basic reasoning, they're enabling computers to reason about ambiguous tasks and actually autonomously complete work. We're seeing that impact on the economy in the short term with things like software engineering and coding agents, customer experience, both customer service and sales. We are in the foothills of artificial general intelligence.

2:51We've already seen recent models prove unproven math conjectures, or in some cases, disprove some of those math conjectures. I think in the domain of information, where digital technology is all that's required, we are probably already at a point of having super intelligent AI. and I think we'll have a huge impact on science, a huge impact on the digital economy. That's why software engineering has been impacted already. I think everyone in the world will have access to a super intelligent financial advisor. They'll have access to the world's greatest primary care physician. Every kid in this world will have access to a personalized tutor.

3:28I think it will be amazing. So I think the other part of this is I think we're going to end up over the next few years seeing the digital parts of the economy dramatically impacted and the rest not. It's going to create a lot of complexity when you end up with sort of more of an overhang in some industries than the other. And then on the risk side, I think in the short term, cybersecurity is probably the biggest, most acute concern because just like the early days of the internet, we often invent these technologies and the world's industries are not necessarily equipped to deal with the security ramifications yet.

4:02I think we're in a period of sort of societal vulnerability. Let's come back to cyber because I think that is a very real risk. and I think you have a unique insight. I tend to agree with you that this will be better in the long run. I can think of many reasons why, but the short run getting through that will be difficult. One of the things that you hear a lot about is there's lots of individual productivity, but there isn't necessarily institutional level productivity that we can measure yet. We see that in kind of the client work we do. My articulation of this has been, it's an 80 % business problem and 20 % technology problem and that the technology is so, it's first of all more advanced than what we can really use it for as we think about redesigning these processes.

4:42Secondly, it is democratized, so it's already being embedded in ways that we don't even fully understand. There's a series of real workflows that you are trying to help your clients, your customers reimagine. So how is that going? So just for a context on CIRA, we help companies make AI agents that serve customer service or revenue-aligned agents things onboarding a merchant into a marketplace or helping onboard or do KYC for a bank or helping with prescription drug adherence for a medical company. We serve many of the largest companies in the world. And we're helping do everything from customer service to more revenue-aligned things like helping people originate mortgages.

5:27I think every company's AI agent will become their digital front door. I love one example. Their agent resolves over 90 % of cases autonomously. And if you look at a company growing that quickly, you've decoupled your costs from your growth, which is just remarkable leverage. And then if you look at the revenue-aligned agents, you can lower your customer acquisition costs significantly because you're essentially using AI agents to onboard, to engage, to sell, which means you can scale it infinitely and at lower costs, which means you can lower your customer acquisition costs, improve retention.

6:00All these things are really fundamental to the business model of these companies. Part of the responsibility of being involved with OpenAI is whether you're optimistic or not, you should have some paranoia about the downside risk because we're a mission-driven company. But I believe in capitalism. And what's happening now is everyone has access to the same intelligence, the same AI. So you can't just get more efficient and pass the profits onto your shareholders because Whereas your competitor could lower prices, they can invest more in innovation, they can do all these other things. And you're seeing that play out in real time.

6:31I think with the benefit of hindsight, adopting AI for these core corporate processes will be an imperative, not a competitive advantage. But as a consequence of that competitive dynamic, you're going to see almost real-time reinvestment, which you're already seeing in software. Yeah, I am almost entirely in alignment with where you are. One of the things I say to many CEOs is this is your legacy moment and failure to choose to actually move quickly is equally a choice. And it will leave a legacy. And so I think there is this moment of inaction is as much a decision as action. And maybe let's go to cyber and let's talk about a little bit of this moment.

7:10How should we think about this moment? And even things you read about, which is within the test environment, sometimes the AI can escape. How much risk do we really face and what are companies doing that are doing this well to kind of do their best to remedy this? Well, first I'll say, I think there's a lot of this which is truly unprecedented given the capabilities of the models. But I would say historically, when we introduce new technologies, there's often this period where our security protocols haven't caught up to the capabilities. So I just want to start by saying, you know, this idea we invented technology and we're sort of grappling with the technical ramifications is normal.

7:52The hard part now is if you look at the adoption curves of technology, I love those graphs, which are like electricity and refrigeration and all these things. And like AI just looks like a vertical line. So unfortunately, both the models are approaching superintelligence and we've connected all of our infrastructure. So I would just say the blast radius is just much, much broader than it is with those. I don't mean to draw too many analogies, but I also think that I believe in the good guys. And so I think right now these models are very good at code. And as a consequence, they're good at finding vulnerabilities, both when they have access to code or even doing more black box style hacking.

8:33I think what makes them good at that also can make them good for offensive security, both finding and patching vulnerabilities and over time active monitoring as well. And so I would just say I think the reason we're in a risky period right now is there's not off the shelf solutions to solve a lot of the active monitoring and patching. So I've talked to a lot of CISOs and CTOs who are in this position where they have these agents and models that identify thousands of vulnerabilities. What do they do with it? They try to patch them all, they'll break all the systems. And so they're in this position like, do you want the system to be running or do you want it to be patched?

9:10Because right now that's the option. So we're just in this very uncomfortable situation right now. So the one thing you didn't talk about is large enterprises have more resource to be able to fix these things. One of the real risks here is you have an awful lot of small and medium businesses that frankly are still running technology that is quite dated and quite exposed. And so we actually have potentially sort of asymmetric risk depending on where you sit. I agree. Right. And it's the same, you know, local hospital clinic that was targeted with ransomware attacks and stuff, too. It's like the same cohort of, I'd say, important, but, you know, from an IT standpoint, you know, vulnerable sort of population of organizations.

9:57I think we need it's going to be like a lot of, I'll say, operational, you know, safeguards in the short term, which is like you're seeing, you know, both the labs and governments or work through. What is a responsible way to roll this out? I'm a huge believer in entrepreneurship as a solution to these problems, which is we need companies to help make automated solutions to these problems over time. And I believe that is possible. And I believe that the companies who do will create a lot of value. And as a consequence, I really think we need more security entrepreneurs to help make out-of-the-box solutions so that that local clinic can have the right security capabilities that they need to be successful.

10:39So I think you can end up with something where we can really democratize access to great technology with this technology as we have done effectively. And I really think there will be some bumps in the road here. But we need to collectively continue to responsibly, iteratively deploy and responsibly, iteratively secure. And I think it will be a pretty complicated couple of years. But I'm hopeful on the other side of it. I mean, just to be blunt about it. these vulnerabilities existed before, you know, and it just, we're just finding them all now. And it doesn't mean that's not a problem, but, you know, fundamentally, I want to fix the vulnerabilities.

11:17And I think that we have an opportunity to do that. And we just need to be collectively responsible about how we're deploying the technology. I mean, this is like the definition of tech debt. Like, I mean, this is cyber tech debt. I mean, we just have, I mean, decades of it that exists. All right, let's talk about, there's like three or four topics that are kind of in my mind that there's probably no one better to talk to. Let's talk about open weight versus frontier. How are you seeing people think about this? And how do you think about kind of the power of, I'm going to call it open source, open weight versus kind of individual entities that are really working on at the edge of the frontier?

11:51You know, I think there's a kind of like three or so reasons why one would use open weight models, or at least that I've heard from CIOs and CTOs in the industry. Number one is cost. So if you can, for a task, use a model that is a lower parameter count and host it in a way that's cheaper than the models you can lease or rent from the Frontier Labs. It's a way of combating some of the tokenomics and the spin. Number two is fine tuning. So with open-weight models, you can do a post-training process and make the model your own, which is a way of taking a smaller model and making its performance close to a large model for a specialized task, which is an important way to optimize spend.

12:42And then finally, sovereignty. You need to sort of own things full stack to secure your data and your infrastructure. The last one, I don't want to say I'm dismissive. I think sovereignty is important, but I think you can solve it contractually. And through a lot of ways, this is the same argument with cloud that you and I sort of went through in the past two decades. And I think I'm not convinced that open weight is a necessity to achieve data security just in the same way you didn't need to remain on premises to maintain security either. As it relates to cost, this is the one where I'm probably most bullish for the frontier labs.

13:20I think the frontier labs have the most compute scale and can vertically integrate their models with the hardware. It's not quite technically this way, but you can think of it as like, hey, if a lab can distill a frontier model and sell it back as open weights, the labs can just distill their own models. And the cost of running these models is in the inference. And so it's not obvious to me why a lab that's distilling a Western model has an advantage over the labs themselves to actually provide the price performance that you need. And so I think there's always going to be a place for open-weight models, particularly given the need for companies to do fine-tuning.

14:03But I think it's really important to understand the jobs to be done. Why are you using it? And if it's just about cost, I actually think the frontier labs will have a sustainable leg up just because of the tie of like the CapEx and infrastructure to basically token efficiency. As it relates to the other strategic issues, I think they're the real fine tuning and sovereignty. And but even there, I think, you know, there's there's arguments that the labs could solve some of those problems in other ways as well. So maybe we can take this into another topic, which is the data center build out itself, right?

14:38How do you think about this, especially kind of in that context of the cost effectiveness that might come from the frontier labs? But with that cost effectiveness, does that create enough capacity to actually make the investments that are being outlined and, you know, which are real? There's a lot of risks because there's so much interest in AI right now. you end up with sort of dumb money and smart money. And just like it's like classic kind of gold rush behaviors, right? The thing I'll say that when people talk about circularity, I don't totally buy that. And so you can make an argument, is that money well spent and all of that?

15:16But it's real money growing at scale, growing faster than most smaller scale businesses. If you buy that AI is going to write all of the software of the future, which I think is almost objectively true at this point. What is the world's demand for digital technology? And then if you include things like sales and customer service, and you include the legal profession, all these others, I'm like, I don't know. You don't need to be that much of an economist. You don't need to make outlandish assumptions about future capabilities to think about the economic impact just with current capabilities. But I think the idea that it's quote unquote circular is largely because of the complexity of financing these large CapEx projects, which is, you know, just not palatable for a lot of investors.

16:05And I get that. But I feel fairly bullish about the demand. And I think that the obvious challenge of all of this is the pace of AI is measured in days. But the build out of a data center and the power, real estate, all that takes so much longer that the time horizons, there's an impedance mismatch between them. So I'm glad that we're building out this infrastructure. I actually think, you know, just as a patriot, I think it's very important that the U.S. lead on this, both in AI and in our infrastructure. I'm not saying there's no risk, but I'm obviously entirely biased on this, but I'm very bullish about the importance of this infrastructure to our economy in the future.

16:47So, and I want to come back a bit to the data center build and the importance in the U.S. because there is pushback there that we should understand. It's more and more in the news every day. But everything you said, it's hard to argue with, right? The rate and pace of growth at this level of scale we have never seen. By the way, it's why valuations we've never seen either for companies that are at this stage of their life, right? It's a phenomenon that is truly unprecedented. But revenue and profit or cash flow are two different things. And if you kind of take the system-level view of the amount of capital we're putting in the ground and saying, are we seeing collectively in the system the operating cash flow to justify, it doesn't close today, right?

17:30Which is why we're seeing more debt financing than we've ever seen. And so ultimately, the question is going to be, does the profitability get there and what's driving that profitability? Or are the unit economics continuing to be where they are in the system? I'm not arguing over one side or the other, but the system level. And that part, I think, is still, and by the way, there's all kinds of things we don't know. We don't know whether or not, you know, we get more productive on inference and therefore using different chipsets. And therefore, that actually changes the economy, you know, that may change the energy requirements.

17:59We don't know whether we continue to innovate at the same level of rate and pace on the new models, which is, and so we just don't know all of these things today. And so you have to believe that the equilibrium gets there, but almost by definition, given the level of depth that we're putting at these things, we're a little bit out of balance at the moment. Yeah, I'm not gonna argue with that. And, you know, broadly speaking too, I mean, you end up with just a huge amount of, I don't know if fixed cost is the right term. You're more of the finance guy than I am, but, you know, inference can run at relatively high margins and is, particularly for enterprise applications already.

18:31I think just like the cloud infrastructure market, it probably wouldn't be profitable if there was 100 companies in the space, but there's basically like four. I think at scale, I think these models can monetize at scale and you can make it work. There's a lot of risk. I mean, that's why there's probably when you have this amount of capital at these stakes, just the risk is higher than ever. So I remain optimistic, but I don't mean to dismiss the risk at all. It's extremely high risk. So the other thing, which you'll just have a great perspective. And so, you know, and it's funny, I just, I understand the perspectives of the general population on some of these things and the fears associated with it.

19:09But you hear whether it's fear of job loss, whether it's which, you know, whether it's fear of I won't be able to afford the power or get power from my home or the cost of water and implications or it's the whatever, you know, how do you think and we're seeing it play out, right? I mean, how many states have recently either pulled incentives and or written laws to limit data center size and scale, even in places you wouldn't have expected? How do you speak to people to get them to understand, if you can, kind of get them through those fears? Yeah, I am worried about this. You know, fundamentally, if you think about, you know, your perception of a data center in AI, it's probably driven by two things that's personal.

19:55You know, one is your perception, will AI be a benefit to me broadly? And will this data center benefit my community? And I think the most AI companies have done a fairly abysmal job at communicating both of those things. In part, because many of the research labs are mission driven. There's been a lot of discussion of the risks of AI. I think it comes from an authentic place because you want to do this responsibly. And so a lot of the focus of those firms is alignment, the impact, basically the resilience of society to AI. And very few people at these prominent labs are talking about the benefits, universal universal access to tutoring, universal access to good financial advice, democratizing access to health advice, enabling small businesses to have the world's best tech at your disposal because you've got a super intelligence there.

20:53And then on the individual community side, I think it's very important that when we build out these data centers, they purely are a benefit to that community, whether it's taking accountability for power so that the bills don't go up, ensuring that just the environmental impact is thoughtfully, but just engaging with the communities because I think there's an opportunity to really lower taxes in these areas, provide jobs for blue collar workers like electricians and HVAC and others. And I think there's an opportunity to re-industrialize our country and provide a lot of benefits. I think we should put the burden on the companies building these data centers and to really get local and having conversations on national television is the wrong level.

21:41You should be in community meetings and actually engaging with members of the community about what actually matters. And I think if we do that and ensure that we're taking on for things like power is an example, taking accountability, I think we can make this. We can shift the narrative around this and smart mayors, smart governors can have smart demands. And I think there's a path here, but we're not the ones who need to be convinced. I am much more optimistic about the impact of this technology on the collective. You know, I start with kind of a very macro level view, which is if you actually follow what's going on in the economy, we are maintaining employment rate with flat to negative jobs.

22:24And the reason for that is we actually have a shrinking working population. And we've never had that before in the U.S. And that is not just a U.S. phenomenon. It is most markets in the world because we have a population that's declining and aging. It's just it's an interesting dynamic. And so at some level, if we want prosperity and we want to see GDP growth, if you don't have labor growth, the only opportunity is to actually get productivity growth. And I happen to think this thing is going to take longer in enterprises. This is my bold case for AI, which is it's helping people do things more productively.

22:57You know, you can have, and I think it will help entrepreneurs in every single state and every single community. You know, if you think about someone starting a direct consumer, you know, small retail business, you can have your AI agent help you with demand generation, help you set up your storefront, help you buy your ads. These things that you might not have been equipped to find someone to do. But broadly speaking, I think the strength and resilience of the U.S. economy is how competitive it is. And I think what's going to happen is because this technology is widely available, it's going to create more opportunities.

23:37And we're all going to compete. And I think we need to continue to embrace this technology with empathy, understand that it will change the way we work. But the opportunities are greater than what preceded it. I firmly believe that. Think about the cost to start a business today. Yeah. Right. It's so low that the level of innovation and entrepreneurship and therefore jobs, I just think it's, we don't yet know what that looks like. Brett, this has been unbelievable. We got an insider's view as to kind of how AI is going to shape us going forward. Well, thanks for having me. I really appreciate it, Eric.

24:07Awesome. Thank you, Brett.