Satya Nadella on Microsoft's agent bet and AI’s trust problem

25:19It takes the hard work of leadership that really allows you to then come up and reconceptualize how to do this stuff in the new efficient frontier.

25:30Alex Heath:Mercury is a modern take on banking built for startups like mine. When I decided to start my media business, Mercury was by far the most straightforward, full featured banking solution for me to set up quickly. The interface is intuitive and simple, saving me valuable time every day. I use Mercury to track my spending, bills, and invoicing. I love that I can delegate permissions to my team so they can keep things running for me in exactly the way I want them to. My favorite part is how forward-looking Mercury is with AI. Legacy banks are stuck in the past, but Mercury is built for how modern software works today.

26:04Alex Heath:I use its built-in command assistant to analyze cash flow and help me move money. and Mercury also connects to other AI tools like ChatGPT and Cloud. I use this feature all the time, and the folks at Mercury actually let me know that I'm one of the top users of it. So trust me. It's finally easy to get real-time financial data about your business wherever you need it. Visit Mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC-insured bank. Banking services provided through Choice Financial Group and Column N.A., members FDIC. I spend a lot of time context switching between meetings, No time to process one before the next starts.

26:41Alex Heath:Thankfully, Granola runs in the background the whole time. It's an easy-to-use AI notepad for meetings that works everywhere, even on phone calls. I use Granola to recall what was said in meetings and create helpful summaries. I use it every day to stay on top of what I need to get done with my team. It connects to my email and suggests follow-ups for me to quickly review and send, saving me valuable time. Granola isn't just a core part of my workflow. It's basically my second brain. Try Granola at granola.ai slash sources. and use the promo code SOURCES for three months off. AI is only as useful as the context it has.

27:14Alex Heath:But when that context is scattered across tools, threads, and DMs, your team and your AI agents are flying blind. That's the problem Jira by Atlassian solves. What's the goal tied to your project? What got decided last week in Slack DMs? Atlassian's teamwork graph pulls all of the valuable pieces together, from Jira, Confluence, GitHub, Slack, and more, so nothing falls through the cracks. you get 44 % more accurate results with 48 % less token usage. With Jira, you can easily share your work context with AI agents you already love, like Claude, Cursor, and GitHub Copilot. Assign them work directly or connect your tools through MCP.

27:51Alex Heath:All of this lets you spend less time digging through endless links and messages, chasing down what got decided and by who, and spend more time actually shipping. Learn more at jira.com. That's J-I-R-A dot com. Framer is the AI website builder that brings agents into the same canvas where your website is designed, managed, and published, so you can move faster without giving up your taste or control. Framer powers the Sources podcast website at podcast.sources.news, where you can find new episodes, transcripts, and a lot more. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.com slash sources for 30 % off a Framer Pro annual plan.

28:33Alex Heath:That's framer.com slash sources for 30 % off. framer.com slash sources. Rules and restrictions may apply. Microsoft just changed its financial reporting structure from three segments to two. You now have agents and infra, devices and consumer. My take, and I want your reaction to this, is that you want investors to see apps, agents, and cloud as one connected business. and to judge you all on how much value you're capturing across that entire AI stack. Is that right? Was that the reason for that? That is correct. In fact, that is exactly correct. So when people ask us, hey, where is your ROIC?

29:13It's sort of our cash flow and how we invest that cash flow as capital and operating expense is to get returns across the stack of agents and infra. And we want to be transparent about each of those, but also the industrial logic that these are not all different investments. It's about one investment that then needs to deliver value to customers in a competitive marketplace, one layer at a time. But all of that will be very transparent to our investors to see, but both the logic of why the firm does what it does, how they can track us each quarter. And also, I think uniquely positions us because it shows.

30:02But the other thing I would also say is we're not trying to build a narrow business where we have three customers for our infra and call it a day. That's not a long-term business for us. We want to be able to serve every small and medium business and large multinational with their token consumption, their need for a fine-tuned model, maybe even their own frontier model. That's more important than, you know, we love our business with OpenAI. They're one of our largest customers. But our goal is not to have one large customer or two large customers or five large customers. It's to have many, many, many large customers.

30:37Alex Heath:You brought up OpenAI. I had Sam on the podcast recently. He said he sees, quote, unsustainable silliness in parts of the AI build-out happening right now. And I'm curious, if you look at, you know, you all are committing a tremendous amount of CapEx. I mean, all the hyperscalers are, it's trillions of dollars. Is there anything you see that would give you pause as you're evaluating, you know, CapEx commitments in the years ahead? Is there anything you're looking for? Yeah, so I think, you know, look, what is happening outside of Microsoft is for others to speak to. But I would look at our own business, and I've been very clear about it, is that you have to build for that full stack, right?

31:18So I care about building our infrastructure for a diverse set of customers, building and serving large customers like OpenAI and Anthropic and others super well, but also building for our own 1P products, right? Copilot. And so it's not like we're going to allocate or build, assuming that someone somewhere will show up to buy infrastructure versus having a disciplined approach to building a book of business. That is based on this secular shift with AI. We are big believers in it. But we're also believers that Microsoft's in here to do a pretty unique, different job. It's not like we're chasing a new cloud.

32:03I'm not trying to build a NeoCloud business. I'm trying to build a hyperscale business that scales into an agent business, and that's the infra and agent segment.

32:12Alex Heath:I'm really curious how you're thinking about the model layer of Microsoft. I recently caught up with Mustafa, and he's building your models. My understanding is the OpenAI partnership had some constraints on your ability to do certain levels of frontier model work, and now you're free to go, and you are. And we've been talking about, you know, you need to be the place where you can have all the models for your customers. How does that square with Microsoft also pushing to go to the frontier? And what's your prediction for being at the frontier? Yeah, so I think, first of all, you know, again, the thing that we still continue to be very, very thrilled about is the progress OpenAI continues to make with their models.

32:56And the fact that we have that access till 32 is a massive advantage for us. And we use it all over the place in Copilot. You see it and what have you. And we serve it in Foundry. And that will continue all the way. And in parallel, we are thrilled about the progress we are making with MAI, right? Which is so the models that Mustafa and team are building, whether it's in cyber, whether it's in code, whether it's in knowledge work, underneath. In fact, the other thing that's very cool is underneath auto, lots of MAI model usage. We are in our RLEs, in our data pipelines, we are sort of getting that data, training our own models and hill climbing, by the way, from the very ground up.

33:39And that will continue. And that said, at any given point in time, Copilot will still have all the other models, right? Because customers will expect that, right? Customers are not going to want any one of our products and say, hey, the only thing I'm getting is the model and the product to be one. We are fundamental believers in the harness and the models and the memory and the context have to be separable. In fact, my entire formula for an enterprise is that you should have your own benchmarks versus just these benchmarks that are all saturated. The real world benchmark that matters is your own.

34:17If you have your private eval, then you want to be able to test with all of the models that best meets your needs and then substitute any model, right? Because that will also give you assurance that if a particular model went away, your ability to actually have that eval still stay and, in fact, continue to climb doesn't go away. And so that's how we're designing our products like Copilot. that's what we expect the any agent system of our customers to be built. And that's at least the philosophically how we'll approach it.

34:49Alex Heath:So if I'm hearing you correctly, the reason that you and Mustafa are pushing at the frontier is for Microsoft's own needs as well. Not so much because we want to have something where we can cut off other models. And we're the sole model source. That's correct. So I think that we will have, we will be at a frontier with certain things that we uniquely can do with our data loops, our customer expectations of it. But at the same time, as again, a platform provider, any one of our products will also have all the other models and will design, as I said, our harness, context, memory, and model loops in such a way that that heterogeneity is maintained.

35:27Alex Heath:You recently said something that I really agree with, that, quote, if the AI we build is not helping humanity and under human control, it's not worth pursuing. And Mustafa has a post, which I encourage anyone listening to this to read, I think it's very important, really warning against the anthropomorphizing, I guess, of models that Anthropic is doing specifically and the dangers of that. And I'd be curious to hear you talk about that and what you're seeing and what's giving you concern there. Yeah, I mean, at some level, it's a bit of common sense, I guess, but it's worth saying, which is to say that, you know, anything that's not serving humans or in human control is not worth pursuing.

36:06I mean, none of us, whether it's anthropic or us or anyone would say that that's not our goal. So then I think there are finer points on this, right, which is the point that Mustafa is making. I think it's a very good one, which is, hey, let's not anthropomorphize AI and then even try to train it to have what we think of as human values. So maybe that's, in fact, what will get it to never be aligned. And so maybe we should take a different approach to this humanist AI code of conduct and use it more as a training regime for alignment. But I think fundamentally, I think on the AI safety, where I come out is, look, I think we should take all of these things seriously.

37:00But we should start with what we need to really first do. One is let's make sure that the bad actors who have access to AI don't do bad things. That's where I think we can do a lot to help us with the diffusion of this technology. There's many, many techniques like having KYC enforced and regular cyber practices enforced. The second thing is we also know that these models need active monitoring, not just during training or RL runs, but even at runtime. And so right now, essentially, let's face it, any of these long running agents can be considered an insider risk because they're persistent. And so therefore, the ability to have observability and then governance around that and essentially real-time behavioral monitoring of agent behavior is going to be very, very critical.

37:55And so that's containment is a word I think we will have to get comfortable with. There are technical solutions for it, and we should do all of that. The third is the hard part, which is how do we take this new experimental science called, as Jakob from OpenAI wrote, growing AI, not building AI, and make sure that the experiments don't go awry? And even there, I think there are other fields from which we can learn, like in biology and others. Realize that the stakes, as they get higher, we should be much more careful. That's where I think, in fact, OpenAI and Microsoft, forever since the beginning of our relationship, essentially had these embedded evaluators.

38:40We have had a safety board that actually monitors and is the gatekeeper of any new release. And so I think having even a broader approach to these evaluations, I think, would be a fantastic thing. Does that speak to hugging face then, the hugging face bridge? Yeah. I mean, like, so when -

39:00Alex Heath:I'm curious what you thought of when you saw that, given the relationship with OpenAI. Yeah. I mean, so obviously it was a challenging thing, right? So which is if you have, again, even some of these things you could say start off as, oh, wow, that's just a DevOps, you know, misconfiguration of having, you know, internet access. That can be fixed. But this idea that swarms of agents can go to work and do deceptive action, where did that come from? How did it get trained? What was the data mix that led to? These are the science problems. With that, now we have to take seriously, right? So therefore, I think, first of all, I mean, scaling laws are working.

39:41And so the bottom line is, will alignment fall out as a natural outcome of the scaling or not? That's what the folks are essentially questioning, right? When I think about the memos from Anthropic or OpenAI, they're not questioning scaling laws because empirically they're seeing the capability jumps. But alignment as we think of it or need it is not arriving. So that's a challenge and we should take it seriously. And so, and as I've always said, when you have a showstopper bug, you stop the show. Um, and so that's at least, at least how I would try to look at it.

40:25Alex Heath:Your colleague, I think I saw Brad Smith say recently that you all back the concept of an independent evaluator companies in Washington, they're trying to figure out how do we regulate this space if at all. But I've been thinking too, I mean, is, is liability not enough? Is the fact that, you know, it was hugging face and luckily it wasn't catastrophic and Clem is cool and, you know, it's the industry and everyone's friends. But had that been a giant bank, I think maybe that would have been an appropriate enough. The incentives of that dynamic that already exist with liability maybe would have been enough to correct the situation for the future.

40:59Alex Heath:I'm curious how you're thinking about that. Do we need do we need a new regime? Yeah, I mean, I think these all have to be thought through. You can. Like, for example, I think even the president, I think, has sort of talked about that, which is he said, hey, there are liability laws and we'll enforce them. That could have a chilling effect, right? I mean, in the sense, if you really are going to enforce liability laws on what is an experimental science that can have great benefit if diffused right, but one mistake, you're out of business because the liability is high, then the right thing to do would be to stop.

41:35And if that is what we want, then we should say that. And that's essentially a proxy policy, right? Which is who, like, I mean, think about it. Tomorrow, if you said, well, you know, here are the things. You scale, compute, you will have more misaligned AI, right? If that is what is empirical, then, yeah, it's game over. So stop now. And so, you know, anyone building data centers, you know, you may want to think again. So I think one has to sort of complete the thought exactly, look, do we want the benefits of this and then mitigate the risks? How do you create the right incentive structure? How do you pace it?

42:18How do you take the time to evaluate? What is a risk-based regime you can have? What is the monitoring you can have when deployed? So there's a thousand things one can do versus using blunt instruments, at least in my mind.

42:31Alex Heath:Yeah, I mean, to be frank, I'm worried about regulatory capture. I think, I mean, it's a concern. I think, you know, in my other job, invest in startups, and I think there's legitimate concerns that the drawbridge could be taken up. And I'd be curious to hear you say more definitively or not, like, do you think we need some kind of new body? I mean, Demis has proposed that and others are talking about it. Yeah, I think the regulatory capture is not obviously the thing that I'm for, right? Which is anybody who sort of says, hey, this is a way to have some kind of a cartel-like arrangement is a terrible idea.

43:05At the same time, does the government or our society demand that there is a certain set of rules that govern safe deployment of this? Absolutely. And so between there is the nuance, right? The nuance here would be, yes, there should be liability. but at the same time, it's better to have a set of, you know, like these embedded evaluators or whatever that's a broad, it's not just about five friends getting together and evaluating each other. It is about having a broad industry body that has got even people from startups that doesn't punish a startup from being able to or to increase the cost of a startup from being able to get to the frontier or what have you.

43:50So those are all the things that I think we will have to think through.

43:53Alex Heath:Do you think the industry has done a bad job of explaining the benefits of AI relative to the concerns that people have? Yeah, I think if I have to grade us as an industry, I think, you know, we should have focused a lot more on, hey, we're building a bunch of new technology. Let the people using our technology speak to the benefits of it. I think we are way too self-obsessed as an industry about sort of looking at us, how glorious we are, and then we go off. And I think a little bit of that, I think, is what's not working, right? Because I think the real world wants to know a couple of things.

44:41One, that this is technology that they can use for their benefit. They can control. They can have an economic future. These data centers that may be coming to their communities are actually going to create economic surplus, like our Quincy Washington 20 years of history shows it can. But it has to be real for them. Any amount that I say or any one of us say is not good enough anymore, I feel. And so I think we've not given it breathing space. In fact, it was interesting. I was reading the Gallup poll on AI. It's not good. It's not good in the West. it's also very interesting. So it's not a uniform thing around the world.

45:24And in fact, that's the thing we should ask. What did we get wrong in the West that these other countries may not have, right? Why are people in Nigeria more optimistic about AI than in the United States? And I think we should reflect on it. And my belief here is do the hard work as an industry to earn the trust, show the benefits to both consumers and enterprises and the communities. And then I think, you know, we'll be fine. But without it, you know, just any amount of just celebrating technology for technology's sake is not working.

45:58Alex Heath:And I do wonder if in the U.S. there's maybe some astroturfing going on and some optics warfare. And it's hard to peel back what is legitimate and what is being pushed. It's a great point. I don't know. I mean, this is where, you know, I'm sure some of that is happening. But there is something broadly. Like when, you know, when you look at students in, you know, at a graduation booing every time AI is uttered by the commencement speaker, I think it speaks to the anxiety because they're all using AI. I'm sure they like using AI, except they're worried about something. And I think we should really come to terms with it.

46:39I think it's like, hey, what's this job opportunity, the economic opportunity, their future? The more we can show that, in fact, there is going to be more opportunity as opposed to the vast inequality or concentration of power or what have you. That, I think, is the work ahead.

46:59Alex Heath:Microsoft is a large company. You have many businesses. With the limited time we have, I have to check in with you on the state of Xbox. It's going through a lot of transformation. How do you feel about where it's at and what you see coming? Yeah, I mean, look, I think Xbox, in fact, I always say at Microsoft, we've had gaming. In fact, I think we've had gaming even before as a category before Windows. Right, Flight Simulator. That's right, Flight Simulator. And so to me, it's in the same core DNA like developer tools and knowledge work. And I feel fantastic about the IP we have right now, which is if I look at the studios, the IP portfolio we have and our ability to then take that and produce great games going forward, I feel fantastic.

47:51There's some amount of streamlining the team is doing and Asha is doing, which is great to see. And then we have to invent the right sustainable business model that allows us to deliver gaming to more and more people. That has always been the goal, which is we want to be a great publisher and a great platform provider for games across both PCs and Xboxes. And so that is sort of our goal. And I think Asha and team have said that Xbox is going to get back to growth this next fiscal year. And that's the plan they're executing on. And in that process, do a bang up job of producing some great games.

48:34Alex Heath:I can't believe we've barely touched on this and we're coming to a close here. But the thing underpinning so much of Microsoft's success for decades, Windows itself, going back full circle to talking about agents. How do you think agents are going to change the trajectory of Windows? Oh, it's a great point. I mean, to me, I'm very, very excited about what I think is going to be the next Revo Windows. First, you know, I would say just like to your Xbox, one thing I want us to be staying very, very focused on the Windows team is even doing the basics right and the fundamentals right, the quality of it, everything from its updates to driver quality to perf and everything else.

49:13So, you know, in fact, using even, like I look at what's happening in cyber and our patch Tuesdays and so on. It's fantastic. We have an unbelievable use of these new tools to do a much better job on the handling of the security aspects of it. But beyond that, you're right in saying that my vision for Windows is simply our ability to do unmetered intelligence. Just imagine having a Windows box and one of these hybrid routers. There's a thing called HydroFusion, which is in GitHub today in circulation, so GitHub Copilot, where you can essentially have a router that uses an onboard agent or onboard model, on-device model, and then goes to the cloud and then routes automatically.

50:02And so if you're using GitHub Copilot and you want credits, you have unmetered intelligence every time you use your own computer for it. And that, I think, is the right vision. It's no longer about, oh, I have a completely local thing and I want to buy a$100 ,000,$30 ,000 workstation, or I want to buy cloud credits. I think having a Windows device that automatically labels a programming model for unmetered intelligence to be part of your token usage is what we would want to achieve.

50:34Alex Heath:Last question, and it's a quick two-parter here. If you're looking at the totality of Microsoft's business in the next couple of years ahead of you, what is, A, the greatest risk you see that you have to navigate through and the team has to really deliver through? And what is the biggest opportunity? Yeah, I mean, I think the greatest opportunity is clear, which is agents and what they entail, both for our infrastructure business and our application business, is, as I said, massive TAM expansion, right? So it's going to be orders of magnitude more than anything we did in the previous era. And therein lies even the risk and the challenge, which is you kind of have to build from first principles systems and form factors.

51:22When we brought office into co-pilot and autopilot, it's not about taking office as we built it the last decade, but to reshape it, to bring it in such a way that it can be discovered. In fact, one of the things we want to do is it's not about default stuffing, right? The model has to prefer the usage of this tool to do a particular trajectory. It has to be earned. And that is the process that we are going through. And therein lies both the opportunity and the challenge. But I think we're at it and we're making great progress. Satya Nadella, thank you. Thank you so much, Alex.

52:07Alex Heath:Banking should feel like modern software. Get everything you need in one place. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech, not a bank. Check the show notes for details. Granola is the best AI notepad I've tried. It works everywhere, on a video or phone call, in person, or an Apple Watch. Try it now at granola.ai.sources and use the promo code SOURCES at checkout for three months off. Jira by Atlassian is where your team and your agents work from the same context. Try it free at jira.com. That's J-I-R-A dot com. Framer is the AI native website builder that lets you build faster without giving up control.

52:44Alex Heath:Visit framer.com slash sources for 30 % off. Rules and restrictions may apply.

53:07Thank you.