Satya Nadella: Microsoft's AI Bets, Hyperscaling, Quantum Computing Breakthroughs

0:00What are the tools that we can put in the hands of people that will give them that sense of empowerment? that's what I would love to work on. I'm not into this anthropomorphizing AI at all. I come at it as it's a tool. There is going to be a job called a software engineer. It's going to be different, but I look at it, you are really taking a software engineer and saying, you're now a software architect.

0:28It's my pleasure to welcome the chairman and CEO of Microsoft, Satya Nadella.

0:40Thank you.

0:48This is the home crowd. All right. Man, San Francisco, you should move to Seattle.

0:56I started my career in Seattle. There you go. It's a very fantastic place. Anyone who's successful starts at Microsoft. That's right. So, Satya, you've emphasized before that AI is going to shape all that we do. What does this look like in practice? At Microsoft, how does this actually drive your strategy and particularly thinking about how AI will influence ideas beyond the immediate incredible product suite, like the broader economy? At Microsoft, I feel we are a platform company. a product company, and a partner company. So I think of those three dimensions. And I've kind of, in my 35 years, I've lived through client-client server, web internet, mobile cloud.

1:49This is the fourth. So that's just at least how I pattern match. So the first thing that I think about is the platform opportunity. When I sort of look at all the folks here, The interesting thing is the compounding effects of all these platforms. So this AI piece, the reason why I think the rate of diffusion is so fast and so wide is because it builds on the previous generation. I think about it. If the cloud was not there, we wouldn't have been able to build the AI supercomputers, which then led to the models, which then led to the products. So that compounding effect is the interesting thing to me.

2:33So that's why you always sort of take the previous platform and build the next platform and you want to be able to get that right. And then you've got to build the next generation products on top of it. With each one of these platforms shift, there's a new workload. I mean, when I first remember looking at the large-scale training job, I mean, it's kind of a very different workload to what we built, for example, the CloudWid, right? It's a data parallel, synchronous workload, which is so different than, let's say, a Hadoop job or what have you. And so the platform itself then completely gets re-litigated and changed.

3:11So to me, that's, I think, the exciting thing on the platform side. It's golden age of system software. We are quite frankly, you know, today, if I had to think about anybody who was building at the infrastructure layer, not just the hyperscalers, but even the startups, I think that's a tremendous opportunity. Obviously, there's a tremendous opportunity in the model side and then the products on top of it. So, yeah, we think of these and then ultimately, what's it for? It's for one thing and one thing alone, which is to drive ultimately economic growth and GDP growth. So if I had to ask me my benchmark for AI is, is it creating surplus in the world around us, one community, one country, one industry, one company at a time?

3:55I mean, it seems like the app level, you know, you guys have built sort of the defining apps at the app layer for so many decades. It feels like we're at this weird lumpy moment where maybe the models have popped up and we're sort of astonished by what's happening. but then sort of the compute and the apps need to actually catch up. And the hope here is actually the people in this room will be the people who build those apps. Yeah, it's a good question, right? One of the questions is, is the model like SQL or is it the SaaS app itself and the model, right? I mean, I think the place where does the model end and where does the product begin because if you sort of say model with some scaffolding and tool calling in some infinite loop is the product, if that is what it is, then I think that that's where it gets a little confusing.

4:55But that's like saying a bunch of SQL business logic is with SQL is what is an app. So I think it's still possible for anyone to build an app tier on top of a model. And you have to sort of abstract yourself and say, yeah, the model is just like SQL was to me. And so I think that, I mean, I always dreamt of a moment when AI slash machine learning will have a SQL moment. Because if you think about it, we never had a stable platform layer in the past because everything was vertically built and integrated. For the first time in this model layer now, we have something like a SQL engine that we can then use to build pretty sophisticated products.

5:41And these techniques also, right? I mean, just the inference time compute plus tool calling is giving us, I think, a pretty robust harness to be able to build pretty sophisticated products. Yeah. It's kind of wild how much it's the integration piece that is also the app layer now. Yeah. It's just, you know, the models sitting on their own are incredibly smart, but right now they feel, you know, sort of there's just a giant gulf between that and the data that really matters to you for business users. Yeah, I think that's a good observation because I think at least my read of the situation is the model is an important piece.

6:20The model scaffolding and all this tool calling, there's a real app server that you kind of need in order to be able to build sophisticated applications. But the interesting thing is the feedback loop, the data path inside the product that then is used in order to post-train, in order to be able to do the right tool selection, that seems to be the place where product creation is all going to happen. AI scaling laws are continuing to hold, and the demand for intelligence appears to be potentially infinite. it. Yesterday, Elon was mentioning that there will be 99 hyper-intelligent beings to one human, which is kind of a wild prognostication, but seems possible.

7:08Given this, where does the building for the future of AI truly demand for global compute infrastructure? How do you anticipate these demands evolving as models don't just become larger, but more intelligent and capable of complex multi-agent interactions? Yeah, I mean, look, if you sort of really step back and say, you know, first, if you sort of go with the, you know, compute or intelligence is whatever, log of compute. And then you ask the question, how much energy does compute consume? Let's take in the United States, maybe 2 % today, 3 % tops, let's say doubles, it's 6%. That's like massive because then the amount of extra energy that needs to be produced in order for AI to use it is pretty high.

8:02I think that's why we all have to sort of keep in mind that if there's one lesson history has taught us is that if you're going to use energy, you better have social permission to use energy. So that means you've got to make sure that the output of this AI is socially useful. In other words, if we really are not creating social surplus, economic surplus, as measured by countries and communities, then we just can't consume energy. And so that, to me, is the bigger thing. Like everybody's today hot and bothered about, okay, what do I do about energy production? I think the real question in the next five years is we've got to produce enough products that are creating great value, which I'm very confident of, by the way, in healthcare and education, in productivity.

8:53So there's many, many domains. But that's the real challenge for us as a tech industry, is to prove unequivocally that what we have created is showing up in real stats that is not just an AGI or AI benchmark. I mean, the hope is that this will show up in sort of the real things that you sort of interact with on a daily basis. 100%. You know, you go use, you get a mortgage loan and instead of, you know, three months or two months of waiting around and you don't know if you're going to get approved or, you know, there's just so many things that are important parts of your life that, you know, get drowned in paperwork or bureaucracy that those things could potentially go away.

9:40100 percent so i think yeah if you take even take some of the the public services right i mean if you take any country you know it's gdp or take health care like in the united states what is it 18 19 percent of our cost is uh health care and a lot of it like everybody talks about the magical drug blah blah blah except all of the cost is in workflow uh and so if you really take something like a simple thing like discharge, the amount, like you take the back end of an EMR system with just an LLM and a prompt, that itself is going to save so much time and money and energy that it would sort of pay for itself.

10:23I mean, it's kind of very direct, right? You spend an incredible amount of GDP on healthcare and rightly so, but every dollar that's spent on clerical work could have been spent towards some sort of treatment that would have saved someone's life. Or the simple time allocation of a physician away from paperwork to the patient is right there to be had. What do you see as being the biggest rate limiter for AI deployment today? See, here's the interesting thing, right? This audience is so young that none of my metaphors would sort of work. But nevertheless, if you sort of came in in the early part of, you know, let's say you were a multinational company, pre-PCs.

11:08How the heck did we do forecasts? Like a simple sales forecast. The way one would do sales forecast was you would send faxes. People would then take those faxes and send inter-office memos. and those inter-office memos would be annotated and a forecast would come hopefully before the quarter end. And then suddenly people said with email and PCs and Excel, they said, let me just send an Excel spreadsheet in email, people enter number and you have a forecast. So what happened was the work, the work artifact and the workflow changed. That is what needs to happen with AI. When someone says, I'm going to now do my job, but with whatever, 99 agents that I am directing on my behalf, the workflow is not going to be constant, right?

12:01I mean, you now are really going to have to change even the scope of your job is going to change. So that change management is the real rate limit, right? Because you're now taking the means of production in an insurance company, in a financial services company, in a healthcare company, in a software company, and saying, we are going to change everything in the way we work. In fact, we're going to change what jobs they are. At LinkedIn, I think they took multiple of these functions, the design function, the front-end engineer function, the product function, put them all together and said, we're going to have full stack builders.

12:39That's a change in scope of even a job. And so how do you then rebuild the product team with new roles, new scopes, and what have you, that's to me more the social rate limiter. Not that there's lots of other things that are on deployment of this technology, getting it out to the world, power is one, there are other issues. But I would say change management. Now, when I look at even a lot of the AI startups, when I talk to them, everyone has now, you worked at Palantir, so you know this, everyone has forward deployment engineers. That's like the exciting thing is the Palantir model, which I think is a fantastic model.

13:15And why is that? That is because of that change management. Because I think you really need to help customers, partners, really understand the benefits of any product you're creating, but not just the technology, but even how to use the technology in a workflow. At YC, we have this funny saying that we tell a lot of people here to do, which is, you know, these are some of the smartest AI researchers, computer scientists who are just starting out in their careers. We tell them, go undercover. So go work as a medical biller and see to what degree, how many, you know, quote unquote, knowledge work jobs are actually copying, pasting from a browser into a spreadsheet, into an email and then clicking send.

13:59And do that for a while and realize, like, actually, these are not necessarily, you know, using your prefrontal cortex and your highest mind kind of jobs. Like these are not, you know, can you imagine so many people like their lives are basically like, you know, we used to, you know, coming up at our age, we would call it paper pushing. But, you know, they're not paper pushing anymore, but they're sending emails. You know, they're not sending faxes anymore, but they're trying to get business done by like attaching files to things. You know, that seems like a pretty big shift, actually. Yeah, I mean, I think one of the most understated things as an opportunity for anyone creating products or fundamental breakthroughs, even at the model layer, is the amount of drudgery there is in knowledge work, right?

14:47I mean, in software engineering, we saw that. I mean, the amount of, you know, we're taking the joy out of software engineering because, you know, you are out of your flow. to be able to stay in the flow, to be able to complete a task. That itself is a great example of what I think is going to happen to all knowledge work. You're absolutely right. The amount of cycles you spend out of band collecting information, because if you think about the prefrontal cortex and the synthesis part, the amount of time you spend there is pretty low. Now, having a sophisticated reasoning model and your prefrontal cortex work together, Whereas a lot of the mundane stuff is getting done by even some core agent or what have you.

15:27That, I think, is definitely the frontier. So beyond simply adopting AI tools, what are the biggest transformational shifts you're seeing in the field today? I think, to me, even like, I mean, look, this field is changing so rapidly, right? I had not even imagined last year, even this time, that we would get this far with RL and with basically test time compute. And it seems sort of pretty limitless. So the way I think about it is pre-training worked. All the post-training techniques on top of it were fantastic. Then this inference time compute seems to have really added. in another massive scaling law.

16:16So now I'm interested in whether there is some new algorithmic breakthrough because I always say this entire regime could be changed by one person here who comes up and says, I have a more efficient thing to do or a way to do this stuff, right? So you have to be open-minded that the last big breakthrough algorithmically has not yet been found. So that's one. I'm always sort of interested in that. The other one is, what is the next step up, right? Because what is the pre-training to RL, the end-to-end training loop that's the next, you know, big sample? That, I think, is also what I think will happen in the next year.

16:59So I would say if that is another scaling law breakthrough, because we will be, like, if you sort of take any lab now, all of us I think will be working on saying what's a more integrated response reasoning model that we can build. And that I think is going to be the interesting piece. There's something very interesting here I think in that if you think about like an LLM instance as a consciousness which I think some people are starting to say you know it's sort of instantiated you do a bunch of work with it and then it sort of goes away and you open a new chat box and it's you know But I guess I'm curious, like, do you think that that might be one of the things that needs, the loop needs to be completed, right?

17:44Yeah, I mean, so I'm not sort of, I don't, to me, this artificial intelligence is unfortunately the worst name we could have ever picked. And so I'm not into this anthropomorphizing AI at all. I mean, I think of it more, I come at it as it's a tool. It's not trying to replicate how we think. And it's definitely showing signs of intelligence, but it's not intelligence that I have. And I think of human agency still will matter, will be there, and we will sort of use these as tools. So that's kind of my position. That said, let's just say, oh, yeah, a memory system is a good thing. These things do need.

18:22If I look at the next frontier, I would say there are three things, right? One is memory. The other one is tools used. And then the third, which I think is perhaps the most important thing, is entitlements, which is basically if I'm going to take action, what entitlements do I have to take action? So these three systems have to be built as first class around the model in order for us to build more sophisticated applications. One of the arguments people are starting to make around the future of software is, well, we have the database, and then you're going to have basically middleware that is, you know, I think, you know, what you call entitlements.

19:03It's kind of like access control list. That's it. Like, you know, what's the business logic? Who gets to do what? And then, you know, you basically put the agent on top of there. Is that sort of the future? That's right. So that's why I think about, like, you know, people say when you think about the scaffolding layer, right, you have a model plus scaffolding. The scaffolding now really gets first class by thinking of these three things. Tools use is one, memory is one, and then entitlements. And you put that stuff together. Then you can create an agent. An agent has an ID. Agent has management and provision control on it.

19:38So that's the way, I think, to think about it. Do you worry with CodeGen? Do you think users at some point will just prefer to make software just in time instead of using package software? I mean, that's something that, you know, we're having lots of conversations in the hallway about that. Because a lot of us in this room, you know, YC will actually fund a ton of SaaS and will continue to do so. But, you know, in the background, we're starting to have that worry. There's some venture capitalist friends of mine who are in the audience. They're actually like, I actually don't know if I can continue to fund B2B SaaS.

20:14How do you think about that? Yeah, it's a great question. I mean, you know, it's interesting. At the same time, I look at the number of people who are forking VS Code and I say, man, we must have done something right.

20:29So therefore, there is something to be said about building a great IDE. In fact, when I think about Excel, I think of it as an IDE. So the fact that there's a great canvas, you can then bring, let's call it, the best analyst model to this IDE and then create a loop between the canvas and the model. So I think, yes, you can generate applications just in time. You could have a prefab application that is really helping with the feedback loop to the model. And I think both of these things will exist together. Do you think there's a role for design in all of this? I mean, basically, a human being sitting in front of VS Code is sort of like the translator between the software and what the end user really wants.

21:24And then I think some of this idea that software goes away presupposes that just normal people walking around are going to want to create software. And I don't know if that's going to work. I think that's a good point. So I think the way I sort of say, because one of the basic questions you're asking is what happens to software engineering, right? I mean, that's the, let's take the following thought experiment, right? If you sort of said some Martian intelligence came in the 1980s and watched how we all work, they'd say, oh, wow, these humans kind of work in the offices and they have a typist pool, they have a slide pool, and people then work with paper.

22:06And then if they came back today, they'll say, God, man, all 8 billion people are typists now, right? I mean, that's what they'll sort of, you know, surmise. And so I think what I think will happen is all of us are going to be creating software, but there is going to be a job called a software engineer. It's going to be different. But I look at it, right, you are really taking a software engineer and saying you're now a software architect, right? I still, the metacognition of your, I mean, one of my biggest things is, man, the wipe coding is fantastic until it does stuff that I don't know what the heck happened.

22:41So that means I have to have the metamodel of my repo and exactly what happened. And I'm looking at the change logs, right? So when I look at my favorite feature of GitHub now is to really look at the complete change logs of all the agents that are working on my repo. And I think that is where a lot of the software engineering will be, like a good dev manager. I don't know which dev manager you worked at Microsoft, but I really looked. And a dev manager's job was to make sure builds don't break and the code has got good quality. And so to me, that is still a thing. And so there will be a level of abstraction uplift, even in a world of all AI agents.

23:25Because one thing that we don't talk about is the legal liability, by the way, until some real laws change, are going to be with humans and institutions humans build. And as long as that is true, we're going to have to really make sure the human is in the loop at a fundamental level. And that means we will need a lot of tools for humans to be in the loop in order to figure out what these things are doing. In AI development, you see so much. What do you think is underestimated and what is overhyped by the broader tech industry from where you're sitting? It's not short of overhyping. Let me put it that way, right?

24:01We're at the, everything is AI all the time. So it's good. You know, for us all in this industry, we live and die by our ability to get into a frenzy about something new, right? What is the Steve Jobs thing or the Bob Dylan thing, which is you're either busy being born or busy dying. It's better to be being busy being born. So that's good. I think the thing that we have to most worry about and most work on as a tech community, I would say, is that how do we earn that social permission? If there is one thing that I feel, to me, one of the demos I saw, which completely really blew me away, was I think in the beginning of 23, when I was in India and I saw a local developer, Daisy Chain, essentially at that time, either GPT-3 or 3.5, with one of these India stacks, speech-to-text, text-to-speech, open-source things, and then showed a local Indian farmer who was able to sort of use a chatbot that was built in WhatsApp to be able to get some agricultural subsidy, right, by going to a government website.

25:14That, to me, was unbelievable. I felt like, man, how could something that was built in the West Coast of the United States get to a real use case that fast, thanks to sort of the diffusion rate and basically people everywhere. That is the story that needs to be told, right, at scale. That is the under-hyped story, I think. Because right now, the over-hyped thing is the model capability. And the model capability is fantastic. But man, if we can somehow get the world to recognize that this is making a real difference in the lives of people everywhere, we're in good shape. If that doesn't happen, this is all about some valuations of us, our companies and our industry, and it's the same repeat, then that is not going to end well.

26:06I love that example. I mean, you can, I don't know, it feels like Microsoft is sort of full of examples of things that lower the floor so that a lot more people can get access to technology. I mean, you could argue GitHub Copilot is one of the biggest pieces. Yeah. By the way, one of the other ones, just you brought it up, there was a World Bank study they did, I think in Nigeria, and now they've taken it to Peru or Chile, one of those in South America. We've been working at Microsoft forever on can there be an intervention in education? That's been the dream. Man, we've been at it, at it for decade after decade.

26:43It's made a difference. But this study said by access to something like a co-pilot is probably the best tech intervention in education in Africa or in Latin America. And that's, you know, that's been the dream, I think, that we've all had in tech. And it's right there within our grasp. I guess, are there any interesting observations? I'm curious because your co-pilot in Windows is often here in tech, maybe people are really obsessed with the latest frontier models, but it's easy to forget. Windows and the integration with Windows is actually the first interaction people have with pre-AGI sort of AI today.