20VC: Microsoft CTO on Where Value Accrues in an AI World | Why Scaling Laws are BS | An Evaluation of Deepseek and How We Underestimate the Chinese | The Future of Software Development and The Future of Agents with Kevin Scott

0:00This is the best time to be alive if you have an entrepreneurial spirit. I can very clearly see what we're doing now and what we're doing next and I don't see the limit to the scaling laws. Don't believe in this one agent for everything sort of theory. I think you'll have a lot of agents and the reason I think you're gonna have a lot of agencies because your product managers are probably going to have to be domain experts. The agents definitely, they will definitely be less transactional, less session oriented going forward. You are listening to 20VC with me Harry Stebings. Now I think the show's day for me just really emphasizes the power of the internet.

0:39I started 20VC with nothing from a bedroom in London as a 17 year old, and today I sit down with the CTO of Microsoft, one of the world's most valuable companies and leaders in AI. So I'm thrilled to welcome Kevin Scott, the man responsible for AI and technology at Microsoft, where he played a pivotal role in Microsoft's partnership with OpenAI. This is an incredible discussion on scaling laws, the future of inference, data, compute, where we go from here, it's time to get the notebooks out and this was such a joy to do. But before we dive in today, turning your back of a napkin idea into a billion dollar startup requires countless hours of collaboration and teamwork.

1:19It can be really difficult to build a team that's aligned on everything from values to workflow. But that's exactly what Coda was made to do. Coda is an all -in -one collaborative workspace that started as a napkin sketch. Now, just five years since launching in beta, Coda has helped 50 ,000 teams all over the world get on the same page. Now at 20VC, we've used Coda to bring structure to our content planning and episode prep, and it's made a huge difference. Instead of bouncing between different tools, we can keep everything from guest research to scheduling and notes all in one place, which saves us so much time.

1:55With Coda, you get the flexibility of docs, the structure, spreadsheets, and the power of applications all built for enterprise, and it's got the intelligence of AI, which makes it even more awesome. If you're a startup team looking to increase alignment and agility, Coda can help you move from planning to execution in record time. To try it for yourself, go to coder .io -2 -0 -VC today and get six free months of the team plan for startups, that's coder .io, slash 20VC to get started for free and get six free months of the team plan. And while coder keeps the engine running smoothly, Shopify puts the pedal to the metal when it's time to sell.

2:33I spend my time looking into successful businesses of today and tomorrow. And often, there's a business that's behind the business, helping drive success. For millions, that is Shopify. Shopify is home to the number one checkout on the planet, boosting conversions by up to an astonishing 50%, meaning way less cars going abandoned and way more sales going into the business. Winner. So if you're into growing your business, your commerce platform better be ready to sell wherever your customers are. Go to Shopify .com slash 20VC to start your $1 per month trial today at Shopify .com slash 20VC in letters.

3:11And while Shopify helps you drive sales, don't forget what really keeps those customers is coming back. Trust. Trust isn't just earned though, it's demanded. That's why over 9 ,000 companies, including at Lassian, Cora and Factory, rely on Vanta to automate their security compliance. So Vanta helps businesses achieve certifications like SOC 2 and ISO 27001, turning months of tedious work into this beautifully fast and straightforward process. Their platform automates compliance across over 35 frameworks. It centralizes workflows flows and it proactively manages risk, all while saving you time with automation and AI.

3:48So whether you're just starting or scaling your security program, Vanta connects you with auditors and experts to get audit ready quickly and build trust with your customers. Get $1 ,000 off your first year by visiting vanta .com -4 -20VC. That's v -a -n -t -a .com -4 -20VC. You have now arrived at your destination. Kevin, I am so excited for this. I was just telling you, I was listening to you and Shrap on my run. I don't think I've ever run quite as fast, which clearly means the conversation was brilliant. Either brilliant or awful in the huge trying to end your run, so you can be done with it.

4:24I've never done a 10k so fast. I wanted to start with a super easy question. My job as a venture investor is to try and determine where value lies in different given moments. And I look at the world today and for the first time in quite a long time, Kevin, I don't know. And my question to you is in this next generation of AI, where does value lie sustainably? Do you think? Yeah, I don't know. I think the thing that you just described, which is like all of a sudden things have gotten a little less clear than they have been, is exactly the thing that happens at the beginning of every big technological paradigm shift and every new cycle that's driven by it.

5:10So it was super confusing in the early days of the internet and I think it was super confusing in the early days of mobile where everybody had these ideas about what was going to be valuable and very few of those ideas were actually the durable ones that proved all the way through. In those moments of transition while there is this confusion, what if you learned is the right action to do? Is it to be active to iterate and learn but you'll make so many mistakes that you regret? Or should you sit on your hands and watch others make those mistakes? Oh God no, like you should definitely shouldn't do the latter.

5:43This is the best time to be alive if you have an entrepreneurial spirit. The thing I think that you have to do in these moments is not forget that the things that you've learned from the past moments about what works. And it's not like, okay, well, like it's do this specific thing, but it's like how you go about doing that exploration that you just described, which is, you know, product matters. I've been saying this for the past couple of years that models aren't products, because everybody was just so fascinated by the infrastructure itself. And like, this is also a characteristic of the beginning of these cycles, is you have technical people who get just swept up in the technical bits and they kind of forget that the only thing that really matters is making good product.

6:32And like that's where we're at right now. Like you have to make good product. And you know, you have to have ideas and have conviction and then you have to go get stuff done really fast so that you can see whether you're full of crap or not about the conviction that you have. And you have very few patterns at the beginning of a cycle to go snap to. like you're not looking at someone else's success and saying, okay, well, like I'm gonna do that, but just a little bit better. Like you're trying to figure out something completely new. And the only way to figure that out is like you got to launch stuff and gather data and iterate and you know, be super, super brutal with your own self about what you're seeing.

7:11Like you can't love your idea so much that you overlook what it is you're seeing about the data and the feedback that you're getting. You said their models on products. I just had Andrew from Cerebrus on the show. And I asked him this question. I said, if we think about compute or kind of hardware, and then we think about models and we think about apps, where does the value lie? And naturally he said compute. But when you think about that kind of three -pronged tier of value, and you said their models on products, if they're not products, does that mean they're not valuable? No, no, they're super valuable.

7:46but they're only valuable to the extent that you can connect them to things that users need via product. So like in the limit, I think product is the most important thing. If you build good models and you build good infrastructure around models and you have good efficient compute and like you have all of these other things, you're going to get lots of ability to monetize all of those things because as people build those products, like they will need to consume your platform and your infrastructure and like all of that's good. But yeah, I mean, like most of the value has to be in the products like, you know, we don't build infrastructure just for the sake of infrastructure.

8:22We build infrastructure so people can make products. This is a leading question then again, if you think about and I think you might know, but if you think about those products, who benefits most? Is it startups who are able to integrate new technologies very easily from the bottoms up, starting from nothing, or is it Microsoft integrating AI into incredible distribution already, Google doing the same. Who benefits most in that respect? Yeah, I don't know. Again, if you look to pass cycles, you've got a pretty good mix of where value gets created across startups and new ventures and existing enterprises.

9:00And so I think everybody's kind of doing the same thing, like you're trying to discover the new. If you're a big company like Microsoft, with a long tradition and a bunch of successful things already in the market, like the thing that you are trying to do is figure out what of the things that you already know super well and like which of the customers that you're already serving super well, can you do for them with this new set of capabilities that you can provide? Yeah, hopefully, you know, like I run among other things Microsoft research. And so like I also have a charter of like, you know, hey, can we go try to shine flashlights in places that no one else has shown them before and like try to discover some like super disruptive brand new things.

9:43But like that's kind of the job of the startup ecosystem as well. And I also am an angel investor and like I advise startups and I've worked at startups and I think it's just really important that you've got lots of people hunting for those interesting new things. And like I have super high conviction on that in the AI platform transition that we're going through right now because it's impossible for any entity like Microsoft or any other big company to have enough imagination and enough perspective to know what every interesting thing is. So having just this vibrant ecosystem, lots and lots of people sort of exploring, you know, where value exists, I think is incredibly exciting and necessary.

10:32And I also think that there has never been a moment where the tools, the infrastructure and the platforms are as cheap and accessible and available and easy to use as they are right now. So it's just super easy to like pick stuff up and just go get crack him. I was listening to your show, as I mentioned earlier, and you push back on the idea of reaching scaling laws and this kind of asymptote of efficiency or effectiveness. When many people suggest that we are hitting scaling more soon, first, why do you think we're not in that to ridiculous statement? I can very clearly see what we're doing now and like what we're doing next and I don't see the limit to the scaling laws.

11:18Like if you're just sort of thinking about the raw capability of the models and like how well you can condition them to reason over increasingly complicated things, like there at some point will be maybe a limit. I intuitively feel like there must be. There's some people who don't believe that there's a limit, that the limit that human beings have on intelligence is like you've got so many neurons like packed into your skull and like you've got about a 20 watt power envelope and like that's the limit. Some people believe that if you have AI's that there is no such limit and things will continue to scale into weird territory.

11:54I don't really, like that I don't necessarily believe. I believe we will get to some point where we'll hit a scaling asymptote and they'll just be diminishing marginal returns and it's so expensive that we will decide it's not worth spending that next dollar to make this thing one unit smarter because we haven't figured out how that translates into something that's useful for the people who are using the tool. I think that point will come. I don't just don't see it yet. It's not in the viewfinder right now. When we think about like three core elements that make up kind of efficiency in this respect, it's kind of data computing algorithms.

12:32When we drill into data, what are your biggest observations on data efficiency, the importance of quality of data versus quantity, synthetic versus human? and how do you think about that today? The mixes synthetic data is going up. High quality data is becoming much more useful, especially in the post -training parts of the model production pipeline than low quality data. So like I think we're clearly at the point now where if you have the right infrastructure and you have super high quality data and super high quality expert human feedback, you can amplify that into the right set of tokens for training bigger and bigger models.

13:14And like that stuff is way more value than just, you know, sort of the undifferentiated tokens that are, you know, like floating around on the web. What questions do we not know about data and its usage that we would like to know and would be most helpful to know? There's sort of a super interesting thing right now that we don't have around assessment. So it's very hard to know quantitatively what the incremental value of a token of data is to the quality of a model generated that uses it in its training. So in other words, if you're sort of asking, like, okay, well, I think my data is super valuable.

13:54And like if this gets used in a model, it's going to make the model better. Most of those assertions that people make are, I'm founded in any kind of science. And like the measurements we do have show that there's a pretty big disconnect around what some people think valuable data is versus like how valuable it actually is to producing capabilities and models that are legitimately useful and In most of what's legitimately useful is like models you people who think of models is Repositories of factual information and they're treating them like the world's worst and most expensive databases like It's not super useful.

14:33Like we got search indices and databases and those things are plenty good enough for retrieving information. Like what you want models to be able to do is to be able to reason over information. So like if you give them access to information, like how well are they able to reason over a set of information to go do something that's useful for you. And so you just need different tokens for training a model to make them good at reasoning than you do at making them, you know, recalers of facts. It's so funny you said about reasoning there, because it just made me think about kind of inference. I get really annoyed by the word inference.

15:08I wish we'd just like delete it and just call it usage. It's usage. There's training in there's usage. My question to you is you've been very clear about the transition of emphasis, importance from training, in which we had over the last few years, to inference. What are we not talking or seeing in inference that we need to spend or think more about? I think the thing that most people miss, although the deep seek R1 line show a few weeks back, clued everybody into it, is that we just have an incredible track record over the past handful of years. And it's many years now of just repeated year over year, mind boggling progress in optimizing the performance of models so that the performance of inference is just better and better and Over time, the models have gotten bigger and the API calls have gotten cheaper.

15:58A little bit of that is because you get 2X benefit price performance from hardware, every generation, like if you're lucky. But you get a much bigger improvement price performance wise from all of the things that you're doing in the software stack. Again, there's just a ton of work happening there. The deep -seek R1 stuff, which was good work, is the way you should think about it is is like a point on a line of price -performance improvement that maybe was invisible to everyone else, but not invisible to the people who are neck deep in optimizing these systems. And it's not the last point. It just marches on.

16:39What was the internal sentiment towards that when it came out? I was surprised at what the public reaction was. Why? Because we've had models more interesting than DeepSeek R1, that we chose not to even launch them. I was surprised at how interesting people thought that it was. They did good work. It was good, solid, technical work, and it was super cool. They chose to release this thing and make it open source. It's really interesting seeing how the public reacted to what they did. Is there anything that you learn from how the public reacted in the release that you take with you to your releases?

17:17Even when you've made it as easy and cheap as humanly possible for folks to go do something, they still have super strong preferences about the how. And so we're paying very close attention to that. We have to give people more how than we have been doing it because developers want lots and lots of choice. What did you believe that you now no longer believe? What have you changed your mind on in the last 12 -24 months? When I was a graduate student, I was like a complete open source zealot. You know, as I've gotten older, like I sort of have become a lot more pragmatic. And it's like, okay, well, like it's probably more important for me to make a set of pragmatic decisions about how it is I'm going to go build these things rather than singly optimizing for my curiosity.

18:09When you look forward to the next three to five years, three to ten years, How do you think about the pervasiveness of open versus closed and which will be more dominant than the other? I don't know. I think there's going to be lots of both. Let's just forget about AI, which is the controversial thing at the moment. It's a thing where industry structure hasn't settled yet and we don't know exactly what it's going to be. You just pick your previous things. Search, for instance. There's a whole bunch of open source search engine projects out there. and people who want to do search to have a search feature in their application or who want to go build a search engine themselves have lots of options.

18:53They can go grab something open source as a starting point. They can stand up a product. They can go load their data into something like Azure, cognitive search, which is a search as a service platform. Like Google has one, Amazon has one, like they're readily available. And then you still have search engines like being in Google that are out there. They all exist. All of the economics in search go to like somebody who stood up a gigantic infrastructure and who's sort of running like a whole search business, like with with its own feedback loop. Yeah, and so I think we're probably gonna have similar sorts of things happening here for the infrastructure layer.

19:36You're gonna have lots of open source infrastructure products and people are going to use them in lots of different ways. But like you're also gonna have a lot of people who don't want to have to go stand up their own infrastructure from scratch or to like take an open source project and go build it out where it's lacking for the things that they need it to do. It's good to live in a universe where you have both of those things. You mentioned earlier the centrality of products. Taking that into account with the current conversation here, how do we think about whether chat is the right UI for the next kind of paradigm of this product realm?

20:17Open AI and chat GBT has made it the default. To what extent do you think it is the right default and how we will see that change. Yeah, I don't know. I think it's a reasonable step in the right direction. The thing I've been saying for a few years now is, I think one of the most interesting things happening with AI is we've had one paradigm for using computing devices for effectively 200 years since Ada Lovelace wrote the first program. So if you want a computing device to go do something for you, you have to be a programmer yourself, which is a pretty high barrier to entry for a lot of people.

20:55Or you have to rely on the fact that a programmer has anticipated some need that you might have and packaged up a piece of software into an application that you were able to run. And those are the two ways you can get a computing device to do something for you. Until now, the thing that changes with AI is it can understand a thing that you want your computing device to go do for you. It can figure out a way to go make that thing happen, and you don't have to be a programmer. It's kind of a profound change, because it basically means, and I don't think this is next year, but it's probably not going to be 10 years.

21:39This whole notion that you have teams of people whose job is to go anticipate a bunch of very granular user needs in some narrow space. And then they're gonna go write a bunch of code and then figure out how to hang that code onto some user experience. And they hope that they've done a good enough job and they've anticipated the needs in the right way. And they've designed the user interface in the right way. And they've gotten all of the code right. And they just sort of grind away on figuring out what that feedback loop is. That's gonna change. like you just aren't going to need as much of that anymore.

22:16What you're going to need instead, some kind of agent actuating those capabilities on your behalf, rather than you having to do this weird impedance matching that we've got right now between how a user has a set of expectations and how a product team has imagined what those expectations are. Is there a role in engineering or product teams that we have today, which you're like in 20 years time, people will look at and go, what you had, secretaries who typed out, you know, voice recorded notes from a doctor, what? The role of engineers are you're still going to have to have people who build capability infrastructure.

22:57So make this thing happen in the real world, like provide access to like this, you know, weirdly situated repository of information, like, you know, it's like just a bunch of capability things that people will need to build, but like the user interface that surfaces those capabilities will probably be agents. And you know, product managers like all don't believe in this like one agent for everything sort of theory. I think you'll have a lot of agents and the reason I think you're going to have a lot of agents is because your product managers are probably going to have to be domain experts like people who sort of deeply understand something like medicine or drug discovery or early around venture investing or, you know, like, you know, just sort of pick your thing, they will have to deeply understand the idiosyncrasies of that.

23:45And they will have to help set up the feedback loops that help agents that are like assisting people doing those tasks, like better and better do their job. It'll be a combination of the product manager and the users of the agents teaching the agents how to be better and better at the things that you're trying to get them to assist you with. I often think that we overestimate adoption in a year or the short term and underestimate in the long term. When I look at the hype around agents, I share the excitement, but I question the immediate adoption or the expectation that some of the world's largest companies will be using agents in the next year or three years even.

24:29To what extent do you think I'm right or to what extent do you think actually this wave of is different given the distribution of someone like Microsoft. Usage always follows utility. So like you make useful things like they get used a lot. Clearly with software development agents, like we're getting a lot of adoption right now. Yeah, we've gone very quickly from, you know, developers being skeptical about these tools to like you will get this from my, you know, coal dying fingers. Like I think of this as like one of the most essential tools in my toolkit and I will never give it up. And the agents are becoming more and more powerful.

25:07Can I ask you, to what extent is there luck in there? When I look at them and when I speak to people about them, you're right, there's user love. But everyone says, oh, but there's no luck in. I'd happily switch to the next person tomorrow. To what extent does that mean it's valuable? There's no luck in in search. Like you can send your next query to a different search engine than the one you're using right now. And yet you don't. It is our job building these agents to grind and grind and grind and go every day Try to make the agent better and better and better and to do more and more and more of value for our users If you do that and you do it well like they will continue to choose you Can I ask when you think about a five -year time horizon?

25:49What will the interaction model that like between humans and agents? I think the thing that's missing right now with their agents is like they are conspicuously missing memory which makes them awfully transactional and Even in the places where agents have memory. It's like a pretty limited form of memory And so like I think one of the things is gonna happen because I know lots of people are working on it right now is Memory is gonna get a lot better over the next year or so So, which means that as you're using an agent, like, and it remembers more and more about your past interactions with it, it will be able to conform itself more and more to your preferences and it will be able to do things that we do very naturally, which is like you solve a problem once and you record the solution to a problem and then you don't go and solve it from first principles over and over and over again.

26:43So memory even gives you the ability with these agents to have some kind of abstraction and compositionality where you can just sort of build up like more and more powerful ways of doing things inside of the agent over time because it's remembering the past things that it's done and learned. The agents definitely, I mean for sure, like this is going to be true. Like they will definitely be less transactional, less session oriented going forward. I hope we get more asynchronous things happening over the next 12 months, which mean like right now, you know, it's very interactive. Like you go to your agent and like you send a prompt in and it goes and does something immediately and like gives you the response back and it's like, yep, I've done it.