How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026 | Asha Sharma (CVP of AI Platform at Microsoft)

26:30Is there any way you, your team, have found a value in working with agents of some kind other than coding and imagining as a big part of it, but just anything there that's like, wow, that's a big deal. At this point, we have AI and agents and many of our workflows. It was like one of my favorite ones. So right now are my engineering partners out. So I jump on the live site bridges when something goes down. And as something as simple as like you can automatically get a summary of everything that just happened because usually there's 15 people talking. You don't actually know where the incident started, where it's going to end and everything.

27:04And then all of a sudden I have that and I can kind of figure out and ask questions and get updates like awesome. Like, I think that kind of the entire kind of DevOps areas is changing. We use Spark to create prototypes, so everybody on the team is expected to code. But like, you know, sometimes just chatting in and like talking in real words actually gets you to a prototype that's more interesting and like more expressive and reflective of your creativity. So we use that. I mean, I think everybody's using AI to write everybody's using AI to find ways to have efficiencies and coming up with documentation and things like that.

27:49And so I think it's everywhere, which is cool. I think that we're just scratching the surface, though, for what's possible in terms of working with agents. That's how I always feel when people ask me how I use AI. It's just like everywhere. It's just like in every little sprinkled in everything I do now. I don't even know how to describe it. Yeah, it's hard to remember a world where it didn't really exist. Yeah, there's a product manager that I collab with Peter Yang who talks about how he just does, I don't even know how to do a strategy doc anymore without AI. I've got it, people do this. So that having someone - You think there will be strategy docs in the future?

28:22That's going to be interesting. I have this like, I have this wrote this post once of like which skills of a PM job will be most replaced by AI. And strategies, though, on that people are the most, have the biggest dip aid on. Like you could argue, I don't know, like let's get into it briefly. You would think if some AI had all of the information you had about where the market's going, your metrics, your product today, it would be so good at developing a strategy for you. Many people think that's the one thing AI will be really not good at for a long time, because that's where we need all this human judgment stuff.

28:57I don't know, do you have any thoughts? thoughts. I think that some of the most consequential products in the world required a bunch of deterministic logical sets of inputs and sparks of creativity and imagination and judgment and vision that could not be achieved without humans. Microsoft is the vision of a software factory and creating what Microsoft did wasn't inevitable. Instacart, you know, there was web vans and web vans didn't work, but Instacart did work because of a different way of thinking about it. That came through judgment and iteration and a bunch of things that you can learn unless you actually went through the process, you know, the iPod.

29:47Like, you go forward. So I think it's there. I think docs themselves, like for every idea, for every, you know, need will just start to kind of fade into, you know, applications and different artifacts in the productivity suite, which, you know, is just a different way of working. Yeah. Like your original question, which I didn't quite answer, but I think it's important you're asking, like, do we even need strategy docs? And I guess it's just like somehow everyone needs to be aligned on the strategy, maybe it's not a good thing. Yeah, it could be some other. I mean, if you architect an organization the right way to keep up with AI, you need a different alignment mechanisms than traditional ways of actually working.

30:32So let me ask you actually about that. So planning right now is just crazy. How does anyone plan a roadmap when there's just like, okay, GPT -5 is out. What works for you for setting an actual roadmap and a strategy for your team? Like how far out do you plan? How often do you have to rethink everything? I mean, I'll caveat this by saying like everyone's just figuring it out and it's a lot harder to figure it out when you're a larger organization than when you're much smaller and you get to kind of run something yourself and there's pros and cons about. So here's what we do. We, the company historically, at least in our product teams, had kind of semesters that they planned again.

31:10So think that as every six months, there's kind of a strategy with back for it all, those things. I think that's very valuable. I think like the idea of six months though and really understanding what's changing out and front is truly challenging to kind of have a over -bake situation. And so we kind of think about it as, you know, what season are we in? And so a season which is very uncomfortable can be denoted by a set of secular changes that are happening in the industry or that are happening from customers. And so you know you can think about season one might have been like you know the prototyping of AI and kind of the early GPT work and then it was all around models and reasoning models and now it's the advent of agents.

31:53And so back in last a year that can last six months that can last three months but But like grounding everybody on the ethos of what are the secular changes, what are the customer problems we need to solve, what is winning look like? So everybody has that shared sense, what is the North Star metric is something that we do. The second thing that we do is that we have kind of loose quarterly OKR. So like, OK, if we believe that, what do we need to do next quarter to actually put ourselves on a path to that? And then from there, you know, teams are operating in squads and they're kind of setting out, you know, four to six week goals that they're trying to go after for problem areas to go ladder up to that.

32:33You know, and especially as the platform for the company and the platform for our Advertisers with AI, I will say we go through lots of changes to that all the time and I think we have to just have an openness that that is the business that we're in. I think the other thing is just like we try to leave Slack in the system not just for the unplanned but for the slope I think that we have to continuously be thinking about how we're going to disrupt the platform in our thinking and what we need to be investing in to make that possible and so we try to do a little bit about. This is awesome. So, what I'm hearing here is there's this concept of seasons and everyone's aligned.

33:11Okay, this is time for agents. This is what's happening right now. We're gonna center around our strategy on agents and then there's these loose Quarterly okay RSC plan for three months roughly and then you leave some slack in the system for things to change Yes, is the current season agents? How would you describe what season written right now? Yeah, okay? It's agents Okay, do you have it the rise of agents? I sound like a terminator movie do you Is do you ever sense about the mic next season might be is there any like oh this might be coming next gosh? I don't But I think that, look, like we have more than 15 ,000 agents that are deployed on our service today, at least at the Azure Service.

33:55There's a bunch of other platforms in the company. And I would just say that I think that we should really focus on making sure that we have all of the alignment, accountability, observability, eVal's to making those agents like great. I think that Manus's breakthrough in the space was that they could do these tool -calling loops and have agents do longer running tasks that really know what their platform was able to do. I think stuff like that is critical memories, critical. There's still a bunch of building blocks that I think are leaving agents incomplete in the wild, but I think we have to really sweat the details on before we move on.

34:36So it's just like agents until the end of time until super intelligence and then we're just on beaches chill and yes agents until dank memes Look like I yeah, I think the cool thing is is like something you could come in three months something you could come in 13 months, I think like we and I've had this conviction on a set of building blocks that we want to provide to enable these agents to you endure and have high endurance and sell, that's what that's all about. When you said there's 15 ,000 agents, what does that mean? Is that 15 ,000 types of agents you can use, or is it like that's how many classes are in?

35:14That's, you know, customers, 15 ,000, I think I should re -reference the numbers. 15 ,000 customers who have produced agents, I think the number of agents is actually like millions. 15 ,000 customers that are building a specific kind of agent on your platform, and they're running, and the number of agents is in the millions and just running in the cloud. Okay, how it's wild. Some crazy numbers here. Okay, so let me just kind of go in a slightly different direction. You're kind of in the center of the storm of a lot of AI, just like seeing everything that's going on. Is there something you wish you'd known before stepping into this role that you're just like, okay, I see, I didn't expect this.

35:52When I first took the role, it was kind of described as like the belly of the beast. and I had spent most of my career building products at the center of machine learning and applications or businesses. And I think that to my surprise, a lot of the learnings have translated in terms of what makes a great platform is what makes a great product. So, and like the thing for me is like it's often in the invisible work or they're like not the pixels that actually drives that. So, for example, one of the first companies that I worked out was a company called Porch Group. I was employee 7 and we knew we wanted to hope people take care of their home and I think we invented so many features like the home report or like a way to manage your home or like house dial inspiration where you should like see all of the houses and it's map every room and the The single most important thing that we could have done and did during my time there was create a matching platform that matched the 6 million professionals with the 1300 service types of the 37 ,000 zip codes and all of the homeowners in North America to actually take care of their home.

37:06And that was just the game of inches and kind of optimizing that engine in order to create higher quality leads essentially. That's what got us to the first $500 million valuation. That's eventually what we built on to actually have other vertical services and software platforms that IP of the company. Same with messaging. The number one learning that I had was, look like WhatsApp didn't win because it had stickers or stories or dark mode. In fact, I don't even think it had all of those things when it won. It won on a few premises because one was the phone book. Like you knew that when you use WhatsApp, you could reach every single person because you had their phone number.

37:51And those were the people that you care about when you're using messaging. It was the reliability and how fast it was. Like I could text my grandmother in India and know that she would get my text message all the time. And then it was the privacy. Like when you were sending 200 messages a day to the four people you care about most, you wanna make sure no one else can read the messages. And so the end -end encryption really mattered. And so it wasn't the hundreds of features. It was all in kind of the infrastructure and the platform. Same with Instacart, right? Like, there are so many loved features of Instacart.

38:25But at the end of the day, it's a billion items that updates 3 ,000 times every single minute to get homeowners their groceries from the store that they love. And so I think I wish I had known that because I think it would have curtailed my learning curve to say that it's not all the features for the platform that matters. It's the data residency, so the hospital and Germany that's fine tuning a model can do so in confidence and the data isn't going to leave the region. It's the availability. It's the reliability. It's making sure you have the right selection of the tools that enterprises need and the right way to retrieve the knowledge.

39:01That's kind of the platform that we've built, but just didn't fully have that picture that those learnings would translate. That's really interesting. So what I'm hearing is people kind of undervalued just the simple bottom of the Maslow hierarchy of things you get of things that help you win in platforms especially in messaging platforms including so it's like reliability privacy. I don't know availability. Yeah, performance reliability, privacy safety, all of those things. Let me ask you a totally different question. When we were going to record this previously and you're like, I have a big meeting with Satya, I get it instead.

39:40We moved there a different time. Very few people get to work with Satya. He's quite a successful leader. What's something you've learned from him about leadership or product building? I've learned that optimism is a renewable resource. Like this company for 50 years has had, you know, every reason not to succeed and it has. And even as it's had early success in the AI era and challenges and other successes, like, and the space is developing so quickly, I think that his ability to generate energy and to use this optimism to renew everybody's dedication to the mission is unbelievable. And I think it's such an important part of the culture.

40:33Everybody talks about the growth mindset. That's a real huge part of the culture. But I think the ability to generate energy and clarity on what we need to go do and use optimism to renew the commitment every single day for every single person in an entirely competitive talent space is pretty amazing. Is that something you think that's just innate to him or something that he's worked on to just generate this optimism on behalf of everyone? I have no idea. We should ask him, but I'm deeply impressed by it. It's interesting that a lot of this comes down to just vibes. It's just like this vibe of, imagine it's not him just the word's uses.

41:11It's just like this energy that he exudes, optimism, and energy. I mean, think about it. We all choose to, you know, someone need just said this to me, and I thought it was great. We all choose to close the door on our kids every single day to go work on something. And so you have to work on something that is like deeply moving to you and is like, you know, you have a deep belief that is going to make the world a better place. And like, I think that's why it's bad. I think you have to follow and have a sense of duty towards a mission that is bigger than yourself. It makes me think about a line that I've referenced a couple of times on this podcast.

41:48That's really hits people really hard. That the only people that will remember you working late are your kids. Okay, I don't know where we're going with that, but that was like, you know, not your life. Too much, too much, too much, too far. Oh man, okay, well let me ask you this, what's driving you? You can upset our customers. We could have gone a different route on that one. This is the real stuff. What's driving you? What's driving you? What's keeping you excited about the work that you're doing? What AI will help us do from a workforce perspective, what it will help us do from a health care perspective, like, you know, my mom has cancer and I think a lot about how wow we might find a way to solve the form of cancer she has in my lifetime and I never thought that was possible three years ago.

42:33Like all of that's some deeply profound. And the thing that like I personally think a lot about now that we know that we're living in this time working with such powerful technology is the the effects of it and how I can you know best build the platform where people can make use of it. So like the reason why I work at Microsoft is because like the whole ethos of the company is like how do I help people and businesses achieve more and like more for me and the thing like I I think about it. And I, outside of GPUs is, I think about, will my son have classmates in the future? And that's not because agents are going to replace them.

43:19It's because the fertility rates are declining, right? Like the average birth rate in the 90s when we were growing up was like three. and now it's 2 .3 and in 2050 it's estimated to be below replacement. And I think that AI can have such a big effect on it and already is, who's just reading about a hospital in London that's able to improve pregnancy rates by using AI to match eggs and sperms and their cutting costs at the same time. You saw the Chatchee PT 5 launch yesterday such an amazing story about how Chatchee PT is helping in healthcare. You have Stanford's one of our big customers with the platform that I build, and they're working on using AI for tumor reviews.

44:08And it's just like, that is like, it is these sets of things that will like move humanity for and expand our lifetime and give us the like privilege to solve 100 year problems. And so that's why I'm excited. And that's why I do what I do. Yeah, especially in your role where you're building the platform that enables all of this, I could see how impactful that could be. Ashtas, is there anything else that you wanted to touch on or share or double down on anything we've talked about before we get to our very exciting lighting round? We touch on it a little bit, but I think that with the advent of agents and products that think and can act in reason, there's going to be this kind of new wave around RL.

44:52And I have a deep belief that that will become one of the most important product techniques of the next season or at least the next few seasons. And RL is reinforcement learning? Yes, exactly. I believe we will see just as much money spent on post -training as we will on pre -training and in the future more on post -training. We talked a little bit about Nathan Lambert's study where his review was that when a model hits 30 billion parameters, it makes more sense to kind of fine tune and optimize that. 50 % of developers, according to surveys, are now fine tuning. And we know fine tuning is good, but like if you actually go through the whole loop, you can get better results.

45:31So I think there's a bunch there. And I think there's a whole new set of infrastructure and platforms and companies that will be created that are all around this part of the stack. And so I think it's an exciting time to be in a platform space, but it's also an exciting time to be starting companies and be thinking about those problems. I want to make sure people truly understand what you're saying here because not everyone truly understands post -training, pre -training. What's the simplest way to understand the difference there and just why it's such a big deal that investment is moving to post -training?

46:02The way that I think about it is, you know, to create a foundation model, it requires a tremendous amount of compute, a tremendous amount of science expertise as we're seeing who's the cost for scientists and average value is raising dramatically. I think an expertise that we've seen is not everywhere in the world right now. And so it's just a big capex investment to do that. And with this explosion of models that we talked about in the beginning, there's a lot of good models to choose from for different domains. And so I think that you just get more leverage economically. you get more leverage from a taste perspective of how you actually want to steer a model if you're actually doing reinforcement learning or some sort of fine tuning to actually start to optimize what's off the shelf for some outcome like price performance quality.

46:58If you think about that, that's not crazy, right? Like, you know, ranking is an age old optimization problem where you don't want to just take what's off the shelf because there's like amazing frameworks and UI and kind of components that, you know, the world is React components that are out there. You still want to tailor the experience to a set of use cases or set of people. I think it's just the same kind of industrial logic. So in practice, what this means is there's like a GPT -5 model. You're saying there's a lot of opportunity and a much more efficient way to spend money, which is take something like that, and then train that on additional custom data that you have, whether it's data or just reinforcement of course, my learning may be even with humans to line it with what you want to achieve.

47:41Yeah, and it could be your own data. It could be data that you buy. It could be synthetic data. It could be something else. But I think that we're going to start to see more companies and organizations start to think about how do I adapt a model rather than how do I take something off the shelf as is or invest a bunch of money in building my own models. Yeah, I forget. I know a cursor when he was on the podcast, he shared that they have a bunch of models that That support your experience with cursor and over time there you just can have their own thing I forget who has Windsor for one of those guys just uses their own model now.

48:20They don't just plug into Claude I'm much more in the model system camp like I believe in a model diversity I think that in experience like

48:35Claude for different use cases, I think that there's some task where you care about the latency of the model. You're like cool with the thinking time or you kind of want quick retrieval and things like that. Like I think the beauty is there's a lot of models that can kind of help you achieve that. And so I'm much more in the like model system rather than one model to rule them all. Is that the return? I've also heard ensemble model ensemble models. I think about an ensemble of models as a set of multiple models that then you can fine tune and deploy independently. But at this point, we're all making of different terminology to define things that we have deep beliefs on that have limited sets of data points because everything is moving so fast.

49:18Yeah. With that, we've reached our very exciting Lightning round. I'm very excited for our Lightning round and I'm like turning down the lights. And then they'll come back on, I imagine, in one second. OK, first question. What are two or three books you find yourself recommending most to other people? At work, it's probably thinking machine. So it's all about treating the cause, not the symptoms. The prototypical example is, if you want to solve traffic, you don't actually put up speed limits. You actually have to solve walkability and mobility and why people actually use cars. Outside of that, personally, the CMO Vince the Cart recommended to me tomorrow and tomorrow and tomorrow and I read it like last month and last year and the year before because I love it so much it's like this like beautiful story over 10 years.

50:13What are some favorite recent movie or TV shows you really enjoyed? Formula One saw twice for all mankind. For all mankind I like season four. I like kind of playing out alternative of theories to kind of how the space race might look. Do you have a favorite product? Do you recently discovered they really love, could be, tech could be gadgets could be clothing? So I just joined the board of the Home Depot and we're doing a little renovation project. And so there's this new kind of new to me, DeWalt kind of power pack and they use pouch cells. And so it's like 50 % lighter, but with all the power and it's like awesome for drills and like things that, you know, I need to lift up with one hand that feel heavy.

50:58So I love that. We also are testing out this new brilliance smart home kind of system. So it's like kind of four inches of high res, middleware that allows you to kind of connect to everything. And I've like reached peak kind of dissat with like the explosion of all the technology required to actually use your home. So it just might be the middleware that like sticks, but we'll see. Do you see dissat? Is that sure for dissatisfaction? Yes. Sorry, I'm speaking in acronyms. Whoa, I've never heard that dissat. It's like, I love that. By the way, I love that you're on the board of the Home Depot. What a different part of the spectrum of work.

51:40Yeah, it's been awesome. The very first board meeting, the head of philanthropy, has been at the company for decades. And she said, welcome to the greatest company on the planet. It's pretty special. You're like, Microsoft. soft. Is there something you've learned from working with that with them that you've brought to Microsoft? Look, it's new. It's this year. But I've long worked on products that kind of had that impact. So like when I was at Porsche, it was pros at Instacart. We had 600 ,000 shoppers. And obviously the Home Depot has associates. One of my favorite things about the company and they have this inverted pyramid where, instead of having executives at the top, the associates are at the top.

52:25The stores themselves are headquarters. Then the traditional HQ is kind of support. It's just so customer -centric. When I think about amazing execution and creating these durable long -term institutions and kind of how culture and ideology and and kind of leadership is formed like I think about that and I think about at the end of the day you know AI is going to have an impact on every single person and every single job and it's like amazing to kind of just spend time with people outside of our bubble and in kind of really try and learn what they're real pain and problems and how they think about AI and how they think about technology and kind of what we need to do.

53:10Okay two more questions. Do you have a favorite life motto that you find yourself coming back to sharing with friends or your family? I used to use the kind of minimize regret framework and it's great and I've used that for a long time. I think that probably once I got into my adult years and started to kind of have a family and things like that, my kind of just worldview changed a little bit and now it was all about about maximizing kind of option value. And it just gave the things that I naturally cared about like family and health and trust and relationships. Like it was just kind of like a new level of like value associated with those because all of a sudden learning rest on the weekend can like compound in the future or you know, having good health can compound in the future.

54:04You don't like kept to trade that off or working extra hours or you know, the importance of family and all of those things. And so I think that my worldview is like when I'm 70, it's not about, what do I look back on in my life and count the number of regrets? It's really about looking forward in the number of adventures I will still have because I have accumulated this wealth of skills and trust and people and family and impact and things like that. Speaking of skills, the internet tells me that you're a second degree black belt in Taekwondo. Why, oh gosh, is this true? And then I have a question about it.

54:43This is true. Okay, that's incredible. Why are you eyes of this embarrassing? That's an incredible thing. I'm generally embarrassed anytime. Anything who's discussed about it. Okay, great. No problem. What's something that you learned from Taekwondo that has helped you with life or work? Ty Quant know is more mental than it is physical. And so I think that's the same with kind of like all of our jobs and make it product. Like I think it's like mental clarity. It's it's courage. It's it's kind of the ambition to kind of see things through and be unwavering. And so I think that's literally you know what it taught me outside of meditating which probably took me the entire time to like actually learn to meditate and clear my head.

55:35But yeah, I think it's awesome. I think everybody imagines like, you know, flying psychics or running up a wall and like you can do those things too, but the real value is like the mental pursuit of it all, you know. And you can do those things too. Wow. Okay. I'm good. I gotta get into this. Ah, sure. This was awesome. Is there actually two final questions? We're can focus finding online if they want to maybe follow up on anything if you want people to reach and how can listeners be useful to you? You can hit me up on LinkedIn or email or text. I think all of those are traceable. Look, how can you be helpful to me?

56:13I think we're all early in this journey and great platforms are built on great use cases and built on great customers. And so if you have feedback, you have ideas, you have things you want AI to be able to do to help you achieve more. I'd love to hear it. I think the thing about all of these changes is that all of these new products and use cases will be developed everywhere. And so I'm always just thinking about how can we be the platform to support that. Amazing. Asha, thank you so much for being here. Thanks for having me. Everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

56:53Also, please consider giving us a rating or a leaving review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny'sPodcast .com. See you in the next episode!