Cloud Strategy in the AI Era with Matt Garman, CEO of AWS

0:00I don't know how many times in the first couple of years I had to explain why a bookseller was offering compute services and storage services. there's not a lot of business opportunities that are as big as cloud computing and as potentially transformational. Our original AWS thesis was we take care of the muck so you don't have to. I want my customers to want to run on us.

0:22Hi listeners and welcome back to No Priors. Today we're talking to Matt Garman who took over as CEO of AWS in May. Matt has been with AWS since it was a$0 billion business to today's$100 billion dollar run rate business. Welcome, Matt. Matt, thanks so much for joining us today. It's a real pleasure to have you. One thing that I think is really fascinating is you actually started on AWS in the very early days. It was just getting started. And you did that as an intern while you were getting your MBA. Can you tell us a little bit about the origins of AWS as well as your own involvement with it? Sure.

0:52So it was in 2005. I did my business school internship at Amazon. And as we were looking around for projects, I talked actually at the time to Andy Jassy, and he was telling me that he was starting a new business inside of Amazon that was technology focused and he couldn't tell me about it before I started. And I thought it sounded interesting, so I joined him and worked on that. And it was AWS pre-launch. I got to work on that as an intern. It was a super cool opportunity. I came back full-time, effectively as the first product manager for AWS, and I'm now on year 18. So I've been working on the business the entire time.

1:30and it's a fascinating space. Even back then, we saw the potential of what AWS could be. Obviously, that was a startup inside of Amazon and so we had visions of what we thought it could be but there's a lot of hard work and some good luck and some things that have gone right for us and a great team over the last couple of decades to go build AWS. And the fascinating thing today is it's still in the very early stages of what the business can be. There's not a lot of business opportunities that are as big as cloud computing and as potentially transformational to every industry out there. And so that's an exciting place to work just as much as it was in 2005 when I was an intern, kind of writing the original business plan of who, when we first launched our services, which companies might possibly be interested in using these things.

2:23What were the other projects that you were offered at the time? I worked at startups before going to business school. And part of what I was looking for is I wanted to see how larger companies did new projects and kind of entrepreneurship, if you will, inside. So there's a couple of few technology companies that I looked at and was excited about Amazon. And there was a couple of other kind of retail businesses that, I mean, there's the internships were mostly in retail. and you know there's some new categories that they were starting up and things like that that could have been interesting but I always also knew I wanted to go back to technology so this was far and away it's what convinced me to come to Amazon because it seems so exciting.

3:02How well fleshed out were the original plans because I ended up using AWS for my first startup in 2007 or 2008 and it was pretty new and there's a small subset of services at the time. And to be honest, at the time, Amazon wasn't thought of as deep of a technology company as it both was and is. And AWS was really kind of like, wow, Amazon is doing this thing that, you know, at the time you thought maybe it would be natural for Google or somebody else to have started with. And so I'm a little bit curious about that sort of early conception and roadmap and what was in place and what you added over time.

3:35I don't know how many times in the first couple of years I had to explain why a bookseller was offering compute services and storage services. I think the approach that we took worked out to be one of the key pieces to our early success. I think if you look at some of those others like Google and later Microsoft, they kind of went at this space. First, we were the first ones out there that had anything like this. But even soon after that, I think they went after this space like that was going to force the developers to change how they build applications and kind of build them in a new way. And we went after it as in we're going to build building blocks and let developers and builders go build interesting things.

4:13And so our view, and this started really from internal Amazon. If you scroll back all the way to 2003 or so, Jeff Bezos basically mandated across the company that in order to move from a big monolithic stack that wasn't going to scale anymore for Amazon, we had to move everything to services. And that was really a lot of the impetus for us looking at after that. And we saw the success of that. We said, well, maybe this would work for other people who are likely going to have the same struggle that we're going to have as Amazon. And so we thought, starting from first principles, what are the things that we would need to go build to help people build a company?

4:45And we knew people needed compute, we knew people needed storage, we knew people needed databases. And so we built those things, right? And we didn't force people to change how they architected, right? When we gave you a virtual Linux server, when you logged in, it was just a Linux server. Like it wasn't magic, you know, it auto-scaled and you got it in 30 seconds, which was pretty awesome at the time where normally it was going to be six months or something for you to go get a server. And this is when everybody was like, when you had a startup, you had to go to Exadata and buy racks of servers in order to get your startup off the ground.

5:17And so that was obviously transformational. But once you actually got the infrastructure, it kind of operated the same way that you were used to. S3 was a little bit different, right? The put, get, delete was a little bit different, but the storage concept wasn't too different. And so I think that was one of the things that really helped us move quickly and helped people not have to have a total paradigm shift of how they develop. It was more how they get the infrastructure to develop on. And then over time, we could add things like Lambda, or we could add things like, you know, Bedrock and AI services and other things like that, that are more different than people weren't used to.

5:49But kind of starting out with some of those things people are familiar with, and they could instantly grab on and build quickly without having to learn a new concept, I think was a really big accelerator for us at the beginning. Yeah, it was huge. I remember when it first came out, I thought there was a few things that were done really well. One, to your point, is just providing really basic building blocks. Second was just the iterative nature of it, where you launched with a small number of services and then you kept adding stuff that were kind of the obvious next steps. Early on, people would wonder, or at least I in the startup community wondered, will they end up with everything I need?

6:20And you very quickly did. But to your point, you also built it in a way where before that, I think most people building today have no idea. To your point, you literally would have to set up your own server rocks or find somebody who would do it for you. It was hugely painful. You had a whole team that was doing that for every company. And in some cases, it caused real problems. My company eventually got bought by Twitter, my first startup. And then at Twitter, we ran into real issues in terms of data center capacity and planning and all these things that you can have to scale on Amazon for. So it's pretty radical in terms of what's been enabled for startups and the decrease in headcount per company that's needed to associate with that.

6:53I remember as late as 2010, 2015, maybe you guys are still having this conversation with the largest customers that, you know, if you're, let's say a large financial, you had like this whole platform team, several different iterations of it. And the line I would get from people would be like, we're never going to do that public cloud thing from a security perspective. You can't compete with us on cost. Our platform is better. Like, you know, it's not reliable. And just like, there's a very strong orientation toward skepticism of this, you know, even in 2010, like upstart company versus like, oh, we are X large financial customer.

7:35I think almost everybody has seen the light at this point. But you described Amazon, AWS in particular, still being quite early in that journey. And I think one thing that people, even those in the tech industry who experience more exponential growth than most, like thinking about how large these markets become is really challenging. And we might be, you know, I think we're at one of the, at the beginning of one of those cycles again with AI. We'll come to that. Like at what point did AWS internally, did you guys like know that this was going to be that large? And how did you talk about size and opportunity early on?

8:15And just to give scale real quick on this, I think you guys went from something like 500 million or so in 2010 to about 90 billion last year in terms of revenue for AWS. us. And so it's just, it's this insane ramp of$89.5 billion in incremental revenue over, you know, 14 years or whatever it is. So that's amazing. Yeah. It's easy to get caught up in those big numbers and the fact that it's so early. And so, you know, back to your question on when we kind of knew we were kind of on that track. I remember it's probably 2008 or 2009. I can't remember exactly, but I definitely remember the trip. We went on a trip to New York and actually there's a bunch of financial services companies that Goldman Sachs and JP Morgan's and they wanted to learn about what is this cloud computing thing.

8:59And they mostly were just fact finding. They were trying to get information from us on how they could more efficiently run their internal IT systems, I'm pretty sure. But we went in there, we're like, well, it's worth a shot. And, you know, they were like, OK, look, our workloads are never going to run you. Like maybe someday our website will run you or something like that. But never any of our internal workloads. and we listened to them and we said, why? And then we said, awesome, tell us why. And they're like, okay, we have to have this compliance. You have to meet this rule. You have to meet this thing.

9:27We have people to do these audits, blah, blah, blah. And we spent the next decade just checking those things off the list. And we basically never said, part of what we did is we said, we want to know what are the most difficult workloads to run. What are the hardest things to do? And let's go solve those. Because if I can solve the JPMC running in AWS, if I can run, solve the US intelligence agency running in AWS, like the reasons for a regular other companies are diminishingly small. And so that was kind of our mentality as excited. And I love getting the startups running up to us, but the big enterprises can kind of easily dismiss.

10:04You're like, well, it's a startup. You know, they don't have all these things. And so what we did is we just did both. We said, look, we're going to go in as much business with startups as possible as we can. And we're going to check off the list, all the things that are going to help JPMC or the US government or Pfizer or whoever it is run on us in a secure, safe way. And that's what we did. And today, those are all huge customers of ours. After I was in AWS for about a year, I remember sitting down with a friend of mine who was a business school classmate of mine who was also working in a different part of Amazon.

10:36And he was like, oh, how's that AWS thing going? And I was like, you know what? I think this thing could be a billion dollar business. And he looked at me and he's like, dude, do you know how big a billion dollars is? Like that seems unlikely. I was like, no, no, seriously, I think we could get to be a billion dollar business. So, you know, we knew it was going to be successful and we didn't know, you know, quite how successful or when, I would say. And so, you know, now that we're at 100 billion run rate, you look at you still go out there. And I think 85 percent of workloads are still running on prem today by most estimation, somewhere in that range.

11:12you know, pick your number, whether it's 80 to 90, whatever it is, like, that's enormous. Like, if I have, there's still 10x growth of just existing workloads. Forget all the new Gen AI workloads that are being created every day. These are just existing workloads to move. There's a 10x number in there. And so that business is massive. And I think there's a couple of, you know, one of the big inflection points we saw is we went after the intelligence agencies for the U.S. government. And we won that contract and it was secret. And we pushed really hard to go in that. It was against all the incumbents, the HBs and IBMs and Oracles and whatever.

11:48And we won the contract to do this cloud workload. But it was confidential and we couldn't share with anybody that we were doing it. And IBM sued. Yeah, I remember this. Because they said it's unfair or wasn't. And so then it became public that we won this deal. And then the intelligence agencies went out to public and said, no, AWS is the most technically sophisticated. They have the most capabilities. They're the operationally strongest. And that's who we're going to go with. And so we had the government out there now saying that we're the best and most technically capable to run these highly important workloads.

12:22And that was a huge kind of stamp of approval for us, if you will. And I do think that that's one of those moments that in some ways we kind of got lucky that they sued. Otherwise, it might still be secret that we were doing that. And I do think that that helped gain a lot of credibility in the enterprises. What do you think is, you mentioned that 80 % of workloads still haven't migrated over. What do you think are the main blockers to that today? Is it just momentum? Are there specific features? Are there big things still to build? There's some technologies that I think, look, if I had an easy button, and by the way, we're trying to build an easy button, but if I had an easy button that would just migrate mainframes to a modern cloud architecture today, almost everyone will push that button, but it doesn't quite exist today.

13:06And it's not as simple as like, great, I'll go run your mainframe in the cloud. Like that's not what customers want. They want to actually modernize those workloads and have them into microservices and canadienize workloads and other things like that. So that's one, is there's just a bunch of workloads like that that are old and their customer's running a big SAP thing and they want to move it to the cloud, but it just takes time because it's tied into a bunch of other things like that. There's also a bunch of workloads that, as you get out of core IT workloads that are in line of business, that are the next set of things.

13:36And whether that's, you know, say, telco workloads, right, that are running kind of the 5G infrastructure around the world. We've slowly been moving those to the cloud and helping those customers get that flexibility and that agility of running those in the cloud as well, but they're slower to move. If you think about all the compute that runs factories out there today on factory floors, most of those have not been modernized. Most of those are thinking, and there's a huge opportunity, by the way, for AI to totally revolutionize how you think about factory workflows and efficiency there. But a lot of that hasn't moved.

14:11And so some of this is, you know, there's on-prem infrastructure that people are still amortizing. There's people whose job it is to run on-prem data centers. And so they're kind of resistant to moving things. So, you know, there's a bunch of factors in there. And so some of it is just takes time. Some of it is technology pieces. Some of that is we still have stuff to go build and innovate and help make it easier for customers to do that. I'd love to hear about just the initial investigation of like generative AI as a technology change and like how AWS began to react to it and invest in it. Because to some degree, it puts us all back in the like on-prem colo era of the world where to get one of these, you know, if you're doing any sort of real pre-training to get your startup off the ground, you're back to, I guess, I'll buy a bunch of DGX boxes somewhere.

15:04And like, I need to think about the cost and management of that. Well, actually, most people are still buying those, but in the cloud, but it is kind of a, it's not a serverless type of a thing. It's, you know, most people are still not buying, you know, H100s and hosting them in a colo or anything like that. And increasingly, I think that's going to get harder and harder as you move to liquid cooling and larger clusters. But, you know, it's a super interesting space. I think we've been working on this space for how many years now? And look, we've been investing in AI broadly for the last 10 years.

15:37And that's why we started five or six years ago investing at the infrastructure layer and building our own processors, because we knew this was coming. We saw this path coming, and we knew that that's also not a short-term investment. So it's one of those things you've got to invest way ahead. And then we were investing in building generative AI models. And then, you know, OpenAI kind of made a generational leap forward with what they're able to do and what's possible. and then many people have talked about this, but it really in some ways was a discovery as much as anything about just what was possible and kind of unleashed a new set of capabilities.

16:10And so we actually as a business took a half a step back and said, okay, these are gonna be transformational abilities. And assuming that this technology gets better and better and better over time, how do we make it so that every company out there can go build using those technologies? And so different than how can I go build a consumer application that people are going to be interested in, we kind of took it from the point of view of AWS, right? Like just what are the building blocks that I can help all of our customers, whether they're startups, whether they're enterprises, et cetera, go build interesting generative AI applications.

16:44And so we started from first principles. Customers are going to care a ton about security. That's not going to change. They're not going to all of a sudden not care about securing their infrastructure. We also had this hypothesis, two more hypotheses. One, that the idea that there wasn't just going to be one model. We thought that there was going to be a lot of models for a lot of different purposes, and there'd be big models and small models, and people would want to combine them in new and interesting ways. And I think the last two years have probably played that out. But I think when OpenAI first launched, it wasn't as obvious.

17:12But that was kind of one of the bets that we made. And then the third one is that we view that every enterprise that was building on us, the interesting IP that they were going to bring to the table was mostly going to be their data. and they were going to care that their data didn't leak back into a model or escape from their environment. And so we built a bunch of what we did starting from those principles of how do we make sure that these things are secure, that their data is secure, that they can have access to every piece of technology that customers need to go build interesting applications, and they can do it in a cost-effective way.

17:45And so that's how we approach the space. And I think we now have a platform in Bedrock, in Tranium chips and Inferentia chips, in and then a bunch of the other capabilities around, as well as the suite of models that we offer, both proprietary as well as open source ones or open weights ones, that I think we're starting to see that really that momentum pick up. And we're seeing more and more customers really like that story. They like that platform to build from. And we're seeing enterprises really lean in and want to build in that space because it gives them a lot of that control that they want as they go and build applications.

18:21How much do you think it matters that AWS has, let's say, like first party models it offers its customers? Because that's clearly a strategy for some of the other hyperscalers. hyperscalers? Google, that's obviously their strategy. They're really the only one that has a first-party model today of the other hyperscalers. Microsoft's done a good job of co-opting OpenAI's innovation. Although in their last, I saw recently, they listed OpenAI as one of their biggest competitors now. So it'll be interesting to see how that all plays out. But for us, I think it's important, which we do. So we are building our own first-party models.

19:00We have our first-party models today. In fact, the Titan embeddings model is by far the most popular embeddings model that we have inside of Bedrock today for people that are building search indices and thinking about things like that. And we are building larger and larger models as well, first-party. I think it'll be important, but not critical. I mean, I think people love using Anthropics Claude models. Those are fantastic. And right now, those are the best performing models in the world, which is fantastic. We just launched Llama 3.1 on the day it launched, and we have a really tight partnership with Meta and their open weights model is fantastic.

19:34And I think increasingly we're seeing customers really love that open weights model because they can go and, particularly enterprises, customize it, they can do fine tuning to it, they can add their own data to it and really customize and distill and do some interesting things. And so I think that is super critical. You know, we're seeing kind of specialized models, if you will, where you see folks like Adobe building, Firefly is all built on top of AWS, purpose built for their own thing that they're building. And, you know, whether the Amazon purpose-built models are, I think they're an important part of that.

20:07Mostly it's, partially it's for us for learning. Some of it's for powering our own applications. And some of that may be for end customers. But it's all kind of that diversity of option, honestly. Like we want there to be the best set of options. And we want them all to run in AWS. And so we want those workloads to run there. And so to the extent that we can do something novel or interesting with our own first-party models, we'll do that. But we're also delighted for our partners to run as well. So I think it's a little bit of both. Yeah, it's really interesting. Alyssa Henry, you know, long time AWS leader as well, is a friend.

20:42And over the years, I would like ask her about some interesting new open source project. And she, you know, it was the most terrifying thing, honestly, because she would always be like, AWS loves open source. We make more money on open source than any open source company does, which is, you know, if you think about all the advantages that AWS has, even if you're very friendly in the ecosystem, it, you know, can turn out that way. And so I think the sea change that's happened in terms of availability of open weights models and multiple players here, Mistraw, Lama, et cetera, that are very competitive is like, I think, you know, a huge, huge boon to AWS's general open ecosystem model.

21:23We've always leaned into open source. We're huge contributors to many open source projects, and we lead many open source projects. And I think we do a good job creating and turning those into businesses for our customers and helping run managed open source projects. And so it's a big area for us. One of the reasons, frankly, is that, you know, we've long said we don't want customers tied to AWS because they're locked into some proprietary licensing piece. We want them to be able to, we want, I want my customers to want to run on us as opposed to kind of locked into a Microsoft license where you're held to some different license that you can't get off of easily or old school kind of like Oracle database that you can't get off of.

22:05we want people to be able to run. And so even where we have something like Aurora, which is our kind of managed database, it is 100 % ProSgress compatible. And if you take that code and go run it at a ProSgress database somewhere else, it won't operate as well, because we do a great job at that, but it runs. In theory, if you run it as well, it will operate as well. And so that's how a lot of our services are built and how we think about things. We'll support proprietary things as well. And at some level, there are some services where you've got to take advantage of the cloud where there are some proprietary things that customers can use, but things like Dynamo and other technologies like that.

22:45But we really embrace open source and I think it's beneficial to the whole industry, frankly. And it's a way to get better security, better visibility, and kind of that license portability, I think, is a key aspect as well. You mentioned the model side of the AI world and there's other main AI building blocks. There's RAG and there's certain aspects of fine tuning and other things that people are increasingly doing over time. There's other parts of it like eval suites. What are the main building blocks that you can talk about in terms of things that are either coming to AWS or how you think about that more fragmented world of all these different components and how they fit together relative to AI workloads today?

23:24That is kind of the idea of Bedrock is that we want to make it easy to do. And I do think actually in many ways today the models is the front and center thing that everybody pays attention to. But I think increasingly it'll become a smaller percentage of the thing that people pay attention to because people are going to care about whether it's RAG or some other sort of knowledge base. So we call knowledge bases because it's, you know, the technology may change over time under the covers, but that like, how do you have a grounding set of truth that you use? I also think grounding data is an interesting thing for, um, for like real-time information that you want as part of your AI systems.