Amazon's Longterm AI Vision — With Matt Wood

0:00The VP of AI Products at Amazon Web Services joins us to discuss what people are actually building with the technology and whether it's worth the investment. All that and more is coming up right after this.

0:13You're used to hearing my voice on the world bringing you interviews from around the globe. And you hear me reporting environment and climate news. I'm Carolyn Buehler. And I'm Marco Werman. We're now with you hosting The World Together. More global journalism with a fresh new sound. Listen to The World on your local public radio station and wherever you find your podcasts.

0:41Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Well, a year later, we have Matt Wood back with us today. He's the VP of AI Products at Amazon Web Services. Last year, we spoke at the AWS Summit in New York City, all about Amazon's AI strategy. And we have a great opportunity now to talk a little bit more about where the AI field is heading. Matt, welcome back to the show. Great to see you. Good to see you too. Thanks for having me back. This is awesome. And congrats on the growth of the show. It's been amazing. I listen every week, so it's a pleasure to be here.

1:16Thanks so much. Oh, that's awesome. So you'll have some context here. So let me ask you, I think, the most pressing question that I have first, which is, when we spoke last year, you said, I wouldn't be surprised if just the AI part of our cloud computing business was larger than the rest of AWS combined in a couple of years. So I'm actually curious where it is today. But before we get into that, here's the sort of disconnect I have. So obviously, we spoke last year, there was all this potential with AI, we've been talking about on the show a lot. And yet I was just speaking with a colleague of yours who referenced this Gartner study that said only 21 % of AI proof of concepts.

1:54So the different programs and products within companies actually go into production. That's a one in five rate, which is which is not great, given how much effort and money it takes to get these things going. So talk a little bit about where the potential and the state of AI building today, why there's that disconnect and where we might be heading. Yeah, happy to talk through it from my perspective. I've been very fortunate over the past year or two to talk to literally hundreds of customers in every single industry. And I have honestly not seen this level of energy and enthusiasm for any technology, probably since the advent of the cloud from customers.

2:33Most customers are investing very diligently. They're making good progress. There is a group which is moving slightly faster than the average, which is somewhat counterintuitive. And that group is actually the regulated industries. And so it's folks like in financial services and insurance and healthcare and life sciences and manufacturing. And they're able to move a little bit faster in part But because all the regulations that they've had to comply with over the past 20 years that probably felt at the time like a bit of a headwind have actually driven the right set of behaviors for that group to be successful with generative AI.

3:12And so they have all of the governance of their data figured out. They understand the quality of their data. They understand which data can be used where, by whom, and for what purpose. They have very, very large amounts of private text data, exabytes of the stuff in some cases, which are market reports or clinical trial results or life insurance documents, those types of things that the models have never seen before, but are really good at looking at and reading and summarizing and connecting the dots and finding disconnects. And they're just earlier in their kind of digital transformation journey.

3:49And so they've probably looked across and felt like they were kind of sitting to the side as other areas like retail and transportation and hospitality and media kind of went through this very aggressive digital transformation over the past 10 years or so, driven by the web and driven by mobile and a lot of other factors, including the cloud. And these organizations are looking to not just use generative AI to catch up, but to actually leapfrog ahead using the data that they have, which is privately held. So So that's one area that I think is probably a little counterintuitive. I don't think I would have guessed even a year ago or two years ago that 160-year-old life insurance companies would be in the vanguard of really delivering value through generative AI.

4:36But they have these very, very large document stores of 90-year-old life insurance documents, which are probably going to pay out in the next decade or so. And they've been scanned at some point, but no one's ever read them. and they're not sure what level of risk is associated to those documents inside their business. And so they're able to use generative AI to be able to piece that risk together and understand it more completely. And so I think... So I see that you're going to say like the structured, the companies with structured data, who have it very organized and have this partnership already are going to be the ones that are going to benefit the most, which is, that makes sense.

5:13But then there's also like some of the more glaring issues. Change management is difficult. The model still costs too much to run. They're not quite good enough yet. Like last year, and we're going to get to it, but last year we were talking about agents and all these other advanced use cases. And they've clearly not hit the way that they are supposed to. So what do you think about these limitations? Aren't they the main things that are holding back the field versus just like getting their data in order? Well, I think there are limitations to the technology today. And part of being successful with the technology is understanding those limitations at a deep level.

5:47And you referenced, I'm not familiar with the details, but you referenced kind of 20 % of prototypes going into production. Honestly, that sounds pretty good to me. If you think of just the amount of experimentation that is happening inside organizations around generative AI, just the number of experiments that are being run on AWS for different companies in the regulated industries and all the other industries that I mentioned, And Bedrock, which is the service that we make available to customers to build generative AI applications, that's one of our fastest growing services ever. And all up, AI and machine learning at AWS is already a multibillion dollar business in terms of ARR.

6:31So there is a lot happening. And I think that that 20 % is actually pretty good because the denominator is absolutely massive. And when technology shifts happen, you really do want customers to be able to innovate, to be able to experiment really safely, really quickly with that technology to find out what works and what doesn't work. And we're dealing here with a technology which is just in its very earliest days, it's much more like a discovery than it is an invention. We discovered that if you build these very sophisticated mathematical models, that there is emergent behavior within them that resembles reasoning, that resembles intelligence.

7:09And we're applying that in some places and some of those applications will turn out to be successful. And it is no surprise to me at all that some of those experiments turn out not to be successful because if you're experimenting in the right way, a lot of those experiments are going to fail. And so it's why customers in part turn to AWS for running some of these workloads, the majority of these workloads, because they're able to broadly democratize the way that these applications are built using generative AI. And they're able to validate the ones that work really, really quickly. And then when they find that 20 % that works, they're able to take it into production very quickly and at a very, very large scale with the right cost structure around it as well.

7:50And so I think that the 20 % is a little misleading if you think the denominator is small, but that denominator is massive because there's just so much experimentation happening. And we see it on AWS and inside Amazon as well. Right, right. Right. And look, this is where I always kind of get tripped up because we talk about these, you know, these big emergent, big things like emergent behaviors and models being able to do reasoning and how it's a discovery. And then we talk about, OK, so what practically are they doing? And it's like, well, they're combing through insurance documents. You know, shout out to all the folks working in insurance.

8:26And I'm sure we have some listening to the show. But I'm like, man, if we made this, I mean, if people in the tech field made this discovery that models are intelligent and can think for themselves and like the thing that's like one side of this. But then we ask when it's applied. And it's like, well, insurance adjusters are a little bit more efficient. And it's like, can that, because the market has valued and the industry has sort of started building around these discovery and use cases, the reasoning, the emergent behaviors. But then we ask practical and it's like the most boring applications you could possibly imagine.

9:05So is that going to change? It's interesting you say boring because boring workloads are boring because there's so freaking many of them. They're just everywhere. And so, yes, I absolutely believe that there will be large step function changes in a significant number of industries that are going to drive orders of magnitude improvements for the organizations that work on them and society at large. One example is just computational biology. And we can talk about that in more detail. But the work that's going on there in terms of using generative AI are the likes of the Dana-Farber Cancer Institute or Genomics England or Pfizer or the work we've done with a startup called Evolutionary Scale to be able to use generative AI to be able to design entirely new molecules, to design entirely new antibodies that are manufacturable, that can go on and find new drug targets.

10:01That is a major opportunity and step function. It's early for sure. The company just went out of stealth. They just published their paper, which is a great paper. I recommend everybody read it just for background on what's happening in that field. But I absolutely believe that there will be many different step functions forward in multiple different industries of that format. I also think that there is a huge number, some of it's going to be long tail, but just a huge number of what you call boring workloads that are going to be completely reimagined through the use of generative AI. And that's okay.

10:34You actually want a lot of that boring work to be automated. You want a lot of that work to be improved. You want to be able to channel the boring work, which has maybe inside some organizations is seen as a bit of a, just as a cost center. And to be able to turn that on its head and channel it into something which drives invention and drives growth. And this is exactly what we saw with cloud computing in the early days as well. I literally could have said that sentence. In fact, I think I did with the advent of cloud computing, that there is a huge number of workloads inside many, many enterprises that can take advantage of not just the cost savings in the cloud, but can take advantage of the agility in the cloud and take something which is traditionally considered a cost center, building out data centers, which offer no undifferentiated value, and turn it on its head and drive the right cost structure and the right agility to be able to use that infrastructure to drive new product creation, new invention, and reimagination of all of these different products.

11:37And so what we consider boring today is going to be rechanneled, in my opinion, into much more high leverage growth opportunities for many organizations. And there's such a big change management component to it as well, right? We talk about the models, right? There's a cost. There's a capability of the model. But also, you know, one thing about trying to reimagine how boring work is done is there's a lot of people who are sort of used to that work. What percentage of the workplace do you think or the workforce do you think is really ready to like, let's say this AI can revolutionize the way they do work?

12:15What percentage of the workforce do you think is ready to take advantage of it? It's a good question. I'm not sure I would peg it as kind of ready. I suspect that whilst there will be these step function changes over the long period, I think in the shorter term, in the shorter outlook, it's going to feel a lot more incremental than we're probably used to. There's an old adage of a story that folks tell that when we finally discover that there's life on another planet in another galaxy, we all have this idea that this will be a huge societal shifting event for the planet that we discover there's life on another planet.

12:57But in reality, I suspect there's just going to be lots and lots and lots of small iterative announcements. And that when the NASA press release comes out that there's life on another planet, it will seem really obvious at that point. And from now to when that eventually happens, Yeah, that's a really big jump, but incrementally, we'll get there incrementally, not in one big shift. And I think the same thing will apply here. There will be over a long term, like big incremental shifts in how we deliver products, in how we deliver technology, and how we interact with data and information and each other.

13:31But it'll probably appear kind of incrementally. And having patience and having a long term view allows you to drive more of that value incrementally and allows you to experiment more and it allows you to have big goals and kind of iterate yourself to iterate your way to greatness. And having that long-term view, I think, is, to go back to your question, one of the most important cultural shifts that organizations will need to make. You're going to need to have the right teams for sure. You're going to need to have the right talent. You're going to need to have the right technology. You're going to need to partner with the right organizations to be able to drive that technology.

14:10But having the ability to be able to take a long-term view so that you can allow those creative, inventive builders to be able to use that technology to be able to iterate and improve and experiment and invent, that requires discipline from a leadership perspective. It requires you to set up kind of small blast radius experiments. and it requires the organizations to be very tolerant to that failure. Because experiments failing, you've learned something there if you've set it up right. And that learning is disproportionately valuable at this point in the kind of technology cycle. And so that cultural element that you outlined is absolutely critical.

14:52I'd actually say it's more like 50 % technical, 50 % cultural in terms of the weighting of the elements of investment that are going to be required to be successful. So I'm not sure exactly what percentage right now is kind of ready. I would guess if I had to put a number on it, I would say it's probably 25%, 35 % in most large-sized enterprises. But over time, if you look three years out, five years out, 10 years out, whatever it might be, with that long-term horizon, my guess is it's going to be 100%. Yep. Okay, I'm going to ask a follow-up on that. But first, you believe in aliens? I think you have to believe in aliens if you understand just how big the universe is.

15:36It just seems incredibly unlikely that we have hit the absolute only magical sweet spot in the whole universe to encourage carbon to be able to animate and dance around as we do as humans every day. So the probability of it just being limited to Earth seems very, very unlikely. Although I acknowledge the paradox that if there's life out there, you know, where is it? So that's why I kind of like that. Or, I mean, yeah, they could also all be dead. And we might be like right now, like the only living. I mean, I think there probably are some sort of life forms out there that have existed in the universe, you know, either before or will come after.

16:18But to have them exist concurrently is. Yeah, I agree. That's the question. All right, let's let's. Yeah, go ahead. I just want to get back to the stuff. I guess we could do another show. I would love it. Goodness. All right. So you're what you're saying about patience, incrementality, you know, 25 percent of the organizations being ready and replacing the boring stuff. That all sounds good. But it also makes me wonder if we're going to end up in a sort of trough of disillusionment with this technology, because there's been so much hype and so much money that have poured into it that are demanding almost a revolution now.

16:58And what you're describing isn't a revolution or isn't a quick moving revolution. It might be a slow moving incremental sea change, but not something that happens immediately. it's not something that um you know the wall street types for instance will be like thrilled to know that it's just going to take a while because they think in quarters so do you think there's a risk here and in like within the next few years sort of the public uh perception of this technology turning a little bit because of the incremental incremental nature of it i think it would be a possibility if and it's a huge if if the technology wasn't poised to improve so if if If you believe that what we have today is pretty much what we're going to have to work with, with only incremental small improvements over the next three, five years, then I suspect that folks will feel like the promise on this occasion hasn't been delivered on.

17:55But technology tends to follow an S-curve over time. And you get to the top right-hand corner of that S-curve, and you end up with the technology, with the capability and you get these, you know, just decreasing improvements over time. You never really know where you're at on the S-curve until you're looking backwards. And so it's kind of hard to judge where we're at. I think most people would think we're probably in that kind of middle section, high gradient piece just because there's so much happening and there's so many improvements. There's new models and new techniques and new technologies from academia and the public sector, private sector.

18:36And I have no doubt that by the time we finish this conversation, there'll be another technique out there that is worthy of our attention. But my guess is that it's probably more likely that we're at the bottom left-hand corner. I don't think we've hit the kind of hockey stick inflection point yet of what this technology is capable of. It's still very, very, very early. So at some point, we're going to hit that hockey stick inflection point. And it always happens with different technology shifts. It can take more or less time depending on the shift and the speed of the technology. If you look at the thing that triggers the S-curve bend is different in a number of different ways.

19:16If you look at the maturation of the internet itself, that hockey stick inflection point, it really, I think, landed with the development of SaaS-style Web 2.0 applications, whether it was web mail or whether it was finance systems, whether it's hotel booking systems, whatever it was, that capability of being able to have access to those types of services and the fact that you could integrate those services through APIs and do interesting things with them, that meant that every new service that was added to the internet made all the other services more valuable. And that's kind of what pushes you up the S-curve in many times.

19:56You saw the same thing with kind of the mobile transformation where we had these remarkable new devices. We had these applications that more and more people invested in, more and more organizations invested in. They became more and more sophisticated. And over time, the operating systems on which those applications ran allow those applications to interoperate and interact in interesting ways, both with the operating system and with each other. And so every net new application added makes all of them more useful, makes the whole system more useful. The whole device in your pocket gets better over time without you having to do anything.

20:30And so that pushes you up the S-curve as well. And I don't think we're at that point with generative AI yet. We have a really robust set of really interesting, really powerful models which are going to mature over time. But customers will, I'm sure, find interesting ways to combine those different models. There isn't one model to kind of rule them all. Each different model has different sweet spots. And it's my expectation that most customers will invest in not building the foundation models, but will invest in fine tuning and improving those individual models and customizing them in interesting ways for their own use case.

21:04And those capabilities are interesting in isolation. But part of what will push us up the S-curve that we're seeing with customers at AWS and at Amazon is that combining those models together, leaning into the sweet spot of all these different models, allows you to build systems that, in aggregate, have a compounding effect on intelligence. It's not additive. It's a multiplier. And so that's going to push us a little bit further up the S-curve. I think another really interesting area, and the one that's probably closest to SaaS applications and mobile apps, is what you mentioned earlier, is agents.

21:40I think agents have a good chance of being the apps for the generative AI world and the generative AI era. And that as we add more of those and we find ways to orchestrate multiple agents together, and there's already customers that are building multi-agent systems on AWS today that combine specialties and combine agents that can goal-seek on your behalf. and collaborate or contest with each other in interesting ways, that means that every new agent that's added to the system drives you up the S-curve. It makes all the other agents more useful at the same time without you having to do anything.

22:14And that's a really big part of it. But are agents an actual thing in production now? Yeah, I think so. Because I'm just going to say, last year we spoke about, you made an announcement about how agents were, agent building technology was on its way. And it's just like a full year has gone by and I haven't seen one example of like a realistic agent going out there and taking action for people. Well, I think I've certainly seen some. I use some on a day to day basis. Let's hear it. Yeah, that's why we do these discussions. I recommend you check out a couple of things that may be interesting to you and the audience.

22:52One is a startup company called Ninja Tech. You can check them out at Ninja Tech dot AI. they have an assistive system. You can interact with natural language, as you may be familiar with. But they also have under the hood, a set of specialized agents that can perform different tasks on your behalf. So they have a researcher agent, they have a scheduling agent, they have a web agent, all sorts of different agents. And just by asking your question, they interpret the question. And then they have an agent which looks at the response and says, hey, this looks like you're doing some research. Let me ask my researcher how I can best help you.

23:32And I'll set a problem to my research agent. And that research agent runs off and does its thing. And they say, oh, there may be some web data that will be useful here. I'll set my web agent off to go and collect that data for me, and so on and so forth. And it pulls back all the information together and allows you to interact with your calendar and with your schedule or your email or whatever it might be, in levels which are much more automated than you could do with just a standard assistive chatbot. So that's one example. But if this stuff is so useful, then why hasn't it broken out into the public yet?

24:05I think it's very early. Today, agent systems or agentic systems, as they're sometimes called, is still relatively early. But I think they are breaking out, to be fair. I think Ninja Tech is seeing remarkable growth. They have hundreds of thousands of monthly active users. We've also built some really powerful and popular agents on AWS. So we have an assistant for builders that we call Q, Amazon Q. And Amazon Q allows you to generate code if you're building software. It will take a question and give you answers and give you guidance on how to build on AWS and all the things you would expect. And that's useful.

24:45That gets you a bump in productivity. We've seen some customers get, you know, in terms of just the amount of code that is automatically generated that they accept. It's usually between 35 % and 50%. It's higher on Q than any other comparable service. But the thing that drives productivity for developers is what we call the developer agents inside Q. And so with the developer agents, you don't just ask a question about what code to write or write a comment and get the function back. You actually set a task to Q. You say to Q, hey, I want to add this feature to my software. Q looks at the software across your repository, it looks at the changes that you've made inside your development environment, it understands the type of change or the type of feature that you want to make, and it goes off and it looks at all of that information and it makes a strategy.

25:36It doesn't just generate the code, it makes a strategy for how to add that feature. So it picks which functions need to be updated, which modules need to be added, which tests need to be run, which documentation needs to be added. And you get a chance to review that strategy. And at some point, you can just say, hey, Q, go for it. And Q will go off and work diligently through its to-do list to create a set of software changes that you can choose to commit, which add that feature to your code. And so if you can imagine a developer going from having to just write or generate that code manually, to having tens or dozens or over time hundreds of those developer agents running around doing the work on their behalf, you get this combinatorial explosion of productivity.