In short
Initial reactions to OpenAI’s Astra model and whether it’s AGI; Anthropic’s move into payments/billing and fraud tooling; a Code2–Maddox joint venture to secure chip components/manufacturing capacity; rising AI-enabled cyber threats driving 20–30% cybersecurity budget increases; AWS overhauling Bedrock (Project Mantle) to win spend from Microsoft Azure.
Guests
Stephanie Palazzolo (author, AI Agenda newsletter). Valita Pau (Deals reporter). Aaron Holmes (Microsoft/enterprise software & cybersecurity reporter). Catherine Perloff (Amazon reporter).
Key claims
Astra is especially strong with detailed instructions for 3D modeling and software engineering; voice/computer navigation is hyped but early interest skews to engineering. OpenAI is more willing to label Astra “AGI,” aided by reduced Microsoft agreement constraints; “AGI” is treated as a continuum. Anthropic is hiring for billing, fraud, order management, invoicing, and treasury—suggesting early in-house payments capabilities. Co2 is exploring a Maddox JV to finance chip component purchases and negotiate manufacturing capacity (first right of refusal). Cybersecurity budgets are rising 20–30% as companies fear hackers using tools like Codex; LLM-based scanners and “red/blue agents” are disrupting legacy vulnerability scanning. AWS’s Bedrock redesign improved capacity/usage and is pulling customers from Azure/OpenAI APIs.
Notable examples
Bedrock throttling delays (up to ~3 weeks) before Project Mantle; Palo Alto Networks finding ~5x more vulnerabilities with Mythos; Rapid7 cutting 12% staff; Tenable revenue surge; AWS Bedrock best quarter in Q2; Maddox tape-out scheduled H1 2027; Maddox valuation ~$4B (Feb) and aiming to double.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOInitial Reactions to OpenAI's Astra Model
1:00 to 2:15
Discussion on the initial impressions and capabilities of OpenAI's Astra model.
“It's going to be a great show, so let's get right on into it.”
Is Astra an AGI? Exploring OpenAI's Claims
2:15 to 4:36
Debate on whether Astra qualifies as AGI and OpenAI's communications on the topic.
“So maybe you have a more higher level task like, you know, make this website look prettier or like make this app more user friendly.”
Anthropic's Expansion into Payment Tools
4:36 to 10:32
Analysis of Anthropic's move into payment-related tools and job postings.
“But I'm also thinking about the social implications of all this.”
KOTU's Joint Venture with Maddox
10:32 to 14:00
Discussion on KOTU's joint venture with Maddox to secure chip manufacturing.
“Well, Stephanie, it's certainly an area to watch.”
Investment Discussions in Maddox
14:00 to 16:28
Learn about KOTU's potential investment in Maddox and the startup's proactive strategies in a hot market.
“I'm not yet, but like KOTU is discussing investing is like a new round they're raising.”
Cybersecurity Budget Increases Amid AI Threats
16:28 to 17:59
Explore how AI threats are prompting companies to significantly increase their cybersecurity budgets.
“Well, Valida, I want to thank you for coming on.”
The Rise of AI Tools in Cybersecurity
17:59 to 19:25
Discover how AI models are reshaping the cybersecurity landscape and their impact on companies.
“Essentially, a whole new raft of security threats across the board that companies are in many ways scrambling to protect against.”
Shifts in Cybersecurity Strategies
19:25 to 21:19
Understand how companies are reevaluating their cybersecurity approaches amidst new AI threats.
“Of course, all of those firms are now also building, you know, LLM powered tools that in some cases use the bottles from these labs.”
Measuring Cybersecurity Success
21:19 to 23:19
Learn about the challenges of measuring success in cybersecurity and the perceptions of security executives.
“When you talk to executives in charge of cybersecurity at these businesses, I mean, there's a lot of demand right now in this moment for protection.”
Eleven Labs: Future IPO Aspirations
23:19 to 24:23
Get insights into Eleven Labs' plans to potentially go public and the challenges they face.
“So it's a little bit like you're trying to monitor something from not happening, right?”
Show all 14 chapters
Revamping Amazon Bedrock: Challenges and Solutions
24:23 to 28:00
Examine the redesign of Amazon Bedrock to address past issues and enhance its AI service offerings.
“They recently disclosed that their annual recurring revenue had surpassed$600 million and they are profitable.”
AWS's AI Model Evolution and Bedrock Improvements
28:00 to 30:20
Learn about AWS's evolution in AI services, particularly the enhancements to Bedrock and customer satisfaction.
“And, you know, you have to remember at that time in 2025, Microsoft was the only cloud provider where you could access open AI.”
Impact of AI on AWS Strategy and Market Position
30:20 to 33:04
Discover how AWS's strategy in AI is adapting to market demands and competition.
“I mean, are all these problems gone now, or what's the status of that?”
Challenges and Future of AI in AWS
33:04 to 35:09
Explore the ongoing challenges AWS faces in AI and how they plan to navigate the competitive landscape.
“I mean, I wonder, is this part of a broader shift in strategy that AWS is pursuing here with respect to Bedrock or perhaps being a more nimble organization?”
Transcript
Automatic transcript. May contain errors.0:13Stephanie Palazzolo:Welcome everyone to the Informations TI TV. My name is Akash Pasricha. It is Tuesday, September 8th. Today on the show, we are going to talk about the initial reactions to OpenAI's Astra model. We also have some exclusive reporting for you on Anthropics to push into payment-related tools. We'll then discuss our reporting on a joint venture that Code2 is talking about with AI chip startup Maddox. We're also looking at how escalating cyber threats from AI are forcing big companies to rethink where they spend on cybersecurity. And to close out the show, we're going to dive into how AWS is drawing spend away from Microsoft Azure following its overhaul of its own AI services and Amazon Bedrock.
1:00Stephanie Palazzolo:It's going to be a great show, so let's get right on into it. OpenAI's newest Astra model has been out for a couple days now. There has been a lot of chatter about whether or not this model is AGI. I want to bring on Stephanie Palazzolo, who authors our AI Agenda newsletter for a conversation about that. Stephanie, welcome back to the show. It's great to have you here. Great to be here. Okay, so we haven't asked you yet about Astra and how people are finding it. What are the initial reactions? Yeah, so, you know, initial reactions, people are definitely very impressed. I think Astra, you know, OpenAI spoke a lot about how the jump between the last model, GPT 5.6, Sol, and Astra was going to be quite a big one.
1:47I think that's definitely true. So, you know, Astra, I think, really shines in areas like, you know, 3D modeling and software engineering. So I spoke with some people that were early testers of it, and they said, you know, one difference that some of them noticed, I think, between this one and Fable 3.1, which is Anthropik's latest model, is that they said that with Fable, it was better whenever maybe you didn't have like a super detailed idea of what you wanted it to do. So maybe you have a more higher level task like, you know, make this website look prettier or like make this app more user friendly.
2:28Versus I think Astra was a lot better whenever you had very detailed instructions and you wanted to do something very specific. And so just some pros and cons. But I think overall people are very impressed with Astra and seem very excited about it.
2:40Stephanie Palazzolo:Seems like they were really pushing the voice capabilities in their commercial. I mean, is it better on that front as well? Yeah. So it does seem in the commercial there is, you know, in the kind of demo video, there was a lot of hype and emphasis around voice and computer use, which is the model's ability to like navigate your computer. Overall, like I have heard good things about that. But honestly, I think more people, at least I'm talking to, are more interested in trying it out for like software engineering versus voice. Got it. Okay, so now the big question that I guess I was seeing trending online over the weekend was, is this model AGI?
3:19Stephanie Palazzolo:I mean, what has OpenAI said in terms of whether or not it thinks it's AGI? I saw Jensen seems to be endorsing the idea that it is AGI. Is it that good? Yeah, well, you know, definitely OpenAI wants you to think so. So whenever I wrote my kind of brief on this, I did mention one interesting thing was during the press briefing, you know, Greg Brockman was there. He was very much pushing that this is AGI, edit the call by saying kind of in a joking way, like, welcome to the AGI era. But I think this is like obviously a very different shift than it feels like maybe with the last couple models. Like, OpenAI has been very careful about their wording, saying, like, you know, we're getting close, we're 80 % of the way there, versus this time they seem much more, like, willing to just straight up say, like, we believe this is AGI.
4:12Stephanie Palazzolo:Is the reason for that, is that at all related to the whole, you know, reframing of the agreement with Microsoft, that they can now say that we are at AGI freely? because I'm sort of trying to think about the communications angle here. They've removed that from the partnership agreement with Microsoft. So that no longer impacts what they have to pay Microsoft in terms of revenue share. That's one angle to this. But I'm also thinking about the social implications of all this. Why do you think they're so willing to say that this is AGI now? Yeah. So as you mentioned, there's definitely less risks to declaring AGI now that that, you know, term has been removed from the Microsoft agreement.
4:55I think in terms of, you know, socially, why now? I mean, it does, I guess, like, taking a step back, I think whenever you initially asked me the question, like, is this AGI? I think my honest answer is, like, does it even matter whether it is or not, right? Like, this is, I think it's very clear that this AGI thing is, like, a spectrum or, like, a continuum versus, like, a, okay, after this point, we're post-AGI versus pre-AGI. So I think in the last couple months, there's been kind of this building agreement that we are in the AGI continuum, right? Maybe we don't know exactly when the moment was that we hit AGI, but it's very clear that we are in that general space.
5:36So I think at this point, a lot of people have kind of accepted AI can do a lot of the things that humans can. And that is like a definition of AGI. So I think like people have come more to terms with the fact that like the tech we have today is like probably AGI or like basically almost AGI. And so I think there's like less risk to them admitting that. And it feels like now the kind of big milestone that everybody's scared of and cares about is like recursive self-improvement or like super intelligence. And like we kind of have moved the bar once again. So people are less worried about it. Like AGI feels less scary to people.
6:14Stephanie Palazzolo:Yeah, yeah, yeah, yeah. I do want to get to another story that you published as well about Anthropic. It was kind of interesting. And this is a story about Anthropic expanding into tools that are related to payments. It's not directly payments. Help us unpack your reporting there. Yeah. So this is, I guess, a very different vibe than deciding if we're at AGI or not. But we published a story last Friday about, as you mentioned, Anthropic getting more into kind of payments-related tools. So I guess essentially how the story came about is we noticed a bunch of job postings that have been put up fairly recently, basically where Anthropic is looking to hire more people that would work in an area like, you know, billing and fraud detection and treasury services.
7:04And These are all kind of in the realm of payments and billing. And it's generally like all related to, you know, my customers like paying me for my service. Like how do I keep track of that payment and make sure I'm charging them the right amount and all these different questions. And so this is very interesting for a company like Anthropic because, you know, this is typically in something that, you know, a company like Stripe would offer. And so the fact that Anthropic wants to build more of it in-house has some pretty interesting kind of implications there.
7:36Stephanie Palazzolo:So, I mean, this is – how much is this related to the story here of Stripe buying Open Router and Stripe expanding on the AI Labs turf versus the AI Labs expanding into payments? I mean, is this the two of them kind of going at it here? Is this actually a rivalry? what do you think? Yeah, so again, this is like, you know, a caveat here. This is like kind of our analysis versus, you know, any kind of direct reporting. But it does seem like the fact that Stripe bought OpenRouter, at least our read of it, is that it was maybe more of a defensive move, right? So essentially what's kind of going on here is, you know, as Anthropic is growing, as companies like OpenAI are also growing, they're getting super complicated.
8:28And you can imagine that kind of charging and keeping track of payments for AI is pretty complicated and maybe even more so than traditional software, where it's just this monthly fee, basically, that you pay. Now it's like usage-based, it's based off of tokens, you can like cash tokens. It's very, you know, it's a very, you know, complicated thing. And so now we see Anthropic wanting to build more of that in-house and maybe potentially, you know, at some point move some of that off of external vendors like Stripe and use more of their in-house tools. So if we see companies like OpenAI and Anthropik doing that more, what position does that leave Stripe in, right?
9:06Like, you know, in the AI boom, Stripe doesn't want to kind of get left out in the cold. So what might make a lot of sense is for them to buy a company like OpenRouter, where, you know, they still get kind of a piece of this like AI token economy and still get, you know, this company OpenRouter that's very quickly growing and is really at the center of this like open source AI business.
9:28Stephanie Palazzolo:Right, right. And I mean, just going back to Anthropic's push here, you sleuthed through some of the job postings that Anthropic had related to this. I mean, how far are they in this push? What specifically are they hiring for? What did that tell you about that? So it does seem like they're still pretty early. So some job postings that we found are, you know, there were a couple for billing. There was another one focused on financial fraud. There was one that basically asked for a person who would overhaul Anthropix order management, provision, billing, invoicing, and other related systems. So this is kind of the whole process of like what happens behind the scenes when you're paying for your Claude subscription or paying for tokens, for instance.
10:17There's other roles related to Anthropics treasury systems, which is an area that Stripe does not get into. So it does seem like they're still in very early stages of finding people that are going to be able to work on these sorts of systems.
10:31Stephanie Palazzolo:Right. Great. Well, Stephanie, it's certainly an area to watch. I want to thank you for coming on. That is Stephanie Palazzolo, author of our AI Agenda newsletter here at The Information. The information has exclusive reporting that investing giant Cotu is looking into establishing a new joint venture to help chip companies address component shortages. Maddox is a chip startup at the center of that story. I want to bring on Valita Pau, who reported that story along with Stephanie Palazzolo and Phoebe Liu. Valita, welcome back to the show. It's great to have you here. Hi. So what is this joint venture that Cotu is looking into right now?
11:10So this is a first-of-its-kind joint venture that Co2 is looking to work with, this chip startup called Maddox. It's essentially helping the startup to finance their purchases of all these important chip components that the startup needs to make their own chips. So think about memory chips or logic dice or even manufacturing capacity the startups need to get at, say, TSMC, which is super constrained. and you'd have to get TSMC to make your chips. So Code2 is trying to be the center of that and trying to help the startups secure these manufacturing capacities, supply these chips.
11:54Stephanie Palazzolo:And this would basically involve Code2 basically negotiating for capacity at these manufacturing companies because they're able to sort of take orders from all these chip startups together? Is that how it would work? So like the specific of these like negotiations is not like really worked out right now. Like it's still sort of like ongoing, but essentially I feel like, you know, this is sort of, you know, first exclusive to Maddox in some way to have a first right of refusals. Potentially like, you know, if it doesn't work out for like Maddox, like KOTU can resell those capacity to like, you know, other startups.
12:32But, you know, it's still like, you know, up in the air, like, you know, how specifically it works out. But like, you know, it's truly innovative in its own sense because we have never seen this before, that, like, you know, an investing giant is, like, you know, partnering with a startup to, like, you know, get ahead of these, like, supply constraints that, like, they may face down the road.
12:51Stephanie Palazzolo:Tell us about Maddox here. I mean, this is a chip startup that we have not talked about too much on the show. How far are they in their chip development process? How big is the startup? What do we know about them? So for a starter, this is a startup that Anthropic actually discussed buying earlier this year. And, you know, it's a startup, like many of the expiring NVIDIA challenges, you know, like they make chips. And then for Maddox's case, you know, they are saying like, you know, their chips can be optimized for both like, you know, inference and training, which, you know, like these startups, these are like, you know, focusing on like on inference.
13:29They have not really have their first chips yet. It's like scheduled to like, you know, tape out, which is like, you know, a first prototypes of their like chips. That will not be until the first half of like 2027. So, but it had raised a lot of money from, you know, like Gene Street, Leopold's situation of awareness with lots of kind of big investors behind it. And, you know, it was last value at$4 billion in February. And it's now trying to raise another round and are talking to investors to at least double that valuation or even more.
14:00Stephanie Palazzolo:Is KOTU an investor in Maddox? I'm not yet, but like KOTU is discussing investing is like a new round they're raising. But we're not sure if KOTU is going to invest yet and how much and what is even a valuation. It's just like new startup. But lots of their rivals like Edge has raised like a$21 billion valuation. So the market's very hot right now. And so if Maddox is looking their tape out for their chip is coming down the road here, sounds like this is a pretty proactive initiative for Maddox. It's sort of anticipating, hey, once we do get our chip to market, we don't want to be in a position where we are struggling to get capacity and stuff like that.
14:46I feel like it's one of the many challenges startups may have because, you know, when the whole world is very constrained on these like supplies and big chips, like NVIDIA or Apple or Google, they can just, you know, use their kind of leverage because they're like one of the world's biggest companies to lock down all this capacity from like the chip suppliers or like FTSMC or elsewhere. And so like this is effectively, you know, trying to help them to overcome this like challenge down the road because they're like a big financial backers behind this.
15:15Stephanie Palazzolo:So Valida, do you anticipate that we're going to see more joint ventures like this pop up between big investment companies, between chip startups? I mean, we've certainly seen the agreements where investment companies will help the hyperscalers with their own chip efforts. I'm talking about the TPU, the Google Broadcom deal, I believe it was, to help get the TPUs out into the market. Are we going to see more tie-ups like this with the startups? I felt like, you know, JVs are like now increasingly popular. And, you know, we don't expect this one to be like the last one in the specific chip financing venture.
15:59We know like, you know, some other investors may have also proposed this, like, you know, other chip startups, too. But like, you know, in this way, this one is the first one that's like, you know, reporter or like, you know, we know that it's like, you know, coming along. So we felt like, you know, as these big financial institutions are like trying to help expand the market and, you know, they will try to, you know, create some more creative or like innovative ways to help these startups to get what they need the best way they can. Great.
16:29Stephanie Palazzolo:Well, Valida, I want to thank you for coming on. That is Valida Pau, our Deals reporter here at The Information. with ai cyber threats on the rise following the open ai attack on hugging face companies are ramping up their security budgets for more on that i want to bring on my colleague aaron holmes for a conversation about a story that he published this week on that aaron welcome back to the show it is great to have you here hello so you have chronicled the cyber security sector for many years now. You were once a cybersecurity reporter, turned enterprise software reporter, turned Microsoft reporter, but cybersecurity is still near and dear to your heart.
17:10Stephanie Palazzolo:And every time you've come on the show, you have helped us understand how this sector very much fluctuates. I mean, there's booms, there's fatigue. Now AI is here. Where are we at in that story right now? Yeah, you know, I think we're about six months out from Anthropic sort of announcing mythos and warning the world that we were going to enter this new era of AI threats. And in the six months since then, there's been sort of a broad recognition across the industry that cybersecurity budgets need to go up. And I talked to almost a dozen cybersecurity executives for this story, and pretty much all of them said that they're increasing their budgets for the year ahead, anywhere from 20 to 30 percent in a lot of cases.
17:52And, you know, that money is going to both AI models to help protect against potential AI threats and also to new tools that they say they need to keep track of how their employees are using AI. Essentially, a whole new raft of security threats across the board that companies are in many ways scrambling to protect against.
18:12Stephanie Palazzolo:So when you say they're spending on the AI models, does this mean that the labs are the big winners here? Or are there specific pockets of the enterprise software cybersecurity tools that are also benefiting a lot from this? Labs are certainly big winners. You know, there's models that are specialized for cyber tasks like Mythos and, you know, OpenAI Cyber Models. But even outside of that, you know, there's a ton of companies that are using just the publicly available models from those labs to scan their software to look for vulnerabilities. And we're also seeing, you know, big incumbents like Palo Alto Networks, Microsoft, CrowdStrike all build new AI agent tools that in most cases rely on models from the lab.
18:59So, you know, a lot of the new spending going towards those types of products are also benefiting the labs at the end of the day. Right.
19:06Stephanie Palazzolo:And I mean, let's let's dive into the labs cybersecurity strategy here, because one way that you just mentioned is they have the models that are specifically built to help uncover some of these cybersecurity threats and stuff like that. It sounds like, though, they're also sort of diving deeper into cybersecurity altogether with product offerings that are a little broader than that, too. Right. Yeah, that's right. We're seeing them also, you know, put out these applications that, for example, OpenAI calls them red agents and blue agents, which are, you know, specific agents meant to try to hack into company systems and see what vulnerabilities they can find and then suggest ways to patch those vulnerabilities.
19:46And, you know, what's interesting is that's potentially going to disrupt this sort of older category of vulnerability scanners where we've had players like Tenable, Qualys, Rapid7, you know, using kind of pre-LLM technologies to do similar vulnerability scanning for companies. Of course, all of those firms are now also building, you know, LLM powered tools that in some cases use the bottles from these labs. But some of the cybersecurity executives that I spoke to said that they're rethinking, you know, how they choose between those more legacy vulnerability scanners and the fact that LLMs can now do a lot of that work themselves.
20:25And so which way are they favoring then in that tradeoff? You know, I think so far a lot of people are still in the testing phase, but I have certainly heard from some who say that they are getting more bang for their buck out of the LLM powered tools. At the same time, you know, some of these companies like Tenable, for example, has still seen a surge in revenue in the last six months, which, you know, Tenable CEO told me is because customers are increasingly, you know, wanting their vulnerability scanners, including the newer kind of AI infused products that Tenable is building. But, you know, for some other companies, they seem more on the rocks.
21:02For example, you know, Rapid7 actually had falling revenue in the most recent quarter. And they also had to cut 12 % of their staff, which they said was about trying to refocus their business on building AI native tools. So it's definitely roiling this sector of the industry and the broader cybersecurity world.
21:20Stephanie Palazzolo:When you talk to executives in charge of cybersecurity at these businesses, I mean, there's a lot of demand right now in this moment for protection. As we talked about at the start of the segment, I mean, this fluctuates, right? I mean, cybersecurity is well known to cause fatigue. Sometimes people just get tired of spending so much on protection, especially if they don't see the results. Do you expect this demand to last? How are the CISOs that you talked to thinking about what results they're trying to see and how they're measuring success? What do you think? So the perception right now is very much that the landscape is favoring hackers right now over defenders.
22:04That has a lot of these companies very nervous. And, you know, that's because in theory, any hacker with rudimentary coding skills can use a tool like, you know, OpenAI Codex to supercharge their ability to find vulnerabilities. You know, even though models from OpenAI in theory will refuse to carry out cyber attacks, we've heard that there are ways to kind of jailbreak that. There's also open source models that hackers are using. And so across the board, you know, I'm hearing from companies who are saying in the short term, we need to spend heavily on, you know, models to defend ourselves and other products to defend ourselves.
22:38And at the same time, you know, I think some companies are hoping that maybe in the next six months to a year, there will be more of an equilibrium where they'll, you know, have gotten their defenses up and hackers don't exactly have, you know, one foot forward against them. But in the near term, there's definitely a perception that companies need to quickly spend more to catch up with these threats.
22:59Stephanie Palazzolo:And how would they inevitably measure the success? It's kind of like you're almost measuring a negative, right? In a sense, it's like, should I be hacked, it will cost me this many millions of dollars to address it. And I guess I'm just trying to prevent it. So it's a little bit like you're trying to monitor something from not happening, right? It's true. Yeah. I mean, like the successful state in cybersecurity is nothing happening. But obviously, you know, which is hard to measure ROI on then, right? Right. Although there is, you know, a lot of companies are using these models to just scan their code for vulnerabilities.
23:40And they say the amount that they're finding is, you know, skyrocketing. You know, Palo Alto Network said that the amount of vulnerabilities they found their first month using Mythos was something like five times the amount that they would typically find in a regular month before that. So there is definitely like a rush in new findings from companies that are using these models to scan for undiscovered vulnerabilities.
Read the full transcript
24:04Stephanie Palazzolo:Got it. Aaron, before I let you go, I want to ask you about another story that you published. You had some reporting on Eleven Labs. What's the latest on that company? Yeah, so Eleven Labs, the voice AI startup, has just hired a CFO. And essentially, they are now looking at potentially going public by 2028, which would make them one of the first of these AI labs after potentially Anthropik and OpenAI to go public. and it's interesting because, you know, like every other AI startup, they are also a customer of OpenAI and Anthropic and will need to see if they can convince investors that, you know, their business can stand on its own two legs outside of the success of those labs.
24:50How big is the company now? They recently disclosed that their annual recurring revenue had surpassed$600 million and they are profitable. But at the same time, their valuation most recently was in the high billions. And so they'll have to convince the market that their revenue matches that multiple, meaning they'll probably have to keep growing
25:16Stephanie Palazzolo:at a consistent rate. And they'll have to see how the OpenAI and Anthropic IPO before that inevitably goes. I trust that will sway their decision quite a bit. But it was a great scoop. And Aaron, I want to thank you for coming on. That is Aaron Holmes, our Microsoft reporter covering enterprise software, cybersecurity, and everything else that businesses care about here on TITV. This morning, the Information's Amazon reporter, Catherine Perloff, and our Enterprise Software reporter, Kevin McLaughlin, published a feature story on how six engineers at Amazon Web Services have overhauled the company's entire AI bedrock platform.
25:58Stephanie Palazzolo:There are some great inside details on how this all happened in the story. I want to bring on Catherine to walk us through their reporting. Catherine, welcome back to the show. It's great to have you here. Hey, Akash. So, Catherine, can you just remind us, I mean, Amazon Bedrock, what is it? It's the platform where everyone can access all the models in the cloud that AWS has, or what is it exactly? Yeah, it is the service that you use if you want to run anthropic models on AWS, if you want to run now OpenAI models on AWS that they were able to access a couple of months ago. So it's the kind of inference service of AWS.
26:42You know, kind of comparable. Microsoft has Foundry. We've heard about a lot of companies coming, startups coming online to help companies run inference together, which focuses on open source models. So this is a way of running models within AWS, which people like to do because that's where their data is. It's more secure, et cetera.
27:00Stephanie Palazzolo:And so, I mean, how good has Bedrock traditionally been? I mean, your reporting seems to suggest that there were some issues with Bedrock in its early rollout. Yeah, it's had a bit of a rocky road. You know, especially last year was a tough year because that was when inference became really popular. So, you know, companies were using anthropic models a lot, you know, to do tasks. And they would sometimes run into issues when using Bedrock. For example, you know, you get sort of a set amount of capacity on Bedrock. And if you want to use get more, you know, some customers could wait up to three weeks to get more.
27:42You know, customers were sort of getting throttled when they tried to use, you know, more of the models. And, you know, internally, there was a lot of concern about these technical difficulties. you know, folks in the sales teams were saying the issues with Bedrock were preventing AWS to, from getting customers. And, you know, you have to remember at that time in 2025, Microsoft was the only cloud provider where you could access open AI. So that was sort of a disadvantage AWS had. Now AWS does have the open AI models, but at that time, you know, it was like, you know, we really need to have a platform where people can access AI models.
28:19So other cloud services like azure like google cloud which had gemini don't make market don't get market share and luckily aws did have a leading you know model on its service which was anthropic but it wasn't always seamless to access that model via aws so you know how does it how does it ultimately end up addressing those issues yeah so a bunch of senior engineers at uh amazon started you know redesigning sort of the engine backend for Bedrock in something called Project Mantle. So they had to sort of, you know, make a new system where they made a kind of a couple of key changes
29:08to better design a system for AI inference, which they realized works a little bit different than standard compute for web services. They sort of designed Bedrock like they had other computing systems that, you know, AWS has been kind of expert in for years. And I think at the beginning of the AI boom, especially, this was more okay. People were using AI for more simple tasks like chatbot services. But as people started doing, you know, more complicated tasks with AI, especially using agents, which could mean a bunch of different types of workloads. So you could, you know, have a workload that takes really long and then takes up a lot of compute and then suddenly not use that much at all.
29:53This kind of called spiky in the cloud business. Right. Right.
29:57Stephanie Palazzolo:This idea that some some tasks take a whole lot of compute and you get a spike. Or on the flip side, I mean, you have these long duration tasks that I guess you have these agents running for a while in the background. You know, I will encourage folks to read the story because there's a lot of interesting detail about who these six engineers are that were really at the center of it. But, I mean, big picture, Catherine, so they've implemented these changes. They've overhauled Bedrock. Are customers happy with it? I mean, are all these problems gone now, or what's the status of that? You know, I think that, you know, customers are happier.
30:34First of all, Bedrock has been growing a lot in the past couple of months. Amazon said in the second quarter it was sort of like its best quarter in terms of usage and customers, like compared to like all of the quarters previously. So definitely growing a lot. Part of that is because they fixed some of the issues. Part of it is that they have open AI models on the service now. So that makes it more compelling, especially at a time when, you know, we've talked about how enterprises are more picky with AI models and switching between models, you know, when they can save money, using them for different tasks, etc.
31:08And I did talk to a few AWS customers, actually folks who work with a lot of different customers, kind of more consultant types, that did say that, you know, in recent months, they had more of their clients spending more on Bedrock and switching from either using Microsoft Azure or OpenAI's own API for accessing their models, which can run on Azure or other cloud services.
31:32Stephanie Palazzolo:So AWS has taken some share away from Azure, it looks like, in this department. Yeah, I think a lot of businesses, you know, might have started using Azure when they maybe wouldn't have already because of OpenAI being there. And now Bedrock was sort of able to, you know, bring them back or bring them back from, you know, maybe just using Azure itself because now they're kind of being able to show that you can run models on their platform. It's fast. It's not buggy. It works. Having said that, you know, capacity is a big issue throughout the industry. And, you know, we do have we have heard of, you know, a few examples that we, you know, is in the article of people still having some issues with, you know, the new bedrock with Project Mantle and sort of having, you know, delays in using the service.
32:19So it's not perfect, and it is a real challenge to run an inference service because, you know, basically you're managing people using a bunch of different, you know, doing a bunch of different workloads at once, which are really expensive and hard to manage. So I think it's something they're still working on, but it has gotten, you know, better.
32:38Stephanie Palazzolo:So, Catherine, you know, we've been covering Amazon's AI approach for a while here on the show. I mean, a couple months ago, I believe they sort of scaled back a bit on their own model ambitions with Nova. And, you know, well before that, Amazon, in some ways, it was seen as a bit of a laggard in AI, although it sort of made back its ground there. I mean, I wonder, is this part of a broader shift in strategy that AWS is pursuing here with respect to Bedrock or perhaps being a more nimble organization? The fact that six engineers were at the center of this. I mean, what did all this tell you about how AWS is thinking about AI broadly?
33:23You know, I kind of feel like the market sort of swung in a direction that benefits AWS in a way. Like, you know, more so than they, I think they were trying lots of different things. And this was sort of the, you know, the sort of AI bedrock. I think it's the AI play that has worked for them much more than their own models. And I think that, you know, originally at the beginning of the AI boom, we didn't really know how businesses were going to use AI. We thought maybe they would train their own models or, you know, everyone would have their own model. We didn't know that there would be such a sort of two-man race in the frontier models.
33:56And it became clear now that, like, you know, very expensive to make a frontier model. You know, only a couple companies are really able to do it well. And then sort of the layer of competition becomes, like, infrastructure and who is good at the infrastructure of running these models, which is very expensive for businesses. And this is an area where, like, AWS is already pretty strong. But we know it's, like, a big area because that's where all the, you know, new startups are. They're in, like, the NeoClouds. They're different inference services. You know, infrastructure is sort of the key layer that matters to businesses.
34:29Even a company like Open Router, you could kind of call infrastructure because it's helping businesses choose between different models. And I think that AWS is kind of already poised to succeed here, probably more so than, you know, in AI research, which was never really their big thing. but um you know even for them it was challenging because again ai is a new type of compute which is kind of what our whole article explores but they they've been able to sort of figure it out and i think so far um because the ai race yeah has changed to who is best at running these frontier models they're positioned to win because they already had advantages and they've figured out some of their their stumbling blocks but they probably have more competitors on their heels than they ever have because it's become such a dynamic area.
35:13So I guess we'll have to see.
35:15Stephanie Palazzolo:Right, right. Yeah. Great. Well, Catherine, I want to thank you for coming on. That is Catherine Perloff, our Amazon reporter here at The Information. That does it for today's show. A reminder, we are on the stream Monday through Friday at 10 a.m. Pacific, 1 p.m. Eastern. If you can't make it then, episodes are available on theinformation.com, on our YouTube channel, or wherever you get your podcasts. Make sure to follow us on social media on X, on Instagram, on TikTok, and on LinkedIn. I'm already excited for our next show tomorrow. Have a great rest of your Tuesday. Bye-bye for now.
From the publisher
Stephanie Palazzolo talks with TITV Host Akash Pasricha about OpenAI's new Astra model and its AGI claims. We also talk with Valida Pau about Coatue's proposed chip financing joint venture with startup MatX, Aaron Holmes about rising AI cyber threats and ElevenLabs' 2028 IPO plans, and we get into AWS's major Bedrock overhaul with Catherine Perloff.
Articles discussed on this episode:
- https://www.theinformation.com/articles/anthropics-house-payments-tech-push-chip-away-stripe
- https://www.theinformation.com/articles/coatue-matx-talks-new-multibillion-chip-financing-venture
- https://www.theinformation.com/articles/ai-threats-reshaping-companies-spend-cybersecurity-budgets
- https://www.theinformation.com/newsletters/ai-agenda/elevenlabs-hires-cfo-eyes-2028-ipo
- https://www.theinformation.com/articles/six-aws-engineers-rebuilt-bedrock-challenge-microsoft
Subscribe:
YouTube: https://www.youtube.com/@theinformation
The Information: https://www.theinformation.com/subscribe_h
Sign up for the AI Agenda newsletter: https://www.theinformation.com/features/ai-agenda
TITV airs weekdays on YouTube, X and LinkedIn at 10AM PT / 1PM ET. Or check us out wherever you get your podcasts.
Follow us:
X: https://x.com/theinformation
IG: https://www.instagram.com/theinformation/
TikTok: https://www.tiktok.com/@titv.theinformation
LinkedIn: https://www.linkedin.com/company/theinformation/
