Ranking AI Labs on Safety, AT&T’s Open-Source Pivot, Stripe to Buy AI Marketplace OpenRouter

0:13Welcome to the Informations TI TV. My name is Rocket Drew. It is Friday, August 21st. Today on the show, we'll take a closer look at Stripe's acquisition of OpenRouter and what the fintech giant might have in store next. We'll then look into whether Anthropic and OpenAI should be worried about the rise of companies using more open-source AI models. And to close out the show, a look at AI safety standards and which companies are currently getting graded an F. It's going to be a fun show, so let's get right into it. FinTech giant Stripe has agreed to acquire OpenRouter, a fast-growing platform for accessing and routing AI models.

0:54We're breaking down the deal in this week's edition of The Editor's Cut with our co-executive editor, Martin Peers, and senior editor, Meredith Mazzilli. All right, welcome on the show, both of you.

1:05Aaron Holmes:Rocket, how are you? Doing well. Good day. Doing well. Let's start with you, Meredith. What do we know about this deal? Yes, go ahead. I just say that it's a relief to have you on rather than a cash. I know. It was high time we did something about that guy. Sorry, Meredith. So let's talk about Stripe and OpenRouter. Big news this week. Stripe officially announced it's acquiring OpenRouter. They did not disclose terms, but reportedly going for north of$7.5 billion, which is much, much more than the company's valuation of about$1.3 billion when it last raised money in May. And this deal comes as Stripe's business has really been exploding.

1:50I think we reported revenue growth of around 30 % last year due largely in part to AI payments. So developers paying for AI models, and that's all flowing through Stripe. So that's all put Stripe in a position to be really acquisitive. It's trying to get broadened out beyond its core payments business. And this is the latest way it's doing that. So growing fast, but a pretty high price tag given its last valuation. Martin, what does Open Router actually do and why is it that Stripe wants it?

2:24Aaron Holmes:Well, it is a kind of a store. If you're a business that wants to use AI models, you can go to Open Router and it makes it very easy for you to sign up to use any AI model for your purposes. And you can pay for it through one billing system so it just it makes it easier and i think all of the major um model providers make their models available through it i just want to correct one thing though just the impression that we got from our last speaker who i don't know i don't know her um but actually open routers growth has been much more than 30 percent it's its growth has actually tripled um in the last quarter that's why Stripe is looking for.

3:12Aaron Holmes:It's just business is absolutely booming because far more companies are now using AI. And this is one place that makes it very easy to use the technology. Meredith, do you want to respond to that? Does that sound right to you? So I always talk about Stripe's business, actually. The AI flowing through payments for Stripe has helped them a lot. Obviously, Open Router is growing a lot more quickly. But I think that this deal makes a lot of sense um i don't know about the specific price tag but you know kind of to martin's point um if you think about this in e-commerce terms this is a way for stripe to have its own marketplace which it hasn't traditionally done it's powered payments for online shops and and some marketplace type setups but now it's really getting into the marketplace game itself which is super interesting and it's also the way for stripe which is you know not being at the core of the AI boom to get into the middle of that market, which is really important.

4:16I see. So that tells us something about how Stripe is seeing the future of AI, the future of its own business.

4:22Aaron Holmes:Well, I mean, Stripe isn't in the AI business itself. It's not developing models. It doesn't run any sort of cloud service. But this gives it a way in. So you can understand why they are looking to do it from that point of view. Right, right. They're seeing growth in this area already. It's a new opportunity for them. Open Router is going really fast and stands to run away with this market, which could potentially be pretty big. Stands to, but we should also point out that there's lots of other companies that are either already in that market of, you know, offering people access to, you know, other routing companies or there are others moving into it.

5:09Aaron Holmes:So there's no guarantee that Stripe will, that this will sort of work out for Stripe, particularly given how much they are paying. So there's risks from other competitors to open router. Is it also a risk that this category of routers stops being such a big deal a year from now? Routers are really in the spotlight right now because people are feeling the costs of really expensive models from OpenAI and Anthropic, and they want to be able to route to cheaper models. But if OpenAI and Anthropic start to be the only game in town for one reason or another, do you see that as a risk for this deal? Rocket, I think you're more of an expert on that than we are.

5:47Aaron Holmes:But I would think that that's not that much of a risk because the open source models in particular are obviously less expensive than the leading models. So I can't imagine that businesses will concentrate all of their AI usage on the top two. That totally makes sense. So after weighing out these factors, where do you both land on this price tag? Reportedly$7 billion to$7.5 billion. Who's getting the better end of the deal here? Is that more than you would have expected Stripe would pay for Open Router? I think it's a crazy price. I think this is what happens when you are private and you can pay with stock.

6:30Aaron Holmes:You don't really think about how much you're paying, but that's just me. Meredith, what do you think? I mean, I do think it's crazy, but I do think it makes sense on a lot of levels. I think Stripe wants to get in early on the next big potential marketplace. Like we talked about before, that might be a miss. You know, it doesn't necessarily have to be open router. I think another risk that actually came up in some of our finance coverage is that you're seeing companies now start exploring ways to sign contracts on a token basis. So the open router allows you to route based on a given price. Those prices themselves can be volatile.

7:12And it's possible that there's some other model that comes in where instead of trying to bounce around, companies are kind of locking in forward prices. That's very early, but just kind of one risk I see. But yeah, I mean, I'm actually pretty, I think this deal is very interesting. It makes a lot of sense. We also reported this week that OpenAI is using OpenRouter to offer subsidies that basically help underpriced competitors there, which kind of makes me think about the other piece of the marketplace model that might develop down the road is if Stripe sees a way to kind of capture those subsidies directly.

7:54And what I'm getting at is like, is there a world where there's more, you know, advertising or promotion that's happening on these marketplaces where Stripe can capture some of that? Obviously, that's a challenge given they're, you know, I assume supposed to be neutral, but that did raise the question in my mind of just how much of that promotional aspect turns into part of the business. That would be fascinating to see. You could even imagine them routing two models that are more inclined to advertise for certain companies over another without inserting advertisements themselves.

8:27Aaron Holmes:I'm not sure I fully understand that. And I also, sorry, Meredith, I don't mean to be itchy, but how can something be crazy and make sense at the same time? It's, that's a good question. I mean, I think it's an early bet that they want to capture the next big marketplace, but it's certainly not a given. Meredith, I'm trying to have it both ways, Meredith. Basically, yeah. I'm just saying it bluntly. This is a crazy price. It doesn't make sense. Meredith, there's also reports that Stripe is thinking about striking a deal for PayPal. How realistic do you think that one is? And would you see these two fitting together?

9:10So it's definitely looking a lot more realistic these days. There are some reports that PayPal is now actually engaging in the consortium that includes Stripe that previously put in a unsolicited bid to buy the business. And I think it'll be interesting to see if anybody else jumps in. Certainly raises the prospect that Stripe and its partners would have to pay more and how inclined will they be to that. But there are certainly pieces of PayPal that make sense for both Stripe and various other fintech companies. It's just how does that all break down? I think ultimately Stripe would take something like the Venmo business, for example, consumer wallets, not the whole thing.

9:57So it wouldn't be a huge acquisition itself. It does not truly fit with the AI piece of the business that we're talking about right now. It really is more in the core, at least right now, more in the core commerce business, just deepening there and boosting margins. Okay, but not obviously at odds with each other either. Not at odds with each other, but I think it's still very unclear how all these pieces will fall into place and what other bidders might emerge. Martin, what do you think? Would it be crazy or would it make sense?

10:30Aaron Holmes:I think it would make sense, particularly if they are only buying a piece of it. Well, I don't totally agree that if they were to buy Venmo, for instance, I think that's one of the most valuable pieces of the company. Certainly, I think the piece that is growing the most, I think that would be quite expensive. And I think this also points up the fact that Stripe is not public. It is not easy to use your stock as a payment when it's not freely traded and it's harder for the, you know, the person who's getting that stock in return to know what it's really worth. So this is one of the issues that points out maybe Stripe should consider going public.

11:17Aaron Holmes:But obviously, that's something that they haven't wanted to do so far. All right. Well, thank you both for coming on and breaking down the deal, the parts that make sense and the parts that are a little crazy. So thanks again. Thank you, Rocket. Anthropic and OpenAI had better hope that more companies don't follow the example of AT &T. The company is aiming to use more open-source models, as models from Anthropic and OpenAI become more and more expensive. Our colleague Aaron Holmes wrote all about it in yesterday's Applied AI newsletter, and he joins us now. All right, welcome, Aaron. Hi, happy to be here.

11:53So what did you learn about AT &T's current approach to using AI? Yeah, so I mean, like most enterprises, AT &T is finding that the cost of running AI internally is significant, And they're basically looking for ways to keep that in check, you know, specifically that the main usage of AI that they have is internal. So, you know, they have 100 ,000 employees that use AI for everything from writing code to looking up, you know, information for customer support staff while they're on a phone with the customer to filling out HR processes. And specifically, you know, I was told by a vice president that they are now using open source models for around 40 % of that AI usage and, you know, want to get it even closer to 60 or 70 % in the coming years.

12:40And, you know, part of that goal is to keep the amount that they're spending on OpenAI and Anthropic flat, even as they use more AI overall. And how do these open source models compare to OpenAIs and Anthropic? They're cheaper. Are they also worse? Yeah, I mean, they're definitely slightly behind the frontier. I think, you know, especially for things like generating code or doing sophisticated coding tasks, you know, I'm told that AT &T and other companies are still, you know, willing to pay top dollar for Anthropic or OpenAI models for those types of tasks. But, you know, open source can also take over a little bit, you know, less advanced tasks.

13:19For example, if a software engineer just wants to generate a summary of all of the AI generated code that they did in a day, they could use an open source model to create that summary. So I think like they, like a lot of companies are trying to use model routers that kind of automatically choose open source versus frontier models, depending on how difficult the task is. So did you get a sense of how much they're saving overall from using open source models? Well, AT &T processes about 45 billion tokens or, you know, small pieces of text per day for its internal AI use. So, you know, just looking at the sticker price from OpenAI and Anthropic, that would cost in the hundreds of millions of dollars annually, at least if they were only using those labs.

14:04I don't know exactly how much they're saving, but I was told that they're able to keep spending, you know, comfortably less than that by relying on open source for nearly 40 % of them. Yeah, yeah. So the savings could be really significant there. Did you get a sense if they're customizing these open source models or developing themselves in any way? they are they um you know have been trying to customize nvidia's nemotron models as well as um you know metast llama and google's gemma i think that that's a relatively early effort but you know they've built kind of internal benchmarks and ways to specifically evaluate how well these models work on you know their own internal uh policies for for generating and submitting code for example um and they also have been running these models you know locally on their own data centers, which obviously there is a cost to setting up your own data centers.

14:56But once you're doing that, it essentially makes it, you know, a trivial cost to run the models, which is also something that they say is helping them keep costs low. Those models they're using are all from companies in the US. They're not using Chinese open source models, which tend to be better. Correct. I was told that they're analyzing, you know, DeepSeek and Moonshot's Kimi models, but so far they're not using them for work. And, you know, I was told by AT &T that that's because they're evaluating potential risks. I don't know exactly, you know, the reasons for that, but a lot of companies are nervous about, you know, potential political backlash from using Chinese models, especially given that the White House has somewhat signaled that it wants to crack down or at least, you know, scrutinize the way that those models work and are being used.

15:42Yeah, I don't think anyone truly understands what the risks are or how real they are. But But on a vibes level, the decision makes sense. Are there other ways that AT &T is using AI that you've learned about? You know, I think that for the most part, the internal use cases are the bulk of how they're using them. I think it also kind of remains to be seen, you know, how successful they'll be in using open source going forward. I think that, you know, for the most part, that this is something that companies are constantly evaluating an open source from what I'm told is like six to ten months behind the frontier, although AT &T told me that they think that gap is narrowing.

16:24But one big question is just whether it will actually remain cost-effective to keep using open source models. Obviously, there are a lot of other costs that go into managing and maintaining your own models versus just using them out of the box through an API. And, you know, one investor recently told me, like, open source is free in the same way that a puppy is free. Like, it might seem out of the box to be a low-cost option, but there might be, you know, costs that come up related to maintenance or just getting the models to work correctly in your environment that you might not anticipate. Sure, and potentially you use even more tokens or you run them for longer without running into rate limits.

17:07It's yeah, the puppy analogy makes a lot of sense. Did you get the sense that they're trying to hold firm on some ceiling of how much money they're willing to spend on OpenAI or Anthropic, or they're just in general trying to keep down the amount that they're spending? You know, I was told that this isn't necessarily like a very strong ceiling, you know, specifically the way that AT &T described it to me is, you know, ideally, we would like to keep our spending on OpenAI and Anthropic Flat, if not decrease it by using open source. But they're very much aware that that might not be the case. And it might be the case they end up having to spend more on foundation models if that ends up being where their business needs go in the future.

17:49So I think it is very much a wait and see, but this is the goal that they've put forward. Or to your point that their spending on open source models could balloon and then their overall budget goes way up. What does this mean for the battle between open source and closed source models sort of overall for the industry right now? Yeah, I mean, I think that right now we're seeing, you know, Anthropik and OpenAI specifically trying to make pitches that, you know, keep companies spending on their models. And, you know, one theme that has come up a lot just this week is this idea that, you know, open source, running open source on your own infrastructure might give you more control and more privacy than putting data into an API managed by Anthropic or OpenAI.

18:32And so I think it's interesting that we've seen both OpenAI and Anthropic come out with announcements this week that they are going to basically roll out more privacy protections for enterprises, give enterprises more ways to do confidential AI computing or make sure that their data is not being retained by Anthropic and OpenAI. And to me, that seems like kind of a direct response to some of the growing interest in open source and in, you know, self-managed AI that we're seeing from enterprises. So I think it'll be really interesting to see if that pitch from OpenAI and Anthropik proves, you know, something that will sway enterprises to spend more on their models in the years ahead.

19:11Yeah, I thought it was striking how quickly Anthropik came out with their announcement right after OpenAI made theirs. It makes me wonder whether Anthropik would have done that on its own at some point, or, you know, it seems like open eyes announcement put a lot of pressure on them. Do you ever read on that? Yeah, I mean, I think like, for the most part, this is something that we've seen the sort of every player that's not anthropic and open AI has been attacking them on this front. I mean, we've seen Satya Nadella and Alex Karp come out in recent months and basically tell enterprises, like, you should be worried about how the foundation labs are storing your data and what's happening inside the black box of their models.

19:51And we've also seen a growing chorus, not just open source, but from those other software firms essentially saying, we'll help you manage open source or manage your own models in a way that OpenAI and Anthropic can't. And so I do think that the labs are this week trying to one-up one another as they both race to get faster enterprise revenue growth. But at the same time, I think they're facing this kind of industry-wide pressure where everyone who's not named Anthropic and OpenAI is looking for whatever pitch they can make to make open source or any other alternative seem more appealing to enterprises.

20:29Great. Well, it will be really interesting to see how it develops, and we'll have to have you back on to continue to chronicle how it's going. So thanks for coming on and breaking this all down for us. Thanks. The AI industry is still reckoning with the cyber attack that OpenAI's models performed on Hugging Face. Many questions remain, including how AI labs can stop their models from hacking other companies in the future. Our next guest has some takes on these questions. Steven Adler previously worked on safety at OpenAI, and earlier this year he co-founded GuideLight, a non-profit that is developing safety standards for AI companies.

21:06This week, GuideLight released a scorecard comparing how well different AI companies are prepared to control their wayward AIs. Welcome on the show, Stephen. Thanks for having me. Absolutely. So we love a good scorecard. They always have some juicy details for us to talk about. Can you break down what exactly you're measuring in your scorecard and then who's acing it and who's flunking out? Yeah, of course. My organization, Guidelyte, developed standards for avoiding the catastrophic risks of AI. And then we go and assess companies against these. The scorecard that we published this week is for our control standard, which is specifically on how companies can stop their AIs from doing bad things, kind of like the open AI and hugging face incident.

21:51Or if you play the tape forward to more capable AI systems, potentially things much worse than that. It looks at things like when an AI company is using one of these systems within their company to do very important work, like securing their network or developing future generations of AI models? Do they kind of keep an eye on what their AI system is doing? Do they scan it for misbehavior? Do they have the right preventative systems? The headline is we are a far away from companies having implemented even the basic control practices. We go through and we score each frontier AI company on a scale of essentially zero to five and no company scored higher than three on anything.

22:32There's a lot of yellow, orange, red for now, but my hope is that companies improve at this significantly over time. Okay, that's a great breakdown. Just to understand, to begin with, though, is it possible for companies to do well on the test? If no one is doing above, say, a C level, what would it look like for someone to get an A plus? Is that even conceivable? Yeah, it's a great question. The things that our standard includes are essentially the basic minimum practices that if you query the technical AI safety community, they will say, these things are all achievable today if a company is motivated enough.

23:08It might mean investing more in safety than companies do by default in this competitive dynamic, but these are all quite achievable. And often companies are part of the way there. There's an additional step that they could be taking if they chose. One thing that I think is really important here, there are some things we might have to do in the future to safely contain AI systems that do involve novel research. They involve big breakthroughs. None of these are like that. These are really, really simple, straightforward techniques, sometimes even simple procedural things like having a containment plan in place to understand if your AI starts misbehaving, how you will coordinate off and what it would mean to safely return it to use.

23:45These are all things doable today, but unfortunately companies haven't invested enough as of yet. Got it. Okay. Yeah. Having a plan for containment seems pretty straightforward. So what companies got what grades? Who did well? Who did badly? There were three tiers roughly across the companies. OpenAI and Anthropic tied for first. They each got a C +, which kind of underscores how much room for improvement there still is. Trailing the pack were XAI and Meta. These each scored less than one out of five on average across their practices. Significant room for improvement. And then in the middle was Google, which to its credit has actually put together a quite detailed forward document for how it intends to handle control in the future.

24:29Unfortunately, as of today, it is significantly less implemented than controls at Anthropic or OpenAI. Okay, so XAI and Meta should copy the homework of Anthropic and OpenAI a little bit. It won't get them an A, but they might pass. So we'll get to what this means for OpenAI and hugging face, but let's start with Anthropic. I mean, they have the reputation of being the most safety-focused of the companies on this list. It looks like according to your scorecard, that reputation is mostly deserved. Yeah, that's right. I mean, they are tied for first with OpenAI. As I mentioned, there's still a long, long ways to go, but here's an example of a thing that Anthropic did really well that I would like other companies to do.

25:13Most of the companies that we assessed participated in this really hands-on deep assessment with the risk assessor meter earlier this year. but for Anthropic they gave Meter much deeper access to actually go in and independently test out their control systems within the company most of the other companies they essentially filled out a survey they gave Meter access to a model to run evaluations but they didn't let Meter go inside the actual company like an employee would and test out the controls and see if they work based on public evidence only Anthropic has done this to date and so though multiple companies participated in this third-party adequacy review, which is part of what our standard calls for.

25:56Anthropic is the only one that really had someone get in there and test the actual system at hand rather than relying on something much less informative. And Meter, interestingly, is one of the two organizations that OpenAI has now called in to diagnose what went wrong in the hugging face situation. So maybe if they had gotten in there to begin with, they could have caught some of the things that went wrong in that scenario. Do you think that's plausible? Yeah, I think it depends on the exact scope, but I do suspect this is the type of thing that Meter and their embedded red teaming identifies.

26:30You know, where are the holes in companies' control systems? And there's also a question of to what extent OpenAI actively knew about this hole. It might be that Meter said to them, would have said to them, there's a gap here, and the company might have risk accepted that anyway. Because as we've seen, there are still these basic gaps that surely companies know about and yet are making the decision not to invest fully enough to close this gap. Sure. And even Anthropic is not best in class for a couple of the categories on your scorecard. What are the two places where they're sort of dropping the ball right now?