Anthropic’s Lead in Safety with C+ Grade, Walmart’s Ad Growth Beats Amazon’s, Chinese Custom Chips

22:40How does Amazon respond to that? Is it making moves to hold on to that lead that it has? Yeah, you know, I mean, Amazon's still a much bigger business. Amazon was like a$60 billion business last year. That's compared to Walmart being about$6 billion. So much bigger, been the ad business longer. Amazon has been sort of diversifying the kind of places it sells ads. So, you know, again, I mentioned the ads on Prime Video. I mean, Walmart doesn't have like a popular streaming service like Prime Video to sell ads on. They have Vizio where they're ads on, you know, TVs, but that's probably not as much.

23:13Like people aren't looking at the home screen of their TV as much as they're like watching a TV show. So Prime Video, definitely an important surface for them. Also, Amazon has been really investing recently in having, like, very competitive advertising technology and sort of taking the place that Google once had. It still has in a lot of ways as, like, the dominant sort of internet ad broker. Amazon has been really investing in that in recent years. And then, you know, Amazon has, like, so many other kind of businesses that we might not think about as much. But Twitch, you know, Audible, Alexa, they have their Wondery Podcast Network.

23:53These are all places that, you know, they sell ads on as well. They're not necessarily, you know, we actually got into this issue in an article I wrote like a month ago. Those places aren't necessarily as popular services for advertising as like Prime Video and Amazon's website. but they still like, you know, they're very like diversified. So, you know, I don't know if Amazon needs to so much like worry about Walmart quite yet. I guess, I mean, the one thing that Walmart has is the physical stores and that's been something that they have been, you know, not as big in and they've kind of stumbled in a bit.

24:33They do have Whole Foods and they have some other physical kind of properties. But, and, you know, Walmart can use that data. But, you know, I think Amazon has a pretty good lead for now. Yeah, the ad brokerage side is interesting. I definitely have a pet interest in those kinds of ad exchanges and the auction structure. But we'll have to get into that another time. The last question I have to ask you, you wrote that one former Amazon retail executive compared advertising to heroin. This is very provocative. So what do they mean by that? I'm sure, as you imagine, Amazon PR loved that. But, so, I mean, basically the idea is it's like something you can get hooked on.

25:13I mean, we've reported that Amazon's, when Amazon kind of calculates the profitability of its stores business, it often includes advertising. Walmart similarly has said that advertising and also membership, so sort of it's kind of equivalent to Prime, has driven a lot of its profits. So there's a risk that, like, you know, you kind of ignore the profitability of your main retail business, which is low margin because the growth you can get from advertising. That's one thing. But also, I mean, you know, advertising is just a great way to make money, but you can annoy consumers getting people complain about seeing ads on Amazon a lot and probably on Walmart, too.

25:51And then you can also annoy merchants. There's a whole antitrust case built partially on the fact that advertising is kind of a tax on merchants on Amazon where they feel like they can't be seen at all unless they pay for ads, which, you know, the government is arguing is unfair. So, and, you know, long term could annoy merchants from using to from selling on Walmart, especially when they're not the dominant e-commerce platform. So, you know, I guess you got to you got you have to be careful. to not over, you know, put your pedal on the gas or yeah. Yeah, yeah. That adds some fun controversy to the whole thing.

26:28Well, thanks for coming on and explaining it to us, Catherine. Thank you so much. Concerns about the risks from AI models are top of mind in the AI industry right now, since the cyber capabilities of its AI models landed Anthropic in hot water with the White House recently. A new report out today ranks major AI companies in terms of their AI safety practices. Joining us to discuss its findings is Max Tegmark, an MIT professor and chair of the Future of Life Institute, which is the AI safety organization behind the report. All right, welcome on the show, Max. Thank you so much. It's a pleasure to be here.

27:05Well, one of the headline takeaways for me is that no one is doing very good in this report. I mean, Anthropic did the best out of everyone, and it got a C+. So the situation is not looking good. Can you explain that to us? yeah maybe not the kind of report card that the kids are happy to bring home to mom and dad but it's not particularly surprising you know the saying show me the incentives and I'll show you the outcome you know right now AI is the only industry in the US that's less regulated than sandwiches so if companies aren't incentivized to uh have safety standards and so on you know then they get into this sort of race to the bottom which i think is reflected in the grades i see and that incentive comes from just the push to develop the business and cut corners in terms of these guard rails yeah this is nothing new for ai you know we saw the same thing in pharmaceuticals back before there was an FDA, there was a company that sold this soothing syrup for babies to make them sleep better at night.

28:13It had such high amounts of opioids in it that many babies never woke up again. And it was anger about things like this that ultimately gave us the FDA. But now in pharma, we instead have a race to the top, right? Where companies apply a lot of talent and funding to make sure to race to see who can be the first to make a safe product. The FDA analogy is an interesting one. I don't know if you saw, but Howard Lutnick recently said that there will be no FDA for AI. What did you make of that? It feels very telling then. Well, there's been a massive pivot from the White House in the last few months, actually, because before then, pretty much everyone in the White House was saying, no regulation, no regulation, no regulation.

28:57And then some folks in the White House started saying, actually, maybe there will be an FDA for AI. And right now, they're sort of duking it out. I think what changed was Mythos, which came in as a massive wake-up call. If a company is basically selling offensive cyber weapons, then a lot of people in the national security community went like, wait a second. Maybe we should be informed of this. Maybe we should even have a say. Imagine if some company started to sell bioweapons in supermarkets or nuclear weapons, right? Of course, the government would want to have a say in this. And sure enough, the government came in and blocked Mythos and Fable 5 and still have a block on ChatGPT 5.2.

29:48Yeah, well, I'd love to get into that and also what you see as some of the solutions there. But first, let's dig into the results from the report a little bit more. So you separate this report card into six categories, and I want to zoom in on a couple of them. One of those categories is information sharing. Anthropic got a B +, Meta got a D+. I was a little surprised that some of the scores were so poor because I think of information sharing as the place where we have the most AI policy at this point. We have state laws in California, New York, as of yesterday, in Illinois as well, that require some at least minimum thresholds for transparency and information sharing.

30:26So why isn't that enough for everyone to be getting an A in this category? Well, first of all, the Illinois law only came into force yesterday, right? So it hasn't incentivized the companies until now. I hope we'll see strong improvements. But there's still a lot of companies who basically share information to the point of saying, our stuff is totally safe. Trust us. It's sort of like if you go to the supermarket and buy a product and it says healthy on it, but there's no ingredients listed. And some companies have done a lot more than others. Anthropic came out on top there because they have actually published a lot of stuff which to some extent incriminates their own products, right?

31:11They've released work showing that their own AI system can, in some controlled experiments, blackmail the guy who's trying to turn them off and threaten to leak his extramarital affair unless he promises to not turn it off, You know, stuff like that. So bonus points for companies who have done this. But you see there's a vast range, right? Down at the bottom of the list, we have three companies, one from China, one from Europe, one from America, who have shared almost nothing about their safety practices. So a lot of room for improvement there. Definitely. The other category that caught my eye because of room for improvement is existential safety.

31:53So I'm hoping you can explain to us what you mean by that. But basically, everyone bombed. Most companies got an F. Anthropic and OpenAI got a D+. This is a big topic for you, I think. So why are they falling short? it yeah this is an interesting one you know the top ceos elon musk sam altman demis asabis daria amade all signed the statement about three years ago saying that ai could cause human extinction and uh the reason they said this of course is because what they're trying to build is not just a better chat bot they're ultimately trying to build what's called super intelligence so if you imagine robots in the future that are vastly smarter than us, more nimble than us, and also stronger than us, and can do everything we can do just vastly better, then we shouldn't think of it as just a new technology like air conditioners or electricity or the internet.

32:49It's like a new species because those super smart robots roaming around earth doing stuff, they can build new robots, they can make copies of themselves, they can reproduce. And it's been known since the origin of the whole field with Alan Turing, that, uh, that tends to be the smarter species that dominates the planet, right? If you go down to the zoo and ask who's in the cages, is it the animals or the humans, the reason that's not the humans is because the smarter species takes to be in, tends to be in charge. So, so the reviewers of this report basically found it kind of jarring that the same people who are saying this could cause human extinction and are trying to build the super intelligence didn't actually provide any plan for how they're going to keep humans in charge of this tech and this has become more urgent recently because you know five six years ago ai couldn't even talk properly right but now that we already have since then we've gone from AI that was kind of high school level, the kind of college level, to PhD level, to professor level, to beyond in some areas.

33:57And we have Dario Amadei from Anthropic talking about a country of geniuses in a data center next year. These very well understood risks that if you build a new species, you got to be careful, you know, have shifted from being some kind of far future thing for our grandkids to worry about to something we need to worry about this decade. in terms of the report though is there anything these companies could have done to get an a like what would it look like for companies to do well in this category according to you well to add some optimism here you know the easy fix is i think to start by just treating the ai industry like any other industry if you had if you treated it like sandwich shops for example you know they just can't sell any sandwich until the local food inspector has checked for rats in the kitchen and stuff like that right so and if you look at pharma right if we never lie awake worrying that someone is going to release a new medicine that that has a side effect that it causes heart attacks and half the people who take it because we know they can't even launch it until they've demonstrated the safety of the medicine right so if we had anything like that for ai like you mentioned an fda for ai well what happened then is it would be still very easy probably to get approval for the cancer-curing AI and all sorts of wonderful productivity-boosting AIs.

35:22But they would probably get rejected companies when they try to release an AI girlfriend for 11-year-olds or something where they can't demonstrate convincingly that it won't teach terrorists how to make bioweapons. And can you imagine someone coming in and saying, hey, I have this super intelligence I'm going to release. I don't know if it's going to overthrow the US government or not. obviously the review the the government review the review panel would be like uh okay you haven't done your past your clinical come back when you when you have a better case you know next customer please bing so so what we would get then is basically a golden age of ai where that we continue to see massive innovation and all the things that the governments can demonstrate have more benefits than harms from curing cancer to amazing revolutions and autonomous as vehicles and this and that, and so much of what people are excited about, right?

36:14And then this sort of freaky stuff, all out human replacement would just have to go to the back of the queue until companies, maybe in 10 years or 20 years, and have some better ideas for how to make that safe. But the key point is the vast majority of what pretty much everyone I talk to is excited about using AI for, right? Does not require some sort of weird sci-fi super intelligence. it requires controllable tools i see so it sounds like to do better on the report card the companies would have to do a significant change in their strategy or at least their sort of long-term goals for the models that they're developing i think many of the goals they can keep it's just this super intelligence business that uh i think they would be forced by any meaningful safety standards to chill out on a little bit you know right now if you're building any other product if you're building a selling a teddy bear or you know you don't make it also have all sorts of unwanted functionality like if you press the this ear too hard it explodes or something you just make it have the functionality the customer needs whereas with ai we tend to do the opposite right even if it's an ai that actually operates in a teddy bear and parks with a kid there was one happening it's happening more and more yeah they put all of chat gpt in it that has all human knowledge and starts telling your baby about snm and so on right so as soon as there is any kind of incentive to have some meet some safety standards what companies will do instead is make narrower ais that are that are to do just the tasks that the user wants for different products so for example uh there's no need that reason why you and i when we just use AI to help us with some of our own work, need to have any capabilities of that AI for making bioweapons.

38:13You can have a separate AI that's used by expert biologists working in biosafety level four labs, but we shouldn't have all these dangerous tech powers in everything. It's just like, if you go to a janitor in the Pentagon and ask what are the nuclear codes, he or she isn't going to tell you, right? I hope not. Even if you're very mean to them and torture them because they don't know. And there's simply no reason for every single AI system to have all of this knowledge in it. Yeah, this point about specialization is interesting because I think it's one reason that people are preferring open source models these days, just the degree of customization that you can get out of them.

38:55But I noticed that on the report, open source models and the companies developing them actually didn't fare as well. I'm curious, is that because they're working on open source models? My sense was like DeepSeq, for example, is lagging behind. Mistral did pretty poorly. Yeah, sort of indirectly. They did get some credit, the open source companies, for information sharing because, of course, sharing that makes it easier for other people to study them and understand how safe they are and so on. DeepSeq scored really low because, frankly, neither DeepSeq nor Mistral nor XAI had really presented much evidence at all that they were really putting even working much on safety during this last half year.

39:43ZAI from China that's behind GLM 5.2 scored higher because they have released more in the way of their safety strategy and so on and you could you could totally imagine open source systems which are great for translating between english and japanese or great for all sorts of very productivity boosting things that will no matter how much you tweak them never teach you how to make a bioweapon because they just weren't trained on that knowledge yeah yeah yeah that makes a lot of sense The only company, in fact, in the report that improved its score since the last time you did the report was Meta.

40:22It went from a D to a glorious D+. What happened there? How did they improve their score? It did jump from sixth place to fourth place also in just six months. So hats off to Meta. You should, of course, ask Mark Zuckerberg and others what the ultimate reason behind this is. But our panel felt that it was really clear that they had decided, they had made a corporate decision to just try much harder on safety. They've been producing safety framework and they've clearly put a lot of people on their staff into thinking about these issues. And I think this is an encouraging data point for the rest of the industry.

41:04You know, if it's possible for Meta to improve so much in just six months, you know, so can everyone else. yeah are there any particular people on the staff that you would point to that you think are pushing them in that direction well i don't want to i know a lot of people admit that i don't want to sit here and and uh and uh and gossip about individual people sure that's a different report card but i i that's one maybe over beer in private All right, well, that sounds great to me. I look forward to that. Yeah, but it's quite interesting to see how fluid the situation is. And I think what's also really, really fluid is the political situation in the U.S.

41:51because I've been quite active, not just with my technical research at MIT on AI, but on these policy issues for well over a decade now. And I've never seen such a big political change as I have in the last three months in the U.S., where the administration, I think, basically had a wake-up call because of mythos. And as a result, is now talking very openly about having some safety standards and so on. And the EU AI Act, you know, comes into force next month. China went ahead last fall and required all AI content to be labeled as such. and also really crack down hard on AI girlfriends and anthropomorphic AI.

42:33So I think we're now approaching a situation where there is a minimum safety bar that there's actually unanimity for all around the planet. And I think that might, in the next six months, translate into some really nice incentives for the companies around the world to start racing to the top. rather. Yeah. I guess sometimes I hear this argument in the US that's like, even if you got your favorite policy position and you could just fiat it into existence, you create the FDA for AI or what have you, and you have some kind of licensing regime, it won't matter because six months from now, the open source models will have caught up and everyone will have access to mythos level cyber capabilities.

43:19So what's the point of licensing? What do you make of this argument? Yeah, I don't buy that. I think it's mostly a talking point from some silicon valley lobbyists who want to avoid regulation at all costs you know first of all china has an ai sorry it has an fda for medicine right why do they have that not to suck up to the us but to protect chinese and and then the chinese fda people and the american fda people and the european ones they like to get together and harmonize things to make it easier for companies that meet the standards in one country to now get to sell the other right and i i think the same thing is happening with ai also why did china crack down on ai girlfriends obviously because ai girlfriends were really wreaking havoc on chinese kids right and do we really believe that the ccp is excited about the getting the chinese government hacked with chinese open source cyber weapons of course not uh if anything the ccp might be even more concerned about loss of control than than western governments are right so they have their own very own incentive to keep the chinese companies in check and i actually suspect that the chinese government knows more about what the chinese companies are up to than the american government has insight into its companies right now.

44:45So I think it's going to start locally. Each country will do stuff, introduce safety standards for egoistic reasons. But then again, once they have that, there's a lot of incentives for them to talk to each other, right? Both to share best practices and also to make it easier for companies to cross-market products. Well, it's going to be an interesting six months then for sure. So thanks for coming on, Max, and telling us about the report. Thank you for having me on. Absolutely. Well, that does it for today's show. As a reminder, we're on this stream Monday through Friday at 10 a.m. Pacific or 1 p.m.

45:23Eastern. And if you can't make it then, episodes are available on theinformation.com or our YouTube channel or wherever you get your podcasts. Make sure to follow us on social media on X, Instagram, and even TikTok. And I'm already excited for our next show. So have a great rest of your Tuesday and goodbye for now.

45:44Thank you.