Breaking down the 2026 Stanford AI Index Report

0:01Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

0:41Welcome to another episode of the Practical AI podcast. Today, it's just Chris and I, my co-host and I in what we call a fully connected episode where we try to keep you updated with some of the things that are happening in the AI news and maybe share some practical information that will help you level up your AI and machine learning game. I'm Daniel Whitenack. I'm CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI and autonomy research engineer. How are you doing, Chris? I'm doing good. I'm excited. The episode we're doing today, we've done a number of times over the years.

1:24The Stanford AI Index Report, we get to go through it. It's always fun. and kind of level set kind of how things are changing. And gosh, I mean, things are changing so fast right now. Yeah. And for context, so some of you may or may not have listened to our previous episodes where Stanford's Human-Centered Artificial Intelligence Center Institute, I forget the exact of what they call themselves, but the human centered artificial intelligence effort there at Stanford, they published this AI index report and they've been doing it for a number of years. We've talked about it before. If you're interested, we're not going to go into like how it was created.

2:15It's very rigorous. It's very data driven. You can go back and listen to episode 276. we had some representatives on from Stanford that actually shared, you know, what it is, how it's created. And I'm sure that's updated somewhat over time, but that would be a great context for today. But there's a lot of takeaways here, Chris. And I think, you know, maybe we'll get through all of them. We can try rapid, rapid fire here to talk through some of these and share them with the audience and see, see maybe our reaction to, to some of these. Some of them were a surprise to me, to be honest, Chris. Yeah, there always are.

2:54Because, I mean, it kind of brings you back after, you know, with the rigorous approach they have. We all have these perceptions. We're all watching the news and all the AI hot things that are out there. And there's times where it kind of level sets you a little bit. And then other times it kind of goes, and I I mean, you know, just kicking us off on number one on their top takeaways list right off the bat. We kind of this is one of those places where we were going one way and then it didn't take the report. We were kind of we kind of realized that things were changing back. But for a while, we were pretty convinced open source models were going to completely catch up with plateau models because that's the trend that we were seeing for such a long time.

3:39We realized a little while back that that wasn't happening for a variety of reasons, which we've actually talked about on previous episodes. But the very first thing they mention is AI capability is not plateauing. It is accelerating and reaching more people than ever. And, yeah, I think we're seeing that in 2026. Yeah. So one of the ways they express this is that over 90 % of notable frontier models were produced in 2025. And several of those now meet or exceed human baselines on a number of things and they go into those things. Obviously, one of the things, one of the hot takes that I'm always sharing, Chris, are these baseline or these benchmarks, let's say, on PhD level science questions.

4:31They're very, they're interesting. And I think they're some somehow representative of how we're advancing. But benchmarks in general are quite flawed. So even with that, you know, caveat in there, it does seem like there is advancement that's happening. And, you know, a lot of that reaching or exceeding human level performance is impressive and maybe scary for some people. I'm not sure. Yeah, I mean, we've gotten these frontier models in recent months that are just so capable, especially when combined with these new agentic systems that everyone's been obviously a huge topic this year and able to productively do a lot of stuff, which is creating a lot of upheaval in the job markets and how different companies are perceiving that.

5:26But yeah, I mean, it's a new, it's very different from a year ago today, I would say, if you look back. Yeah, I think like they talk about the majority, four out of five university students using Gen AI. And I was actually thinking, Chris, as like a gauge on this, I was thinking back to my own PhD, which is about five years long. Like how long would that amount of work taken me with the tools that are available now, both research wise, there was a coding element to it. There was a writing element to it, obviously. And I think like the amount of work I did was a lot. But I think with the tools now, it's got to at least cut that down by half, I would assume.

6:17And I don't know, like, obviously, universities are wrestling with this and how to deal with it. And, you know, maybe people are just getting more done in their PhDs now, which would be which would probably be good. And I know we're already blowing through. We had a bunch of items to get through, probably not going to get through them all. But I will note that I think that also translates into the workplace. I know in my own job, I am much more productive with the tooling that we are all using here. and so the notion of what not too far back it would have been like a research project and you would have been trying to think about all the things you got to do for that and now that's like, it's like I'm going to dive into it and by the end of the week I'm going to have this thing done that would have been a large body of work prior to that and so yeah, I mean it's definitely changing jobs which is another thing that I think we're going to get to down the road number two is AI model performance between the United States and China has closed.

7:19It's effectively closed. And we're looking at no longer kind of a leader follower effect, but two co-leaders in the world market. Yeah. And actually, this was one of the ones, well, I don't know, maybe there's different elements of this, Chris, I would say, just in my own practical experience on the, you know people use and govern you know both closed and open models and in our platform now so i get exposed to kind of both of those i would say on the open model side just practically china seems to clearly have the lead now maybe that's that's different on the i guess different on the closed model side.

8:05So maybe there is a little bit of nuance there where maybe in my mind, I had this perception of Chinese models being almost superior, but that doesn't really factor in the frontier model, closed model provider side of that. I mean, that's true. And we talked about that recently on an episode just a few weeks ago. And, you know, the notion of, in a lot of ways, the U.S. has kind of walked away from open models to some degree. And, you know, Meta, you know, with Meta walking away, they're now going entirely closed. And that was kind of leading the U.S. contingent. It's not to say that there aren't smaller, but in that top tier, we've kind of walked away, whereas China has largely embraced the open model approach.

8:56which I think the fallout from that in the West will be interesting in terms of how much, you know, there's a certain amount of geopolitical division across, you know, between the West and the East in terms of how they adopt models and what models are okay to use in different contexts. And so as we see open models predominantly in the large scale happening in the East, closed models only in the West, how that ends up shuffling things will be an interesting thing to watch in the months and years ahead. Yeah, and connecting a little bit more of that dynamic kind of U.S. and abroad dynamic. Takeaway number three on the AI Index report was that the United States hosts the most AI data centers, but the majority of their chips are fabricated by a single Taiwanese foundry.

9:54So this one, actually, Chris, so not the chip fabrication piece. I knew that piece, but you always hear about China just spinning up data centers everywhere. So it was interesting to me hear that the the u.s still hosts the most ai data centers because obviously you know if you're putting in a data center in some town in the united states you could get uh the local city council against you and people are up in arms and there's more hoops to jump through whereas in china you You know, a lot of a lot of that can just happen. And there's a huge amount of investment. So, yeah, this one was actually pretty interesting to me to hear that current state.

10:46It'll be interesting, like when this same report comes out next year, you know, year by year to track that and see if our if the if the United States kind of that 10 time number that's in here shrinks, maybe even shrinks very rapidly. and so we'll see. We'll see what happens on there. Yeah, yeah, for sure. So takeaway number four, we're gradually working our way through here, Chris. AI models can win a gold medal at the International Mathematical Olympiad, but cannot reliably tell time. An example of what researchers call the quote jagged frontier of AI. Interesting. It almost seems like I remember we had a guest, I'm pretty sure, I don't know.

11:34I remember talking to them. I think they were on the show. Sometimes I forget what was on the show and what conversations I had in real life, Chris. We both talk to a lot of people. Yeah. We do. But at some point there was a conversation that happened with someone from the Allen Institute for AI and a lot of what they were looking at for some time was around like common sense. So AI models can do a lot of really impressive things. But when it comes to common sense, it's like they fall over because there's no actual connection to the real world, right? They're producing tokens or they're, you know, producing, you know, tokens based on probabilities of what they've seen before.

12:18And so there's these many seemingly, seeming coherence and impressive things that happen. And then all of a sudden they can't do the most simple thing that involves some connection to the real world, like here, you know, telling time, for example. Yeah, the example they talk about is Gemini Deep Think getting the gold medal at the IMO, but only being able to read an analog clock 50.1 % of the time in terms of accurately. And they offer some other stats along the way. And I think, you know, this goes back to another topic that we've talked about a number of times. And that is we're still talking about language, you know, being what these models are trained on.

13:00And we've talked, you know, it's becoming increasingly popular to talk about the notion of what a genuine, you know, there's several names that goes by, but a world model, you know, something that actually has context for all the things in life. because you don't have that with our existing frontier models that are based on training on language. So there's a lot of research. I know that's famously Jan LeCun, one of the godfathers of AI, has talked about that many times over the last few years, the need to move past LLMs and have world models that actually have context. So and increasingly, especially, I know as my world over the last few years has gotten more and more focused on edge cases and autonomy, the notion of world models and how they would impact our field has become increasingly important.

13:56So it'll be interesting as well to see how this measures up next year in the same report as we get to that point on those kinds of upgrades. Maybe a bit of an opinion here, Chris is I almost think we're not being fair in some sense to the I don't know if it's to the models or to the the way people do AI because you know like Claude for example like an anthropic model knows nothing about the tickets in my ClickUp platform right and so you could call that model dumb. It doesn't know like blah, blah, blah. But I can perfectly well just connect Claude via Claude code skill to my ClickUp. And all of a sudden now I have all that context about what I should be working on this sprint and all of that context is there and it knows about my PRs and all of this stuff.

14:52Right. So it's kind of like what we talked about with when we were talking about Hermes agent where the no one would expect a brain absent a body to be sort of take useful action in the world and so that agent harness around which we surround the model is really part of that connection so it could be that like there are these world models and such that are relevant and that seems like good research but also like part of this is that these models need a body. They need a harness around them. Right. And we've addressed that a whole bunch of times on the show. A model in isolation doesn't do you a whole lot of good.

15:39You've got to have that connection with the world. And I think as things evolve, my own personal belief is that world model development requires the same. You have to have feedback from the real world to be to incorporate that into training to actually get you what you're looking for. And as inspiration for that, if we look at our own human brains, we're born, we're babies, we don't, you know, we have these amazing baby brains, but they haven't gotten a lot of experience against the real world. And it is all those feedback loops in those first two decades of life that kind of get us to functioning.

16:16So there'll be some sort of analog, presumably, to that notion in the world model development world as we go forward. So speaking of AI in the physical world, takeaway number five, robots still fail at most household tasks, even as they excel in controlled environments. You have any any robots in your house, Chris? Well, only the ones that most people have. We have the vacuum going around and such. But, you know, I keep seeing, you know, especially this is a much bigger thing in China than it is here with robots in production in a lot of households. I saw something just a few days ago about a humanoid robot that, you know, is doing elderly care and washes the dishes and things like that.

17:05And that was exactly what I was wondering is like, if you took that out of their version of CES and kind of explore it, I wonder when you have a somewhat unique environment, as different configurations of households and what duties, how well that really performs. And I honestly, I don't know. I haven't had direct exposure to robots of that nature. Yeah, yeah. But I want one. But I want one. I'm ready. You're ready? I'm ready. And I guess controlled environments here, they're referring to maybe manufacturing facilities or they also mention kind of software-based simulations, right? And yeah, that will be an interesting one to follow for sure.

17:54I'm just curious that when I get my first humanoid robot and we assign it the tasks around the house and I have five dogs running around the house and the dog starts jumping up on it to play, and it may be the puppy at first, but then the big dog that weighs 80 pounds jumps up on it. it will be interesting to see if it can survive the chaos of the Benson household. Listen, one of the, I forget the, so we had a friend in the restaurant industry and he gave my wife and I tickets to, I forget the name of the show. It's like the Chicago restaurant convention or something. It's basically you could go there.

18:33There's a bunch of vendors that sell products into restaurants. So everything from like utensils to appliances to software to whatever, you know and uh so big networks of restaurants go there and look at things and they had all the robots there in you know different sections that did different things and i have to say you know not having exposure to that world a ton but kind of expecting way more than i saw i guess i was fairly disappointed like most of the robots that did like cooking for example it seemed to just be, you know, like the washer dryer that spins and, you know, it spins around is basically like a drum like that, that heated up and you, it just sort of dropped ingredients in the drum and it spun around and kind of stir fried them or something like that.

19:29It's like, I was, I was disappointed that I didn't see any like humanoid robots, like chopping up, you know, making sushi or something. Very far from, yeah, the heated washing machine was very far from what I expected to see. What you're envision. I think, and while we're obviously questioning, you know, how real the performance capabilities are, I do, at least my personal belief, is that I think you're going to find a certain leveling up in China above what we've done in the U.S. and probably a fairly substantial one at that. Is that because of safety restrictions in the U.S.? I'm not actually sure.

20:14I think that my sense is that robots have just been a higher priority for quite a long time, drones and robots, obviously. And, you know, if we go back to the turn of the millennia, like looking way back, and we're looking at old school drones going up and, you know, doing performances way back before we were really even thinking about such things at all here. I think that if you've been doing something for a long time and it's more of an evolutionary step each way, whereas for us, we are surging here in the U.S., but I think we're coming from behind on the experience side of that. So it'll be interesting to see how that plays out over time.

20:57Yeah, makes sense. Well, something that is playing out over time, takeaway number six, responsible AI is not keeping pace with AI capability, with safety benchmarks lagging and incidents rising sharply. Chris, obviously this one hits close to home for me. This is part of what we're hopefully helping people deal with. But it also reminded me of, you know, of course, incidents and other things that we've seen from my work with Prediction Guard, but also a show that we had actually somewhat recently. So back in February, episode 346, which was AI incidents, audits and the limits of benchmarks, which basically hits directly at this.

21:45We had Sean McGregor on that show and he was talking about the AI incident database that he was helping create and manage. And yeah, it was very interesting, just the diversity of AI incidents that we're seeing now and the sharp rise in those. And this is also only documented AI incident cases. There's many things that are happening just anecdotally that I see that I'm sure aren't being documented in that AI incident database. Yeah, you know, I'm going to make a comment that probably will surprise most people tuning in. And that is, first of all, making it very clear I only speak for myself and not for my employer or any other organization.

22:36I actually think people would be surprised in the defense industry that there's probably more guardrails and responsible AI efforts around our industry than most commercial industries. There are federal regulations here in the U.S. that prohibit certain things that in the commercial space people might just surge and go do in terms of safety issues. And so as I was reading that earlier before the show, I was thinking, you know, I'm actually in an industry that it may hold us back at times because we're not surging the way the commercial industries have the freedom to. But I think it also, there's quite an intense focus on keeping things that need to be safe, safe.

23:23And I have noticed that. And sometimes as someone who is always enthusiastic on new technologies, I get a little bit frustrated, but then I stop and go, no, I'm glad we're that way. So I just thought I'd mention that. I think it's encouraging and a good inspiration for the rest of us to consider those responsible AI, governance enforcement, policy, whatever form that takes in your level of maturity as an organization. I think there are those out there that are pushing that direction. And we've seen development even over this last year where people, I think, are moving beyond the sort of trust me phase of AI governance towards, you know, exportable proof and even certification types of things.

24:20And I think we'll see, you know, one of my predictions this coming year, I think, is we're going to start to see some of that needed exportable proof being part of even audits and certifications, whether that's SOC 2 or AI specific types of certifications for companies. I not only agree with that, but I think that the marketplace will demand it as we go forward. I think we will continue to have some big news events where things are going off the wheels from various organizations and industries at different places, probably a variety of them. And there's going to be a point where people say, we need to know that there are safety measures in place before we're willing to deploy it within our organization.

25:04And so I think the market will demand that going forward. If you've been listening to the show over the past few months, you realize just how transformative agentic AI is, whether that's Claude Code or Hermes Agent or custom-built software that you're deploying for operational efficiencies or as new products to your customers. Regardless of your maturity now, this is the world that we're headed towards, this agentic AI world. And there's a lot security and governance teams that aren't letting these agents go into production because of risks related to agency and autonomy and how do you take care of things like prompt injections or insecure tool usage there's a lot to take care of and that's why i'm personally spending my time outside of the show working with an amazing team of ai engineers to build prediction guard Prediction Guard is an AI control plane that you run in your own infrastructure behind your firewall.

26:08Developers can build on top of this control plane using everything that they want to use, OpenAI and Anthropic-compatible APIs, MCP servers, frameworks like LangChain. But all of this is plugged into a built-in governance harness that enforces your organization's AI policies. And all of that telemetry goes back to your monitoring and alerting systems. I would encourage you to check out what we're doing at predictionguard.com slash practical AI. You can schedule a demo with me and the team, and I'd love to get your feedback on what we're doing. So visit us at predictionguard.com slash practical AI.

26:47That's predictionguard.com slash practical AI. um well chris we are almost half ish way through uh the the the takeaways we might not get to all of them in detail but number seven the united states leads in ai investment but its ability to attract global talent is declining interesting yes it is interesting it's uh i think we are and I'm not terribly surprised by that one either, as we have seen diversification and also, frankly, as political priorities in the United States have changed. I'll leave that to folks tuning in to decide kind of how they're looking at it. But I think that that has also impacted that.

27:42It's certainly hard to attract, if you're talking about global talent, it's hard to attract global talent if things that are ancillary to that, things like immigration, are a challenge. So I don't, for me at least, given the current circumstances, that wasn't a surprise to run across that. I'm hoping that we don't lose our edge in that capacity. Yeah, the number that shocked me in this report was, I guess, the scale of that. So apparently there's been an 80 % decline just in the last year in terms of the number of AI researchers and developers moving to the U.S., which, yeah, is pretty astounding.

28:28Yeah. Yeah. So it's you reap what you sow. So we'll see how things change in the in the years ahead. But yeah, interesting, interesting point. I'm curious with any thoughts on kind of future of AI investment. You know, we're still we're leading right now. Do you have any any positions on your side on where you think things will go? Yeah, I mean, I I think the the U.S. still kind of corners the market on VC driven startup startups that are funded by that world. And, you know, Silicon Valley still holds a special place there in terms of the VC funds that are there, you know, other places to New York and kind of growing markets in the Midwest and that sort of thing.

29:21So I think that's going to be that's going to hold true. The report also shows still the greatest number of companies that are being started are starting in AI companies are starting in the U.S. and maybe funded by U.S. VCs. But now in this world, I think a couple of things are true. It's becoming more and more possible to maintain smaller teams and get a lot done. And so there's less people needed to build a company that's even doing tens or hundreds of millions of dollars in revenue. But then also teams are necessarily distributed these days and maybe it's not necessary for those AI researchers or developers to move from other places around the world to those countries to do that work.

30:12So even if the companies are funded there and the VC is here, that that might not reflect where the individuals in the company live or operate. Right. That makes perfect sense. And, you know, talking about, you know, VC and the adoption rate and stuff, the last few months, the amount of AI adoption has just skyrocketed. But people were, until 2026, maybe late second half of 2025, I knew a lot of people who were not in technology and they were touching on different model programs here and there. Most of the people that I would be talking to would be touching the closed frontier models, whichever one they particularly went for.

30:59But we've seen a marked improvement in terms of adoption. I really don't think I've run across anyone recently, including all age brackets. I have friends who are in their 90s who are using the tools that are out there now. Mostly free, mostly free. And that's what Stanford had noted is the different free tiers. But that's been interesting to see. But I noticed that I think the one thing that may have surprised me was that Stanford had noticed that we are still in the U.S. ranked 24th at only a 28.3 percent adoption. So obviously I'm not I'm not the representative in terms of my own experience of that.

31:47What are your thoughts there? Yeah, it's it's super interesting. I don't know the full economics of some of these free access systems and how much usage, for example, like Google Gemini is getting off of usage on Android phones from free users versus paid workspace accounts. I don't know how all the economics play out there, but certainly I know just anecdotally, the last couple of flights I've been on, for example, I look around and of course people are on their phones and a significant number of those people on their phones are chatting with an AI app of some type and some chatting with that AI app throughout the entire flight on, you know, Wi-Fi.