949: Why AI Keeps Failing Society, with Stanford professor Alex “Sandy” Pentland

26:56Jon Krohn:And so would you say that these same kinds of solutions, like an AI buddy that can be giving you tips on what people are doing in a different domain that could be relevant to what you're doing? Yeah. So we've built something and you have to look at sort of community deliberation with my name. But there's an archive paper about exactly this. These are the tools we're building. It's actually pretty simple. It's an AI buddy that says, where is there in the space of ideas around you a place that nobody else is looking? You know, so there's two things you need. One is people aren't looking there. So you can see that through the citations and the data.

27:40It's not a subjective thing. It's an objective thing. And you can also say, hey, but this blank spot that nobody's looking at has lots of people nearby. so that if I can figure out something to do there, then what it's going to do is it's going to be of interest and affect everything else. And it turns out those blank places are the places where you get superstar papers. You know, the papers that are cited a bajillion times and, you know, change the science. So that is an AI buddy that's helping us with our blind spots. It also turns out that you can build the same AI buddy and run it on patent databases.

28:21And what it will do is find you places that there's lots of people patenting things, but not in the center. And so you can say, oh, what am I going to do in the center here? This is likely to be commercially very, very valuable. And if you put the two together, it's like, OK, where am I going to put the research money and where am I going to put the development money? And then the third one that's really funny is that law works the same way. So when they do legal decisions, they cite other people. And you can figure out areas of the law that are going to have a lot of, let's say, churn. And you probably want to stay away from them.

29:00Maybe you want to dive in, but I would stay away. Great.

29:04Jon Krohn:It's nice to have your insights on what we can do society-wise with AI Buddies, as well as at the academic level. It's incredible, all the different initiatives that you're associated with. Another comment that we came across from you or another perspective that you have is that you've argued that data have become a primary means of production, your words, and that just as labor and capital eventually gained unions and community banks, you argue that communities will need their own data unions to counterbalance corporate concentration. I had never heard of an idea like this. Do you want to tell us more about it?

29:42Well, so first of all, it's pretty clear that data is a means of production. And I will cite sort of the loudest voice in the world, Xi Jinping, who's the head of China, right, who says this. He says, you know, so that's really weird. Here's this guy who's Marxist. That means that his whole theory is around capital and labor, right? And he's saying, no, no, no, no, it's no longer capital and labor. It's capital, labor, and data. And if you control the data, then you can manipulate the other two. So that's what the head of China is saying, not just me. He apparently did get it from me, but that's okay, right?

30:23Happy to give away ideas. And of course, that's where all the hyperscalers come from too, right? It's all like huge concentrations of data. And then you ask, well, so what happens to people like me or you? Well, we're sort of at the mercy of all these people who know a lot of stuff about us. We don't know what they know about us. And they're trying to get us to buy things, right? It's one of the reasons that personal agents are interesting, because personal agents don't have to share data to get results. They can look at what's out there. They can ask specific questions, but they don't have to tell you, oh, I'm a male this age.

31:06I make this much money. you know, it can sort of be a lot more sort of private inquiry sort of model. So that's actually really interesting that it's a chance to sort of take back your data. The reason the loyalagents.org thing is with consumer reports is because when you get a group of people who are using their data to figure out what things are scams and what things are a good deal and what works and what doesn't work, then you have the power to be able to begin to compete with fairly big people, people like Walmart, right, who know a lot about you, but now you know more about you than they do.

31:48And by sharing among the community, cooperative, you can operationalize that to get better prices, avoid scams, all sorts of things. This is not a new idea at all. So in the mid-1800s, the only sort of banks in this country were in New York City and Boston and Philadelphia. And all the people who lived to the West were getting killed because they had no financial access except on exorbitant terms. And so all these agricultural communities started agricultural banks, cooperatives, and broke the monopoly of the Eastern banks. It was something that looked impossible. But by pooling their resources, they were able to do this.

32:38Those same co-ops, incidentally, are the ones that electrified America. You hear about government initiatives to, you know, put in electricity and stuff like that. But the majority of our electric grid came from communities building their own bloody grid, right? Because the government wasn't doing it, and they didn't want some company to own their electricity. So they built their own, right? And it's where we have credit unions today. That's where they've sort of evolved. After a while, the laborers noticed that this was happening and said, well, gosh, we're going to have our own unions. And these are unions of labor.

33:15And famously, this is the beginning of the 1900s. And eventually people changed the laws and said, no, you can't require kids to work and you can't require people to work more than a certain number of hours a day, et cetera, et cetera. And so there's this battle that happens between these means of production. And that's what Marx was about. But now we have the battle with data. And I think the same solutions that work with capital and with labor will also work with data. And I say this sort of AI agents type of thing where you get to not share your data, but just share questions and answers. If you gather people together into a co-op, you can have something that's much more powerful than you alone could ever be.

34:01Jon Krohn:I like that. I like that you keep coming back to the agent theme since it's so topical. I don't even have to guide you there. The thing about the agent theme, right, is, yeah, some of it's hype. Maybe a lot of it's hype. But it's a different way to organize compute and data and money also for that matter, right? And whenever you get that change, there's an opportunity to do things in ways that we will all enjoy. It hasn't happened for two generations or a generation, at least. Now we have an opportunity to sort of change the basic blueprint of the web. That's something to jump on. Yeah, yeah. And so beyond agents, you've shaped the architecture of global data governance.

34:51Jon Krohn:This is going to be a bit of a long question, so you're going to have to bear with me here, Sandy. But you've co-led the World Economic Forum discussions in Davos that produced GDPR. You served as one of the UN Secretary General's data revolutionaries who forged transparency mechanisms and the sustainable development goals. Your work spans advising the UN, G20, OECD, founding the Trust Data Alliance, building open source infrastructure that makes AI and data safe, trusted and secure. and now we face unprecedented threats from AI-generated social engineering, so things like phishing, computer viruses, misinformation engines that we've already talked about earlier in this episode.

35:34Jon Krohn:And in the final chapter of your new book, you propose a solution, which is, well, a whole bunch of solutions beyond just agents. So there's things like what you call ex-post accountability through open audit trails, strict liability, trusted execution, and continuous monitoring rather than pre-approval gatekeeping. Do you want to highlight any of these solutions? I'll tell you, first of all, a little funny story. So I was helping this thing called the Club de Madrid, which is all the prime ministers and presidents from Europe. And we were talking about governance of AI, right? What are we going to do?

36:12And a lot of the people there were the heads of the EU commission, the regulators. And I was saying the solution I'm going to tell you. And they said, you're too Anglo-Saxon. We're going to go another direction. So what you're about to hear is Anglo-Saxon. But it's not just Anglo-Saxon. I think it's something that works in lots of parts of the world. Not so much Europe. The idea is that, look, unless we have sort of consistent regulations about AI and its use and performance. And we're talking about things like, is it self-dealing? Is it biased? Is it doing the right thing at all? You know, unless we have consistent regulations about that and ways to check, it's just not going to happen.

37:01And consistency is not within the country. Consistency is across the whole world. Because people are building data centers everywhere in the world, and your query is going to happen in wherever the energy is cheapest. That may not be in the continental U.S. And the things that you buy, just like today, don't necessarily only involve U.S. production. There's logistics chains from overseas. So what are the systems we have that actually work consistently across the world? Well, one thing is purchasing systems. You know, when you buy something, you buy it. You buy it with some amount of money. You can convert it to other money.

37:41If they screw up, you can sue them. Well, sometimes it's not so good, but, you know, at least people agree what counts as being screwed up and what doesn't count. If you don't get the thing, they owe you the money. Now, you may have good luck finding that money, but, you know, at least they agree that that's bad. Whereas today in AI, nobody agrees about anything. But what we can do is we can treat AI the way we treat money or trade and say, well, look, you don't give away money without writing down who you gave the money to or where the money came from and what it was for. You have an audit trail so that, you know, in case bad things happen, we can find out what really happened.

38:25We need the same thing with AI. I mean, we have logs of all sorts of stuff on computers anyhow. Not a big deal, but you need to have logs of what your AI is doing to who with what data. And if you have that, then you can find when you're running off the rails, you can detect which agents are better than other agents. And you can do this internationally. And so what you could begin doing is you can say, well, look, country X, your agents really suck. Here's the audit trail, right? Right. And so they're going to like raise their standards. Now you said maybe they're going to fake the audit trails.

39:04There's ways to get around that through encryption, zero knowledge proofs, stuff like that. So you need to have audit trails for AI and you need to be able to have open, have them open so that people can look and say, is this what we want? Because sometimes the harms emerge over time. A lot of things that are wrong with social media are not things you can see in a day or two. They're things that occur when you have it running for a long time in a changeable situation. So you need to be able to have audit trails that go back that far. And that's really what the suggestion is, is, look, we don't know what regulations we need and what we don't.

39:46We know that in things like, say, electricity, we don't have like 37 ways of how you have to use electricity. We just know that electrocuting people or causing fires are bad. And we're going to approve designs, not law, but regulation standards to keep that from happening. So like, for instance, you know, on every electrical cord in the United States, there's a little UL tag. that's a cooperative, a not-for-profit cooperative that tests things and has standards. And if a company tests their thing and meets those standards, then they don't get sued as often, and they don't get a black eye in the press.

40:30That works. That works everywhere. And we need to begin that way until we find out things that are particularly, things that AI does that are especially bad. And we want to be able to detect those quickly and jump on them. So that's the suggestion.

40:50Jon Krohn:So it might not be the most surprising thing if not too far in the future, the big frontier labs agree to have the same kind of regulation as that UL electrical example that you had where we can kind of trust AI systems to a certain extent and not burn the house down. Well, so that's already beginning to happen. There's a California law that got passed that says that the big guys have to reveal what they do for AI safety, right? They have to publish that openly. And so now it's pretty interesting to go and compare who's a little more cautious than whom. And now we're going to be sort of asking, like this is part of the stuff that we're doing at Stanford with Consumer Reports, which ones are really sort of in your court and helping you and which ones are just like fleecing you, right?

41:37Because you'd like to know that, right? And the fact that they have their safety things, you can begin to correlate that with what they do and figure out which safety things work and which are just hot air.

41:49Jon Krohn:Right, all right. Thank you for summarizing the things that are happening today with that AI safety regulation as well. So we've talked a lot about data regulation, data governance. We've also talked about your academic work, your book. I would like to now with the kind of the final chapter of this podcast episode to talk about your entrepreneurship. So your entrepreneurship program has spun off more than 30 companies, including the largest rural health care service delivery system in the world, secure privacy preserving payment solutions, a popular delivery route optimization platform and a biotech startup that reduces livestock methane emissions with microbiome interventions.

42:30Jon Krohn:So some amazing enterprises or startups there, companies there that all have a positive social impact. So I don't know if you have general advice for the audience, maybe lessons learned about scaling innovation responsibly, ensuring that beneficial social impact isn't just a byproduct, but a design principle from the get go. Well, I think there's probably two things that I would sort of recommend. And one is, is you need to look where other people aren't looking. And most people are pretty short-sighted. They're looking to make a buck and that's about it. But a sustainable buck is a more fundamental problem that nobody's really doing anything with.

43:14Like the healthcare thing, we put simple AIs on phones before there was Wi-Fi, right? And now nurses and doctors and things like that could begin serving people and helping pregnant women and stuff like that. People just weren't thinking about it. They said, oh, that's too high tech. We can't bring that into these poor countries. Well, actually, things get better pretty fast. So that's the other principle. For a given level of performance, the cost of AI halves every three and a half months. Think about that. That makes Moore's law look like the slowest thing in the world. It's 10 times the performance per cost each year.

44:00It's like, oh my God. And so you can begin doing things that sound really crazy, but by the end of the year, it won't look so crazy. And two years from now, it'll look like, why didn't I think about that?

44:16Jon Krohn:I just really appreciate that entrepreneurial advice related to your entrepreneurial advice is maybe some of your research that bridges both academia as well as having potentially lots of real world commercial applications. you helped pioneer the concept of the living laboratory, designing research that unfolds within real world human contexts. Tell us more about that. And as kind of a follow on question, once you've told us about these living labs, tell us about how it might evolve with new technologies like digital twins and synthetic data becoming so capable, you know, following the same kind of, I don't know if digital twins, actually they must be things like digital twins in synthetic data would be following that same kind of 10x performance per cost capability that we're seeing with LLMs in general.

45:11Jon Krohn:So yeah, tell us about Living Labs and how that 10x multiplier per year is impacting them. I mean, again, go back to sort of the social media thing. Social media was supposed to be the greatest thing ever when it first got released. Oh, we're all going to be have a voice. Isn't that wonderful? And then it turned out over time, things were a little darker than we thought. And the only way you can really tell that, other than people sort of finding little data to look at, is to have a group of people who agree to live in the future. So you can have a town that says, look, we'd like to try out AI transportation systems.

45:50What do you think? And if everybody says, sure, let's give it a shot. Then you get to see how well the AI transportation system does in a real world thing. And you get to see the unanticipated effects because those are almost always the ones that you care about is the things you didn't think of and turn out to be significant. It's one of the things that we're doing with the Consumer Reports Loyal Agent Project is trying to ask, well, what does this AI agent stuff do that we aren't thinking about and really ought to be doing something about now? So it's having people live in the future, giving them this sort of state-of-the-art or beyond state-of-the-art sort of tools to do.

46:33And so digital twins are part of that because what you're really trying to do is understand human behavior with this new component like AI agents. And so you'd like to be able to try different things, but you can't try everything with the people because they've got to get on living their lives. So you build a digital twin. So you build something that says, well, this is what each of these people or these people in aggregate. this is how they behave. This is a good model of what we've seen. And you try to validate that it's a predictive model of what you will see. So now you can look at different designs and say, well, what's this going to, what are people going to do with this?

47:18Are they going to like run off the end and drown in the pool? Or what are they, is it going to be great? And the way you do that is you build a model of the people, digital twin. In fact, the digital twins are possible and not just possible. That's really where we're going when we talk about these things. The compute is dropping dramatically in cost, and it's hard to believe how fast that's dropping. Data, there's more and more data. The big question about the digital twins and the Living Lab is how do you keep the data safe? How do you make sure that when you do these experiments, you don't do anything really bad?

48:03And digital twins help a lot with that, right? Because they aggregate the data in a way that can preserve privacy, for instance. And if the digital twin does something really stupid, at least you haven't killed somebody, right? You better go back. Rethink what you're doing, but it's a way to avoid the investment and the harms. All right.

48:24Jon Krohn:Fascinating view on digital twins there as well, Sandy. Thank you so much for that. This whole episode has been sensational. Really appreciate your insights across society, academia, entrepreneurship. We've benefited so much from your rich experience throughout this episode. Before I let my guests escape, I always ask for a book recommendation. Do you have anything for us other than your own book, which, of course, I'll mention again now, Shared Wisdom, new release. If you enjoyed today's episode, I'm sure you'd love picking up a copy of Shared Wisdom. Sandy, what else do you have for us? Well, I think a book that I've seen just recently that I think is really interesting is a thing called Flash Teams.

49:11and it's Valentin and Brookstein, I think it is. But basically it's how you can build new products using these AI tools really, really fast. I don't know that this is the technical book that you want, but it gives you examples and sort of stretches your minds in ways that I've been alluding to, but it gives a lot more in the way of examples.

49:36Jon Krohn:Excellent. Thanks, Sandy. Yeah, it doesn't always need to be a technical recommendation. And we actually get a lot of kind of like novels recommended on the show. And so, yeah, it's your choice. Appreciate that one there from you. And final question for you is just how folks after this episode can be following you and getting your thoughts. You know, should they follow you on social media? And if so, which platforms? Yeah, certainly you can follow me. I post sort of news of what we're doing on Facebook, on X and LinkedIn. And so any of those work, you can just sort of Google me and see. They're usually Alex Petlin with either underscores or dots or something like that.

50:26Jon Krohn:Well, we'll also, we should have them in the show notes for folks as well so that they can find it very easily. Thank you so much, Sandy. really appreciate having you on the show and maybe we can, we can check in again in a few years and see how research on your side is coming along. I'd love to. Yeah, that's great. Thank you very much. Good questions. Enjoy the conversation. I'm lucky to have an amazing researcher, someone named Serge Massis, who's a great data scientist in his own right. Very, very kindly spends a lot of time getting great research done. And yeah, so tracking me down. Right.

51:04Yeah. OK, good. Good. Exactly. All right. Yeah. Thanks again and catch you soon. Good. Take care.

51:13Jon Krohn:What an honor to have Professor Alex Sandy Pentland on the Super Data Science Podcast. In today's episode, he covered how AI systems fail when they treat humans as logical engines rather than social foragers who learn by watching others. He talked about the parallels between Soviet central planning through early AI and today's systems, why Washington, D.C. residents wanted AI to defend them against city bureaucracy rather than optimized city services, revealing what real people actually need from AI. Data unions as the next evolution after labor unions and agricultural co-ops and living labs and digital twins as tools for testing AI systems in real world contexts before deployment.

51:54Jon Krohn:That's it. As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Alex Sandy Pentland's social media profiles, as well as my own at superdatascience.com slash 949. Thanks, of course, to everyone on the Super Data Science podcast team, our podcast manager, Sonia Breivich, media editor, Mario Pombo, partnerships manager, Natalie Zajski, researcher, Serge Massis, writer, Dr. Zahra Karche, and our founder, Kirill Aromenko. Thanks to all of them for producing another super episode for us today. For enabling that super team to create this free podcast for you, we are deeply grateful to our sponsors.

52:29Jon Krohn:You can support the show by checking out our sponsors links, which you can find in the show notes. And if you ever want to sponsor the episode, sponsor the podcast yourself, you can get the details on how at johnkrone.com slash podcast. Otherwise, help us out by sharing the episode with folks who would like to listen to it, review it on your favorite podcasting app or on YouTube, subscribe if you're not a subscriber, but most importantly, just keep on tuning in. I'm so grateful to have you listening. And I hope I can continue to make episodes you love for years and years to come till next time.

52:58Jon Krohn:Keep on rocking it out there. And I'm looking forward to enjoying another round of the super data science podcast with you very soon.