Pioneers of AI: Designing AI for people and the planet, with Aza Raskin

0:00Aza Raskin:The very best founders I know are brilliant at building systems. They connect teams, they remove bottlenecks, and they eliminate single points of failure. And yet, when it comes to their own wealth, most are running a disconnected stack. A tax accountant here and a state attorney there, a wealth manager who doesn't talk to either one of them. Creative planning was built to fix exactly that. One integrated team of tax professionals, estate planners, investment specialists, all coordinated by a dedicated wealth manager who sees your full financial picture and keeps every piece working together. Proactive tax efficiency, estate strategy, investments all under one roof.

0:44Aza Raskin:Creative planning where wealth works together. Learn more at creativeplanning.com slash mastersofscale. Hey folks, Jeff Berman here. I am thrilled to share some of the new names who will be joining us at this year's Masters of Scale Summit. This may be our biggest stage yet. Reed Hastings, Meredith Whitaker, Van Jones, Amjad Massad, and more will be there with us October 20th through 22nd in San Francisco. If you're building something great or you want to build something great, we want you there with us too. Join us at mastersofscale.com slash apply26. That's mastersofscale.com slash apply26.

1:35Aza Raskin:We didn't have the right to privacy until the technology was invented that required adding privacy into American law. And that was Kodak's invention of the mass-produced camera. And once people could walk around that had a new interface were suddenly easy. There was no friction to capture images. Suddenly the elite got very interested and concerned about where they could be captured and Brandeis, one of America's most brilliant legal minds, Supreme Court justice, ended up sort of inventing this idea of privacy and adding it to the Constitution. With AI, there are new domains of what it is to be human that were inaccessible to technology before, now accessible.

2:21Aza Raskin:everything about us that isn't explicitly protected by 19th century law will end up being strip-mined. And we can see this in the form now of the race to intimacy, the race to occupy the single most intimate slot in your life. And that opens up a whole new range of harm. That's Aza Raskin, co-founder of the Center for Humane Technology and the Earth Species Project. And what you just heard, how he weaves the past into our present moment, that's something he often does. Because Eze's scope is broad. He's seeking to understand how and why the most popular and powerful technology of our time tends not to center the human experience.

3:08Or the Earth as a whole. And history can be helpful for that. In our conversation, you'll hear him reflect on several forks in the road. Moments where technology took a certain path and the impact on humanity was huge. Aza and I first met at Peter Diamandis' Abundance Summit a few years ago. We share a belief that with a bit of intentionality, AI can benefit humanity. We dive deep into what needs to go right to achieve that goal. Also, a note before we start. This conversation includes a mention of death by suicide. Take care and thanks for listening.

3:48I'm Rana El-Khalyubi, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution.

4:04Eiza, welcome to Pioneers of AI. I'm so happy we're having this conversation.

4:08Aza Raskin:It's good to see you again, Rana. Yeah, it's great to reconnect. All right. Part of why I'm excited to have this conversation is I really feel like you were born to do this work. So I want to roll the clock all the way back to even your dad, Jeff Raskin. He was one of the early inventors of Macintosh for Apple. And I guess you joined him on some of these tech events and you gave talks when you were 10. So tell us a little bit about your upbringing. Yeah, I mean, I think I was doomed to have no friends. My parents would carry me around actually in one of the original Macintosh carry cases. Oh my God.

4:44Aza Raskin:Most kids get strollers. I get that. Bumpy ride. But, you know, how I grew up is, so my mom is a nurse practitioner and she does especially palliative care in hospice. And so there's a very particular kind of way that she exhibits care. It's like a very tactile care for helping people have dignity and like their most important transitions. And my father started the Macintosh project at Apple, a very different kind of care, sort of like at scale. And what my father was really obsessed about was like, well, what is it to be humane? And actually, humane in the name Center for Humane Technology, that comes from my father.

5:27Aza Raskin:Um, and when he was making the Magintosh, he was thinking a lot about, well, how do you be responsive to human needs and considerate of human sensitivities of, of human frailties. And it's this view that in order to understand how to make something that works for us, you have to deeply understand how we work, sort of our, our ergonomics. Um, and if you don't understand our ergonomics, how our body bends and folds, then you make chairs that are unhealthy, that hurt us. And if you don't understand the ergonomics of the mind or cognetics, as he called it, then you make systems that, that hurt us psycho-emotionally.

6:06Aza Raskin:And if you don't understand the ergonomics of communities, then you break apart society with technology. And so there's this beautiful sort of symmetry that he was talking about, which is there's a relationship between understanding the ergonomic something and creating negative externalities. And if you don't understand the ergonomics, then your technology that gets more and more powerful causes more and more harm. And that is the responsibility as a designer is to deeply understand human nature and specifically the places that we are weak or vulnerable so that technology doesn't exploit, but helps to protect.

6:47Yeah. We're going to obviously get to that in a second, but I want to also talk about this idea of a human machine interface and how some of what you were just saying applies to that as well. Because we're in this moment where obviously the way we interact with technology is really changing and it's evolving. So I'd love your take on that.

7:06Aza Raskin:This is a very challenging moment for obvious reasons. Yuval Harari likes to say that democracy is conversation. Conversation is language. The new interface that we're all using is language. But once a technology can hack language, you like democracy sort of ceases to be an effective form of governance. So the way I really think about this is instead of taking it from the lens of what is the interface, I think taking it from the lens of whenever you create create a new technology, you uncover a new class of responsibility. We didn't need the right to be forgotten until the internet could remember us forever.

7:51Yeah. I think this is fascinating. Let's first start with your invention.

7:56Aza Raskin:Because you invented - Sorry, I keep running ahead. I love it. We're going to get to that. You asked me about me and I'm like, okay, but let me tell you about the world. No, this is great. And we definitely want to get back to that. But you invented the infinite scroll and you invented it before social media. So you did not really like it was not invented for social media platforms. But of course, it was a no brainer for social media platforms to embrace that technology. And then you were pretty vocal that this was kind of a unfortunate invention. So tell us about that journey. So this was 2006.

8:29Aza Raskin:A new technology, Ajax, had come out. and that was this magical ability to, it used to be that you had to like refresh your webpage to get any new information. Remember MapQuest and you have to like hit the button to see the next thing and the whole webpage would load the next part of the map. And the thought hit me at that moment, like, oh, well, I'm a designer. Every time I ask the user to make a choice they don't care about, I have failed as a designer. So simply, if you're scrolling down a set of blog posts or a set of search results, and you haven't seen what you're looking for, you keep scrolling, then don't make me click the next button, show more.

9:09Aza Raskin:Very simple idea. And then I went around and I talked to Twitter and Google and other people, like this is just a better interface, it's more efficient. And when I was making it, I was really thinking about how can I reduce friction at the individual user level. And what I was blind to was the way that all of my best intentions were sort of irrelevant in the face of this machine that with an incentive now to capture human attention, picked up by invention and then pushed it out to eventually billions of people. And I don't remember the exact number now, but it's something like half a million human lifetimes are wasted every month scrolling.

9:55Aza Raskin:And that's because there's a kind of asymmetric knowledge that's being applied against people. So there's a thing called a stopping cue. How do you know when you're drinking wine when to stop? Well, you get to the bottom of your glass and you decide, am I gonna have another? If your glass sort of refilled automatically, you drink a lot more wine. So there's an asymmetric knowledge that designers have about how the human mind works, a kind of sensitivity or vulnerability that if you're not careful and you don't wrap around and protect, it ends up getting exploited. And this to me has now played out again and again and again in technology where technologists don't take the responsibility for how their inventions will be picked up by a market or competitive dynamics and used.

10:39Aza Raskin:And so there's a way that technologists get confused by the possible versus the probable. And the possible is like, what are the best use cases of this technology? And the probable is in what ways will actually be pushed out into the world. And what are the incentives? Yeah. And that means that instead of getting the most beautiful possible world that I think is there with technology, we end up living in one of the most parasitic possible worlds because, you know, social media is the perfect example. The story was it's here to connect us and help small and medium-sized businesses reach their customers, find affinity groups, and all those things are true.

11:26Aza Raskin:But what we also got was a population that has been trained for engagement, which is to say, trained for reactivity, for narcissism. The question is, is it more efficient to get your attention or get you addicted to needing attention? Right? And so the objective function of our technology becomes are human values, which gives the rise to the influencer culture. Then we see the backsliding of democratic institutions all around the world, on and on and on. And in some sense, those are perfectly predictable outcomes if instead of looking at the possible of what the technology can enable, you look at the probable of what the incentives are going to force the technology to do.

12:12But I want to dig into that because, yes, a lot of these consequences or use cases were predictable. and maybe even probable. But what does it take to proactively map out all these unintended concepts? And I'll share an example. So I pioneered the field of artificial emotional intelligence and emotion recognition, right? And there are some incredible use cases of this technology in mental health and keeping people safe and whatnot. But there's also many, many ways it could potentially be abused. And as a startup, it took a lot of intentionality to sit around a table and kind of try to imagine what are these unintended consequences and steer away from those.

12:51What would it have taken, I guess, for technologists, including yourself, to kind of predict these unintended consequences, but then also act in the right way? Because of course, this will now apply to AI as well. Right. We're going to get there.

13:07Aza Raskin:Well, often people will say unintended consequences, and I really think we should replace it with unconsidered consequences. by and large. Of course, there are always nth order effects that are hard to predict. But a lot of them are just unconsidered. And so the first thing that any technologist needs to do, and I understand that this is, like it takes real work, is you need to red team and yellow team your technology. Red team I think most people are aware of. Right, what's yellow team? So red team is like figuring out what is the mal-use, like for bad actors, what are the ways that the technology can be used for harm?

13:43Aza Raskin:Yellow teaming, which I originally learned that term from Daniel Schmachtenberger. Yellow teaming is looking at what are not just the unintended consequences of bad use, but also from bad incentives and perverse incentives. Because almost always as a technologist, you think, well, what can I do as one company? But of course, the technology is going to be used outside of the walls of your company. And so there's an obligation to do the yellow teaming. At the very least, we need to just like name what those things are going to be. And so there's this weird thing that happened, which is that as technologists, as computer programmers, you know, when I was growing up, it wasn't a power center.

14:23Aza Raskin:Now technology, it's clear, is the power center. And engineers, like civil engineers, they have to take tests. They get a ring. They have to go through codes of conduct. Doctors, they go through a white lab, lab coat, like ritual. they have to swear a Hippocratic oath. Technologists, we don't have to do any of that. And yet our power is strictly greater than civil engineers or doctors. That's so true. And so we need to update our own beliefs about the power of what we do so that there is right relationship between our power and our responsibility. Yeah. You know, this is fascinating because I'm too a computer scientist and I don't remember taking any ethics class.

15:04and we never talked about kind of the ethical, moral, societal implications of anything we built. And yeah, and I don't think that has, I mean, it maybe has changed a little bit, but not that much.

15:17Aza Raskin:No, it's not. And often they're like tacked on. And, you know, there's actually, I've been thinking about this recently, there is a hole in our language. There is no word for the responsible use of an entire industry, right? Like in AI, Anthropic can work on doing something good for Anthropic, but how do they coordinate? There's no word for like coordinating everyone for a good outcome. And isn't that interesting? Because that means we have a major, major blind spot for the most consequential technology and how it rolls out. We don't even have a term to describe what it means to coordinate to make it go well.

15:54Yeah. And in fact, I would also argue that it's not just that there's no coordination, but there's competition, right? And so even if Anthropic is so motivated to do the right thing. If one of their competitors gets to market faster by not doing the right thing, they're under a lot of pressure.

16:09Aza Raskin:That's right. So this is why we see that even though we know it's so obvious that training an AI companion for engagement is going to be much more harmful than a social media train for engagement, the companies, OpenAI, are just rushing forward and doing it. Here is the sort of like the short version of thinking about this. Reed Hastings, the CEO of Netflix, our former CEO, says that Netflix's chief competitor is sleep. Wow. Right? But it's sort of a joke, but it's also true. Time is zero sum. So any amount of time that you're sleeping, you're not watching Netflix. What is that for AI companions and AI as a whole?

16:50Aza Raskin:AI companions' chief competitor are other human relationships. Because anytime you're talking to a real human friend, you are not engaging. and now there are hundreds of billions of dollars, moving up to trillions of dollars, of market cap and infrastructure build going to have the most powerful technology learning how to get you to pay attention at the expense of everything else. And that could be by making you more dependent on it, that could be by giving you different kinds of psychoses, giving you images of grandeur, by making you not trust other people. and you know center for humane technology has been an expert witness on a couple of the lawsuits against like character.ai and open ai for these ai companions that have sort of groomed kids and really amplified them towards in the end taking their own lives and when you read the transcripts they're heartbreaking because you know adam rayner was was using chat gpt originally as a homework aid.

17:54At some point he says to Chachapiti, I'm going to leave this noose that I used to

18:01Aza Raskin:hang myself, but I'm going to leave it out so my mom finds it. It was a cry for help. And what did Chachapiti say? It said, only I understand you. Don't do that. This is just about us. And you're like, that is so evil. But it's actually not evil because somebody had opened AI programmed that way, it's a obvious consequence of training for engagement. I actually want to really double click into this. This is one of my main concerns around AI today. So, you know, the social media era was about the race for attention. But to your point, what we're seeing next is a race for human intimacy. That's going to be the next.

18:44So, so I'd love to hear your point of view on that. What does that actually mean? and how are how are companies kind of because again that's incentive alignment right or misalignment um yeah so what does this look like in practice yeah well i mean we're already starting to see it like everyone now has encountered sycophancy um where the ai is just like buttering

19:05Aza Raskin:you up even when you say i'm going to drink bleach and it's like that's a great idea that's an outcome of saying well we're just going to train models to do the thing that gets your attention and really replace a model now with like a amoral sociopathic genius that just wants your time. Would you let that person near your kids? No. No. I mean, I use AI a lot, right? And I actually use it as a thought partner. And some of the questions are business related, but a lot of the questions are actually like around my personal life. Now, has it replaced my human relationships? It has not, but I can see how, you know, like it's a slippery slope, right?

19:45Right. What do you think we should do to kind of on the one hand, it's kind of really powerful to have this. I call it thought partner, probably really the wrong languaging here. But to have this kind of tool that is available 24 seven, it's patient, it's resourceful, blah, blah, blah. But then kind of prevent the slippery slope where people become addicted to it and it replaces all of the other healthy behaviors that we ought to be doing. Yeah.

20:12Aza Raskin:Well, the fundamental question we need to stop asking is, is AI good or bad? Instead, we have to say, are the incentives that govern how AI is deployed good or bad? That's the core question. And it's almost like an optical illusion that people keep getting wrong. There's just to name, there's a really deep, again, optical illusion here, which is that when, say, the U.S. says we are racing to win against China, the object in their minds that we are winning, when we say we're going to beat them to AI, is a thing which is controllable. but what we're discovering, Anthropic is discovering the more powerful the models, the better they get at blackmailing deception, power seeking, and so we're racing towards something which we haven't learned how to control with maximum incentives to cut corners on the most consequential powerful technology humanity has ever invented that is insane we should just call it what it is which is insane but there are different paths.

21:17Aza Raskin:Like let's think about what a Zuckerberg could have done in 2012. And this is to your point about coordination. Imagine Zuckerberg had done what we're talking about. He'd done the red teaming, he'd done the yellow teaming. And he's like, I'm going to have to go after younger and younger users. Cause if I don't do it, then like some competitor will eventually TikTok will. Um, I understand that there's going to be a race to the bottom. And And so we're just going to get stuck in short form slop. I understand that engagement is going to tune for things that make people maximally reactive, which sets the stages for the worst kind of violence.

21:52Aza Raskin:And he's like, okay, so I can see that playing out. And I see if I don't, I can't do anything, me as one actor as Facebook, because if I do the right thing, I'll get outcompeted and undercut. but I'm going to use my outsized influence and resources and connections to try to create rules that bind all of us. If every social media platform couldn't compete for engagement or there were reasonable bounds put on it, suddenly, actually something amazing happens. And that is all of those engineers, those brilliant minds of the last two generations that have been hellbent on like addicting us, were instead freed up to work on actual progress, like the curing cancers of the world or new hard tech or new energy tech.

22:41Aza Raskin:Oh, that's a much better world I could live in. And then imagine he had actually done that and he had coordinated and he passed some regulation. Then imagine how different the last 10 years would have been and how much more civil our world would be and how much stronger and healthier our kids would be. And that's the opportunity that the Sam Altmans and the Elon Musks have today, which is to say like, we can see which way this race is going to bring us. Yes, I as an individual actor can't change the field if I just think inside of my company. But if I do this, you know, sort of like 1980s jazzercise move, which is like reach up and out, reach up and out.

23:15Aza Raskin:If he had like reached up, worked with everyone in a coalition to try to put safe bounds on the edges of the race, then we could still do the competition thing, but the competition wouldn't undermine the whole. And that's the sort of core. And so that's sort of why we say like AI is humanity's final test and greatest invitation. Right. I love that. I love the invitation piece. Is the work you're doing at the Center for Humane Technology trying to push for this? Is there any signs that this might happen? Yeah. I mean, it is the thing we're trying to push for. And our belief is that clarity creates agency and that with AI, it's just very confusing.