Talking to Animals with A.I. ethicist Aza Raskin

0:06What if you could understand animals? I don't mean understanding what your dog is saying to you or what your cat is saying to you. I mean, what if you had a technology that you could understand any animal? Well, that's exactly what Aza Raskin is doing. He is using AI to create complex language patterns that actually allow him to decode what animals are actually saying. And the results are astonishing. And the work he's doing is going to revolutionize how we are able to understand the world that animals are living in. But that is just some of what Eze Raskin is doing. He is one of the wunderkinds of Silicon Valley, a talented engineer who also invented the infinite scroll.

0:53And as AI technology advances, he is obsessed with making sure that we advance it ethically. Something I think we are all concerned about. this is a bit of optimism.

1:09I've actually wanted to talk to you for a while since I heard you give a talk about talking to animals, using AI to talk to animals. It's really refreshing to have a conversation with somebody about AI where the technology didn't just make something go faster, but it actually made something possible that was hereto previously impossible. incredible. Eza, how are you able to talk to animals? Well, it's first important to say it's not just me. It's an entire project called the Earth Species Project with co-founder Katie Zakarian. He's our CEO. We work with something like 80 institutions and biology labs.

1:51So it's a whole field that's now working on talking to animals. And actually, even saying that is the wrong way to say it because it's not really about talking to animals. It's about learning how to listen to the natural world. And what I think most people don't know is actually how much is already known. It turns out dolphins have names that they call each other by, even in the third person. So they will talk about each other when they're not there, one of the major hallmarks of language. One of our partners recently discovered this last couple of years that orangutans have a kind of past tense, that they can talk about things that are not happening now.

2:30So we have not here and not now, two of the big hallmarks of language. Off the coast of Norway every year, there's a group of false killer whales, just sort of like an orca, and a set of dolphins. And they each speak their own different way when you listen to them. And every year they come into a super pod and they hunt. And when they do, they speak a third different thing, which is wild, right? Just think about it. Humans have been around speaking vocally, passing down culture for, I know, 100 ,000 years, 300 ,000 years tops. Whales and dolphins have been doing this for 34 million years, passing down culture for 34 million years.

3:07And of course, that which is oldest, you know, correlates with that which is wisest. If something has lasted that long, it probably encodes some deep truth, maybe something we really need to know to like solve all of our biggest problems. And we're at the cusp of being able to learn from these other species, when we think of Earth species, I really think of it as a project in using AI to make the sacred legible, make that thing which is valuable beyond worlds comprehensible to human systems. Now, I know I didn't answer the question how, but I thought I'd start there because most people like talking to animals.

3:43Is that even really a thing? I have one more example if you want to hear it. Yes, yes. This is a 1994 study. It's an unpublished study from University of Hawaii that actually one of our collaborators, Diana Reese, originally turned us on to. And here they taught dolphins two gestures. And the first gesture was, do something you've never done before. And if you think about that, that's a very complex concept to be able to share, to articulate, right? It means innovate. I understand like from going to dolphin shows as a kid, that the trainer would make a hand motion or make a whistle and the dolphin would do a flip.

4:17So what you're saying is the hand motion or the whistle told the dolphin, do something you've never done before. Correct. Yes, that's exactly right. Which if you think about it, they have to remember everything they've done before, understand the concept of negation, not one of those things, and then invent a whole cloth, something that they hadn't done before in that session. And they'll do it. It took the trainers a lot of fish and patience, I'm sure. But then they taught the dolphins a second gesture, do something together. And they say to two dolphins, do something you've never done before together.

4:48Innovate together. And the two dolphins would go down, exchange sonic information, come out and do the same thing they had never done before at the same time. I was going to say the first part, the cynical bastard inside me could say, well, that could be classic conditioning, you know, where they got the treat when something was new. The whistle means if I've seen it before, you're not going to get a treat. I get it. They had to negate something, do something new, but that they could coordinate and communicate, let's do this thing and come out together. That's amazing. Yeah. Right. Like if they're able to do that, and it's not just one dolphin pair, they replicate it over multiple dolphin pairs.

5:26Like it sort of says they're probably communicating and we should start figuring out how to disprove that. So that's a 1994 study. So at what point, I assume relatively recently, because the amazing thing about AI I find is the speed. And so I'm so curious as to when this community that you're working with, when they started to take advantage of the new technology, and I'm very curious what they discovered recently that again, was impossible without the technology. So the reason why we started Earth Species in 2017 is the AI community figured out how to do what was essentially impossible, which is if we're going to translate a language that has no Rosetta Stone or examples of translation, like how are you going to do that?

6:11The first thing to know is just how hard field research is doing. We've spent the first three years at Earth Species, we as co-founders, going out into the field to Alaska to study the humpbacks, to the Congo rainforest, to work with forest elephants, to learn from all these biologists. And it turns out it's super hard. So to give an example of how hard this is, belugas have this incredible communication. When they speak, they will include their name and their clan identity and their hellos. But it turns out Dr. Valeria Vergara, who does this research, she has tags that sit on animals that record audio, record video, record motion.

6:49She still could only use 3 % of her data because she couldn't tell who was speaking. They're overlapping. They're in these groups of 20, 30, 40, 60. They're constantly moving around. They're always communicating. They're always talking. It's family dinner. Exactly. It's family dinner all the time with like dozens of families on top of each other. It's a cafeteria. There we go. That's right. It's a cafeteria. Here we have the most vocal underwater animal with probably the largest vocabulary. And Western science has been unable to peer into 97 % of say herd data because the technical problems of listening to multiple animals, separating them out into their own individual tracks, denoising all the background, all of that is really, really, really hard.

7:31And so a lot of the original work that we've been doing are building these sort of models that let us do these foundational things. You know, the first paper that we published in Scientific Report was the first time we were able to listen to multiple animals speaking at once, separate them out into their own tracks. We published the very first like big benchmarks for animal language to know whether AI models are getting better. So I just want to like give people that background before saying, you know, we're starting to work with these incredible crows. Yeah. They have a unique culture, like they're genetically identical to all the other crows, but then this culture, they do communal child rearing.

8:05It's sort of like a commune or kibbutz. Okay. Um, and they have unique vocabulary that no other crow group has, and they will take adult outsiders in, teach them their vocabulary, and they will become part of their community in raising their children together. And we're working with these incredible researchers that have tags on the crows. So they record motion and they record audio. And we can see it. There are things that the crows will say to the young before they leave. There will be specific things they're all talking about, it appears, just before they come back to the nest. We're really starting to be able to do this translation from motion and behavior into audio and meaning, but it's too early yet to be like, and here we go.

8:52Have you been able to discern a conversation, like a warning, or can you go get the kids at three o 'clock from school or some sort of something that we would relate to? So an example of this, although this is not our work, but we're building on it, is campbell monkeys have alarm calls and a simple suffix. So hawk for them means eagle, crack means leopard, and then hawk-oo means predator that's up, crack-oo means predator that's down. So they definitely refer to things and then even have these very simple, at least so far as we've discovered, syntaxes, morphologies that programmatically change the meaning of their vocabulary.

9:31Wow. So they can, it's not just an alert to danger, but it's alert to danger with position. Yeah. A type of, a category. A category of. That's astonishing. Have we yet been able to doctor do a little this? Have we been able to take the language models and be able to type something into a computer and the computer can then speak to the animals and get the response that we hope to get? That's a good question. So the step that we need to do before we even touch something like that. And of course, it is unclear yet how much of what we can say directly in human language will be meaningful to an animal and back and forth.

10:06But one of the things we're doing, one of our research engineers, Jen Yu, he is working on a real-time audio language model that speaks in the language of a kind of bird called a zebra finch. They're a songbird. They do vocal learning. So they learn their call as young. And here's the weird thing. This is the big plot twist. Before we understand what we're saying, we will be able to communicate. That's what language models let you do. Imagine you had a superpower and your superpower was you could go to anyone in the world who speaks a language you don't understand. And you just sort of listen for a while you put your head on your side and you're like, oh, I don't know what anything means.

10:53But when this sound comes, this other sound comes in this contingency. And then you just start to babble. You're just like, I don't know what I'm saying, but I just babbling and you're babbling. But the other person's like, wow, what you're saying is so meaningful. And you're like, I don't know what's going on over there. But when I do stuff, they seem to have a meaningful interaction. We're going to be able to do that. We're doing our first experiment in a controlled setting next year with these zebra finch, that we end up in a two-way conversation before we fully know what's being said. What about these amazing stories of somebody being shipwrecked and dolphins or orcas come and save the person?

11:30We've heard these stories. Have we been able to discern that animals recognize when we're there? Are they communicating about us? That is the perennial question. I mean, they obviously can discern that we're there. And the question is, and are they communicating about us? I will just say two experiments that I love. Do you know the mirror test? Go on. This is a test for self-awareness. So in this test, researchers will paint a dot on an animal in a place they can't see. They'll then give them a mirror. And if the animal looks in the mirror and starts trying to touch the dot, it shows that the animal has connected the image in the mirror with themselves.

12:11that they say, that is me. I am aware of myself. And dolphins do this. Chimps do this. Elephants do this. Although they didn't for the longest time until researchers realized they were just using a small mirror. They needed a big mirror for elephants to see themselves. Who knew? So it shows you the anthropocentrism. And so if animals are communicating, they may well be communicating about a rich interiority, a sense of self-awareness. And like, what is more profound than that? Pretty much all of the myths start with humans being in connection with nature and able to talk with animals. And what happens?

12:47It's at the point that humans disconnect, are no longer aware really of the world they live in, that we lose our ability to talk to animals. And I think that points at like the fundamental sin of humanity is disconnection, first from ourselves, then from each other and the natural world. Another way of saying that is if you really zoom out, it's like if you look at all of humanity's biggest problems, whether it's an opioid epidemic or climate change or pollution or mental health crisis or the attention economy, these all take the same form. and it's the form of a narrow optimization at the expense of the whole, focusing in on something little and losing everything.

13:34And I think that's what those stories talk about. And I think about what Earth species is about. It's fundamentally about making the sacred legible. It's about making that which is valuable beyond words comprehensible to us and our systems so that we may reconnect and be aware of the harm that we may be causing. Where my mind went was we can dehumanize other human beings. In war, both sides will dehumanize the other because it's easier to kill that which we do not perceive like us. We do it in society all the time. We even do it in politics. our ability to reduce other human beings to a point of view or an opinion or a non-human makes it very easy to attack them and hurt them.

14:17And this is the basis of peace talks, which is you bring warring parties together and you force them to have conversations that have nothing to do with the thing that they're warring over, but to recognize that we have families and that we have children and that we have friends and that we have lives and we have ambitions and we have insecurities. And only when we humanize each other, do we find hurting each other much more difficult. This is sort of a ghost that which is an elephant sounds his trumpet. Okay, that's because it's a call to danger because he's giving us a warning. She's giving us a warning because their baby is near.

14:49The snake rattles its tail to tell us to stay away. That's as far as we got. But if we can start to understand that they're making jokes or they're telling each other about the weather or saying, hey, some good food down yonder, it anthropomorphizes it, it humanizes the animals? What if plants are communicating, you know, that we anthropomorphize and humanize the nature, then it makes it much more difficult to blindly kill the nature that we share the world with. I think that your work can only be good for the challenges we have in climate change because we see the world around us as disposable so often.

15:25Yeah, that's our hope. We are aware that climate change, species extinctions, these things are driven by complex systems, driven by geopolitics and incentives. But within that, there is a shift to human perspective, where when we see the natural world as something that we're part of and not apart from, and that actually, you know, if animals speak and hence they have consciousness, it becomes much harder to pretend like we can just blithely ignore them and kill them. I think it really changes our orientation. And I think about that album, The Songs of the Humpback Whale from the 1960s. That's Roger and Katy Payne released this album.

16:13It was the first time that Western society had heard whales sing. And it's haunting and it's beautiful. It's emotional. That album went multi-platinum. It went on Voyager 1 to represent not just humans, but all of Earth on the Golden Record. Aza, we actually have a sample from that album that was sent on the Voyager. It was Roger Payne's Song of the Humpback Whale. Here's what it sounds like.

16:57it was the reason, you know, getting played in front of the general UN assembly, it ended up beginning the movement that caused the ban of deep sea whaling. It's why we have humpbacks today. So I do think these moments of increasing our sphere of care and empathy have massive implications on policy, on our identity, eventually on geopolitics. Like it really can and be the basis for changing systems. I want to change tacks here. Your work in trying to help us define ethics, a code of ethics for the internet and for AI. Thank you. And I'm sure a Sisyphean-like task. Let's go back in your own history.

17:41You are credited with inventing infinite scroll, which creates the behavior of doom scrolling. I'm sure it was an amazing discovery when you made it. What are your feelings about your own work? Is it a bit of an Oppenheimer moment, you know, when he first sees the blast? He's like, ah, shit, you know, I am death destroyer of worlds. Well, it was, I mean, a little bit. I mean, not that I want to compare Infinite Scroll to the invention of like, you know, the atomic bomb. No, but the realization that you've maybe let something out that perhaps should have been left in the lamp. The really big lesson there for me is that thinking locally, acting locally morally can become globally immoral.

18:26Say more about that. What does that mean? So infinite scroll, the reason why I invented it was I was thinking about, and I just want to be self-aware that if I had invented it, it's not so big of invention that it wouldn't have been invented multiply elsewhere. But what does infinite scroll do? It like solves the problem of if you're scrolling and you haven't found the thing you're looking for, show me more. Every time I, as a designer, ask you, a user, to do something you don't care about or you didn't need to do, I failed. So it actually makes sense. And I invented it really before a lot of social media had gotten really going.

18:57So this was for like blog posts and search results is when I was imagining it being used. And it is, it's just objectively a better interface. Yes. As long as it's being used in your service. Yes. And what I was blind to is that making it better for you locally when it's you who are driving the thing is very different than when it gets picked up by the machine, which is the attention economy and social media and apps that need to keep you and keep your attention and keep your engagement. And suddenly it weighs, you know, a hundred thousand human lifetimes a day. And so acting locally, morally, I was doing a good thing, but blind to the incentives that would drive the technology's actual adoption is to act globally immorally.

19:47But in your defense, that's impossible for people to imagine. Most inventions, I'd even go so far as to say all inventions are solving a local problem. And it could be something completely silly. Like I can't get a good sandwich. I'm going to start a sandwich company and make my own sandwich. All inventions are solving a real problem in the moment. And very few, if any of us are able to understand what will happen months or years down the line as our invention becomes commoditized. How do we even invent a code of ethics? Everybody thinks they're on the side of good solving local problems. Nobody's aware of the global problems that they could produce.

20:27The internet being one of them, social media being another one. We are circling now the very heart of the hardest problem humanity is going to have to solve because the power of technology keeps going up and it's going up exponentially. An important question to ask is, if you're to go back a thousand years, how much damage could a single person do with an accident? And you'd be like, well, not very much. You know, they might be able to like, I don't know, drop a table on someone or like, I don't know, maybe a cannonball misappropriately fired or something. I guess they didn't even have cannons back then.

21:04But they're not very much damage. But then come to now, how much damage can a single person do? They can do a lot. If you accidentally release a virus out of a biosecurity four-level lab, you could potentially kill millions or hundreds of millions of people. So what does that mean? And the next question is, is that power to cause harm going to go up or down? It's going to go up. It's going to go up at a faster rate or a slower rate? Well, science is progressing. It's going to go up at a faster rate, especially with AI. And that means the ability for us as human beings to make these kinds of mistakes where we invent locally, but we don't project out to how that might cause harm to everyone.

21:47The cost of those mistakes is going up and up and up. And at some point, we will not be able to bear them. But all the incentive structures inside organizations or in society are encouraging us to solve local problems and be willfully blind to the global repercussions. That's exactly right. The only solution – I mean good luck passing legislation on that. Legislating ethics is asking people to sign on to a treaty that they don't follow. It seems the only solution is an authoritarian regime because if you look at nations like China where the Chinese Communist Party has taken a more heavy-handed approach to this stuff.

22:24And I'm not talking about propaganda. I'm just talking about healthy things like we have trouble getting our kids to get off social media and get off phones and get off addictive devices. In China, the device won't work for the kid after a certain amount of time. Like, it's regulated. When the global challenges show up, they just turn it off or they just restrict it. Other than that, I can't think of how the West is actually going to take care of itself. I think in the way the West is currently set up, that's probably right because we've let ourselves get debased with social media where we can't pay attention.

22:58We've become polarized, misinformed. so that makes it very, very hard to coordinate. But I just want to be clear, that's not endorsing the China model, but it's accurately naming the problem to which I think there are reasonable democratic solutions. It just doesn't look like the way our country is set up right now. Oh, Isa, this is really uncomfortable. I didn't expect that. I thought we were going to have a nice conversation about how to communicate with animals. And this got very dark very quickly. This is why I have to work on both of these projects because I need something that gives me wonder and awe, and I then need that so that I can show up.

23:38Optimism is in the name of your podcast. But actually, people ask me all the time, are you optimistic or are you pessimistic? And I hate that question because to choose whether you're going to be optimistic or pessimistic is, I think, to abdicate responsibility. It's fundamentally about asking the question, how do I see this situation clearly? And then how do I show up and get as many people as possible to show up to do something about it? I need to go back down this horrible rabbit hole. The criticism the West has of the more authoritarian control over the internet is that it is a foundation for propaganda, which is true, which is when the government can control the information and control the narrative, then there's a point of view, which is the government's point of view, which in America in particular, that like sends shivers down our spine.

24:33However, in the model that we've embraced, which is a bit of more of a free for all, propaganda is now everywhere. Like we're all propagandists who are able to get our messages out there and have a single point of view that influences people to see the world one way and not ask questions and lose their curiosity. It's not a question I think of better or worse anymore. It's kind of like what Churchill said, which is democracy is the worst form of government, but it's better than all the others. It's the same thing here, which is the lesser of the two evils because they're both pretty awful. But again, I think that's setting up a false dichotomy, a false choice.

25:05There are other ways of doing representative democracy that, for instance, Audrey Tang is pioneering in Taiwan, where you have groups of people come together in person with access to experts where they deliberate over the course of a couple of days. And it turns out people, when they do this, come up with really good solutions that cross party lines. There's an incredible film, Goodbye Elections, where they ran one of these kinds of things, an advanced democracy session in the most divided state, Michigan, on the most divided topic, COVID, in the year 2020, with a representative sample. So there were people there who were self-identified as part of conspiracy groups.

25:48Well, they didn't say conspiracy, but they would say that was, you know, I believe in QAnon. I believe in whatever. There are people from the suburbs. There were, you know, African-American folks had lost three family members. Like it was a full set of Michiganders. And at the end of this session, you know, it was done over six weeks over Zoom, lots of tiers, people coming together, and they ended up with COVID policies that had between like 80 and 90 % consensus. It turns out people are generally smart if you give them the space to process information and talk to each other, but we're not doing any of that.