Democracy in the Age of AI (4/5): How to govern AI — even if it's hard to predict | Helen Toner

0:00Hey, it's Sherrell, host of TED Tech. Election season is here and this year there's a new player on the ballot. So I put together a playlist of talks around one urgent question. What happens when AI gets a vote? You'll hear how AI could hack the system we use to govern ourselves. Who we can trust when reality itself can be faked. Why your creativity might be one of democracy's best defenses. how we can govern a technology no one can fully predict, and how AI and democracy could actually fix each other. The stakes are real and the clock is running. So I hope you get as much out of these talks as I did.

0:51If we're not technologists coding and developing AI, Should we still be allowed to weigh in on its usage, implementation, or governance? This is a question that often comes up in conversations about AI. But I find it to be a little out of touch. Because the real-world consequences of AI impact all of us, whether we're scientists, engineers, or otherwise. As this new digital species becomes more commonplace, we deserve to have a say on how exactly it shapes our lives. This is TED Tech, a podcast from the TED Audio Collective. I'm your host, Sherelle Dorsey. Today, we hear from Helen Toner. She's an esteemed AI policy research and a former board member of OpenAI.

1:37In this talk, she shares her insights on how to foster trust and transparency as AI systems evolve. She also proposes some compelling ideas on the safety measures AI companies should adopt for public protection. But before we dive into the talk, a quick break to hear from our sponsors.

2:20attention away from what really matters. That's where ServiceNow's AI specialists come in. They don't just tell you what you should do about your busy work. They actually do it. Start to finish. Cases closed. Requests handled. No extra work for you. To learn how to put AI to work for people, visit servicenow.com.

2:42This episode is brought to you by Planet Visionaries, a podcast in partnership with the Rolex Perpetual Planet Initiative. What connects a hidden laboratory in the Amazon, giant pandas in China's mountain forests, research relating to the ocean, and communities living alongside wildlife in India? That's what you'll hear on season six of Planet Visionaries, stories from the explorers, scientists, and conservation leaders whose work addresses research, biodiversity, and conservation issues in specific locations. Planet Visionaries, hosted by environmental advocate and professional climber Alex Honnold brings you real stories of discovery, determination, and research and initiatives relating to environmental change.

3:22In partnership with the Rolex Perpetual Planet Initiative, this is Planet Visionaries. Listen or watch wherever you get your podcasts. This episode is sponsored by Charles Schwab. Wouldn't it be great if life's little upgrades didn't feel like a splurge? Like getting the aisle seat with extra legroom for a one-hour flight, ordering extra guac with zero hesitation, or paying for assembly on that swing set because you'd rather spend your Saturday doing literally anything else. It's called Living Filthy Comfortable, and whatever filthy comfortable looks like to you, Schwab can help you find it with investing, trading, education, and human help all in one place.

4:05Learn more at schwab.com. Investing involves risk, including loss of principle.

4:15And now, Helen Toner, an AI policy researcher and a former board member of OpenAI, takes the TED stage. When I talk to people about artificial intelligence, something I hear a lot from non-experts is, I don't understand AI. But when I talk to experts, a funny thing happens. They say, I don't understand AI, and neither does anyone else. This is a pretty strange state of affairs. Normally, the people building a new technology understand how it works inside and out. But for AI, a technology that's radically reshaping the world around us, that's not so. Experts do know plenty about how to build and run AI systems, of course.

5:01But when it comes to how they work on the inside, there are serious limits to how much we know. And this matters because without deeply understanding AI, it's really difficult for us to know what it will be able to do next, or even what it can do now. And the fact that we have such a hard time understanding what's going on with the technology and predicting where it will go next is one of the biggest hurdles we face in figuring out how to govern AI. But AI is already all around us, so we can't just sit around and wait for things to become clearer. We have to forge some kind of path forward anyway.

5:43I've been working on these AI policy and governance issues for about eight years, first in San Francisco, now in Washington, D.C. Along the way, I've gotten an inside look at how governments are working to manage this technology, and inside the industry, I've seen a thing or two as well. So I'm going to share a couple of ideas for what our path to governing AI could look like. But first, let's talk about what actually makes AI so hard to understand and predict. One huge challenge in building artificial intelligence is that no one can agree on what it actually means to be intelligent. This is a strange place to be in when building a new tech.

6:31When the Wright brothers started experimenting with planes, they didn't know how to build one, but everyone knew what it meant to fly. With AI, on the other hand, different experts have completely different intuitions about what lies at the heart of intelligence. Is it problem solving? Is it learning and adaptation? Are emotions or having a physical body somehow involved? We genuinely don't know, but different answers lead to radically different expectations about where the technology is going and how fast it'll get there. An example of how we're confused is how we used to talk about narrow versus general AI.

7:15For a long time, we talked in terms of two buckets. A lot of people thought we should just be dividing between narrow AI, trained for one specific task, like recommending the next YouTube video, versus artificial general intelligence, or AGI, that could do everything a human could do. We thought of this distinction, narrow versus general, as a core divide between what we could build in practice and what would actually be intelligent. But then, a year or two ago, along came ChatGPT. If you think about it, you know, is it narrow AI trained for one specific task? Or is it AGI and can do everything a human can do?

8:00Clearly, the answer is neither. It's certainly general purpose. It can code, write poetry, analyze business problems, help you fix your car. but it's a far cry from being able to do everything as well as URI could do it. So it turns out this idea of generality doesn't actually seem to be the right dividing line between intelligent and not. And this kind of thing is a huge challenge for the whole field of AI right now. We don't have any agreement on what we're trying to build or on what the roadmap looks like from here. We don't even clearly understand the AI systems that we have today. Why is that?

8:39Researchers sometimes describe deep neural networks, the main kind of AI being built today, as a black box. But what they mean by that is not that it's inherently mysterious and we have no way of looking inside the box. The problem is that when we do look inside, what we find are millions, billions, or even trillions of numbers that get added and multiplied together in a particular way. And what makes it hard for experts to know what's going on is basically just there are too many numbers. And we don't yet have good ways of teasing apart what they're all doing. There's a little bit more to it than that, but not a lot.

9:18So how do we govern this technology that we struggle to understand and predict? I'm going to share two ideas, one for all of us and one for policymakers. First, don't be intimidated. either by the technology itself or by the people and companies building it. On the technology, AI can be confusing, but it's not magical. There are some parts of AI systems we do already understand well, and even the parts we don't understand won't be opaque forever. An area of research known as AI interpretability has made quite a lot of progress in the last few years in making sense of what all those billions of numbers are doing.

10:02One team of researchers, for example, found a way to identify different parts of a neural network that they could dial up or dial down to make the AI's answers happier or angrier, more honest, more Machiavellian, and so on. If we can push forward this kind of research further, then five or ten years from now, we might have a much clearer understanding of what's going on inside the so-called black box. And when it comes to those building the technology, technologists sometimes act as though if you're not elbows deep in the technical details, then you're not entitled to an opinion on what we should do with it.

10:42Expertise has its place, of course, but history shows us how important it is that the people affected by a new technology get to play a role in shaping how we use it. Like the factory workers in the 20th century who fought for factory safety, or the disability advocates who made sure the World Wide Web was accessible. You don't have to be a scientist or engineer to have a voice.

11:13Second, we need to focus on adaptability, not certainty. A lot of conversations about how to make policy for AI get bogged down in fights between, on the one side, people saying, we have to regulate AI really hard right now because it's so risky, and on the other side, people saying, but regulation will kill innovation, and those risks are made up anyway. But the way I see it, it's not just a choice between slamming on the brakes or hitting the gas. If you're driving down a road with unexpected twists and turns, then two things that will help you a lot are having a clear view out the windshield, and an excellent steering system.

11:53In AI, this means having a clear picture of where the technology is and where it's going, and having plans in place for what to do in different scenarios. Concretely, this means things like investing in our ability to measure what AI systems can do. This sounds nerdy, but it really matters. Right now, if we want to figure out whether an AI can do something concerning, like hack critical infrastructure or persuade someone to change their political beliefs, our methods of measuring that are rudimentary. We need better. We should also be requiring AI companies, especially the companies building the most advanced AI systems, to share information about what they're building, what their systems can do, and how they're managing risks.

12:43And they should have to let in external AI auditors to scrutinize their work so that the companies aren't just grading their own homework.

12:57A final example of what this can look like is setting up incident reporting mechanisms so that when things do go wrong in the real world, we have a way to collect data on what happened and how we can fix it next time, just like the data we collect on plane crashes and cyber attacks. None of these ideas are mine, and some of them are already starting to be implemented in places like Brussels, London, even Washington. But the reason I'm highlighting these ideas, measurement, disclosure, incident reporting, is that they help us navigate progress in AI by giving us a clearer view out the windshield.

13:37If AI is progressing fast in dangerous directions, these policies will help us see that. And if everything is going smoothly, they'll show us that too, and we can respond accordingly. What I want to leave you with is that it's both true that there's a ton of uncertainty and disagreement in the field of AI, and that companies are already building and deploying AI all over the place anyway, in ways that affect all of us. Left to their own devices, it looks like AI companies might go in a similar direction to social media companies, spending most of their resources on building web apps and fighting for users' attention.

14:23And by default, it looks like the enormous power of more advanced AI systems might stay concentrated in the hands of a small number of companies, even a small number of individuals. But AI's potential goes so far beyond that. AI already lets us leap over language barriers and predict protein structures. More advanced systems could unlock clean, limitless fusion energy or revolutionize how we grow food or a thousand other things. And we each have a voice in what happens. We're not just data sources. We are users. We're workers. We're citizens. So, as tempting as it might be, we can't wait for clarity or expert consensus to figure out what we want to happen with AI.

15:13AI is already happening to us. What we can do is put policies in place to give us as clear a picture as we can get of how the technology is changing. And then we can get in the arena and push for futures we actually want. Thank you.

15:35That was Helen Toner at TED 2024.

15:41I concur with Toner's viewpoint. We cannot afford to wait for a consensus on how AI should be deployed or managed. AI is already making significant societal impacts, including potentially harmful ones like deep fakes and misinformation. In a world where this technology is advancing at an unprecedented pace, the need for proactive governance is pressing, but this must be approached thoughtfully. Data and social scientist Dr. Ruman Chowdhury, who gave her talk on the TED stage in 2024, works to engage and educate people about AI. She knows what she's talking about. Dr. Chowdhury is a CEO and co-founder of Humane Intelligence, and she's also the responsible AI fellow at the Berkman Klein Center for Internet and Society at Harvard University.

16:29She believes we should not fear AI. She also thinks it's important to create guardrails for this tech. In 2023, Dr. Chowdhury testified about AI in front of the U.S. House of Representatives. She offered four solutions to advance trustworthiness in AI innovation. arguing that this was in the national best interest. First, we should support open access to AI data and models. This would enable independent audits of AI systems made by private companies, and it would create more public transparency about how AI works. Second, we provide more funding and support to the National Institute of Standards and Technology to further develop the USAI Safety Institute.

17:13This group would supplement and inform existing government efforts to regulate AI. Next, we should invest in independent ethical hacking, which involves contracting trusted groups who aren't associated with big tech companies to test out the strengths and weaknesses of their AI systems without any conflict of interest. And finally, we should develop a minimum requirement standard to evaluate whether or not certain AI systems should be adopted by the government. This requirement could consider whether an AI system is an improvement to the current system, evaluate its efficacy, and proactively combat adverse outcomes.

17:52Like Toner, Chowdhury does not ask that we stifle AI. But we do need to have checks and balances to ensure that AI is serving the best interests of humanity, not taking advantage of or taking away from it.

18:12And that's it for today. TED Tech is part of the TED Audio Collective. This episode was produced by Nita Bird-Lawrence, edited by Alejandra Salazar, and fact-checked by Julia Dickerson. Special thanks to Maria Latias, Farah DeGrange, Daniela Belarezo, and Roxanne Hailech. I'm Sherelle Dorsey. Thanks for listening in.

18:44Hi there, it's Adam Grant from Ted's Rethinking Podcast, and this episode is brought to you by ServiceNow. I get to spend my days studying how people think and what it actually takes to change our minds. It's work I find deeply meaningful. But even in meaningful work, there's still busy work. The admin, the repetitive processes, the invisible load that pulls attention away from what really matters. That's where ServiceNow's AI specialists come in. They don't just tell you what you should do about your busy work. They actually do it. Start to finish. Cases closed. Requests handled. No extra work for you.

19:17To learn how to put AI to work for people, visit servicenow.com. This is a paid partnership with Apple Card. The coolest flex at brunch these days is not the Bloody Mary with the bottomless gold flake caviar topping. It's not whipping out your wallet to pay. It's not having a wallet at all. Jack's been an Apple Card user since 2019. That's the same year he tossed his wallet in the trash. because Apple Card holders earn 2 % daily cash back on purchases made using Apple Pay. It's the only credit card designed for the iPhone. I tap to pay with Apple Card at the grocery store, at the gas station, at restaurants.

19:49And yes, I use it at my second home, the farmer's market. Jack tapping to pay with his Apple Card with Apple Pay is his favorite thing to do. And the pleasant vibration and chime sound means I just got 2 % daily cash back on my latest purchase. Apply in the wallet app on your iPhone today. Subject to credit approval, Apple Card issued by Goldman Sachs Bank USA, Salt Lake City branch. Terms and more at AppleCard.com. This is the story of the one. As a shipping manager, he's running the dock, stocking supplies, and making sure every order goes out on time. And Grainger helps him stay prepared. From shipping supplies to PPE to hand tools, he counts on Grainger to keep products moving and trucks rolling.

20:28So his operation runs smoothly, no matter the shift. Call 1-800-GRAINGER, click Grainger.com, or just stop by Grainger for the ones who get it done.