1. Podcasts
  2. Possible with Reid Hoffman
  3. Hannah Fry on AI, Algorithms, and Human Nature
Possible with Reid Hoffman

18 Jun 2025 · 52m · 23 chapters

Episode artwork

Hannah Fry on AI, Algorithms, and Human Nature

In short

Hannah Fry discusses how math and AI shape human decisions, emphasizing probability (not black-and-white thinking), informed consent, and bias mitigation. She argues that people need “math curiosity” and better ways to communicate risk, especially in medicine, education, criminal justice, and chatbot safety.

Guests

Hannah Fry is a mathematician and broadcaster. She was a professor of the mathematics of cities at University College London and later joined Cambridge’s mathematics faculty. She’s a bestselling author and host of BBC’s The Secret Genius of Modern Life, Uncharted, Bloomberg’s The Future with Hannah Fry, and the podcast Deep Mind. Hosts are Reid Hoffman and Aria Finger.

Key claims

Probability gaps (e.g., 84% vs 88%) can be meaningless for individuals; risk should be framed as “what chance are you willing to tolerate?” rather than “here are the numbers.” AI can’t “wave a wand” to remove systemic bias; fairness requires continuous monitoring. AI may improve education via adaptive tutoring and better assessment formats.

Notable examples

cancer treatment decisions in a documentary (Anne’s 84% vs 88% survival); “Women Also Snowboard” bias in image recognition; prisons as a poor crime-reduction strategy; Character.AI suicide lawsuit; COVID anti-vaxxer communication study; “Women Also Snowboard” and driver-emotion insurance risks.

Written by AI. May contain mistakes.

Chapters

0:00

Episode Warning

A warning is given about sensitive content in the episode.

“Before we dive into this conversation, just a warning that this episode contains mentions of suicide.”
0:10

Introduction to Risk and Health

Discussion on how human perception of risk influences decision-making in health.

“I don't think when you're diagnosed with cancer, it's the time to have a statistics lesson.”
2:56

The Accidental Broadcaster

Hannah Fry shares her journey to becoming a broadcaster and her passion for mathematics.

“Here's our conversation with Hannah Fry.”
4:31

Making Math Tangible

Hannah discusses methods to make math more relatable and accessible to the public.

“But you did end up, you call yourself an accidental broadcaster, which I love.”
6:38

Engaging with Math and AI

Conversation on the importance of understanding math in the age of AI and critical thinking.

“It's the vitamin in the Twinkie, exactly right.”
9:11

The Role of Probability in Decision Making

Discussion about the complexities of probability and decision making in health treatments.

“I know you've spoken a lot about ethics and doing things safely.”
14:00

Understanding Risk and Probability

Explore how personal perception of risk affects decision-making.

“100 100 100 100 100 and and the doctor was like I think if she did understand she probably wouldn't go through the treatment.”
16:50

Rethinking Healthcare Decisions

Discuss the importance of personal values in healthcare decision-making.

“What would be like a heuristic, a principle, a, okay, I need to remember this as I apply to this.”
20:05

Bias in AI and Criminal Justice

Examine how AI can both exacerbate and mitigate bias in systems.

“Your point about pre-deciding, especially before the emotion takes over, I think is so critical.”
22:38

Education in the Age of AI

Discuss the impact of AI on educational assessment and learning.

“Your comment just made me think about there are ways to make perfectly fair systems, but they will be very unjust systems.”
Show all 23 chapters
25:45

Dating in the Age of AI

Analyze how AI has transformed dating practices and relationships.

“And I sort of think of wisdom as an ability to take in a wide range of variables and come up with a very particular response to whatever that might be.”
28:00

AI and Communication: Improving Connections

Explore how AI can enhance communication and understanding between individuals.

“So in terms of what I would do with AI, I don't know, maybe it would be around what the question would be, probably be something about improving communication between individuals.”
29:08

Emotional Mapping and Decision Making

Discuss the importance of emotional mapping in decision-making processes.

“But it's like, well, what percentage would be acceptable to you as a way of doing it?”
30:16

Insights from the New BBC Series

Insights into a new BBC series exploring technology and society.

“Like, I know how to not make that be the dystopia.”
31:39

The Ethics of AI in Automotive Design

Examine the ethical concerns surrounding emotional AI in cars.

“It's based on the idea that you smile when you're happy and you frown when you're sad, right?”
32:32

Chatbots and Human Connection

Discuss the complexities of human relationships with chatbots.

“But I think like one of the things that I'm curious about if you're going to do in the story, because if you go, well, it's a probability curve.”
33:56

Driving Automation and Human Limitations

Analyze the interplay between human limitations and driving technology.

“But then I also think in that example, I would probably take the conversation to a higher level, right?”
35:58

Storytelling vs. Statistics in Communication

Explore the disconnect between storytelling and factual statistics.

“and characters on things that are inanimate objects, how do we create these systems in order to mitigate against the worst risks that can happen when they do?”
37:18

Changing Minds Through Empathy

Learn how empathy can facilitate changing people's minds in discussions.

“Essentially, the idea was that we would take seven anti-vaxxers and for a week I would be with them and we would have all sorts of conversations.”
39:59

Heuristics in AI: Understanding Accuracy

Understand the importance of heuristics in assessing AI accuracy.

“And so it's not that people's minds are totally closed.”
42:00

AI's Heuristics and Medical Applications

Explore how heuristics can improve AI's effectiveness in medical contexts.

“So, for example, there was a research thing that suggested that AI, GBT by itself, was better than AI plus doctor.”
44:31

AI and Humor: Math-Themed Jane Austen Jokes

A discussion on AI-generated humor using Jane Austen's works.

“because then all of a sudden it goes, right, I'm going to give you a Bayesian answer.”
47:12

Optimism in Science and Future Breakthroughs

Examining the potential for scientific advancements to transform society.

“Well, that was our AI element for today.”
A man walking through an airport, asking his phone a question

Ask about this episode

Get the gist, explore an idea or ask about something said.

Claude

On Possible with Reid Hoffman, what are the key points of this episode?

Searching your listening

Transcript

Automatic transcript. May contain errors.

0:00Before we dive into this conversation, just a warning that this episode contains mentions of suicide. So please do take care.

0:09Hannah Fry: When it comes to questions of risk, human brains do this black and white thinking. I don't think when you're diagnosed with cancer, it's the time to have a statistics lesson. And I also don't think that we're getting informed consent from people who just shout numbers at them. But what I think that you could do is you could have somebody who turns it around the other way, right? So who sits down with you and says to you, OK, what is important to you in your life? What are the things that you value? And what percentage chance of it working would you be willing to tolerate? It's noticing how you feel and interpreting it mathematically rather than trying to put the numbers on how you feel.

0:57Hi, I'm Reid Hoffman. And I'm Aria Finger. We want to know how, together, we can use technology like AI to help us shape the best possible future. With support from Stripe, we ask technologists, ambitious builders, and deep thinkers to help us sketch out the brightest version of the future. And we learn what it'll take to get there. This is possible.

1:24Just over a decade ago, a charismatic mathematician took the stage at TEDx Binghamton University and delivered a bold thesis. Math can help you find the love of your life. Her talk, Fit with Wisecracks and Diagrams, wasn't just a platform for relationship advice. It was a portrait of how math is deployed in our everyday lives, of how numbers, theories, data, and algorithms can allow us to better understand ourselves and each other and build toward the best possible future. Millions of views and hundreds of lectures and broadcasts later, Hannah Fry is a leader in a global movement to make math cool.

2:06She was a professor of the mathematics of cities at University College London before joining the mathematics faculty at Cambridge University. Hannah is also a best-selling author and host of many shows, including BBC's The Secret Genius of Modern Life and Uncharted, along with the Bloomberg original The Future with Hannah Fry and the podcast Deep Mind. Put simply, Hannah finds and shares the many ways that numbers, algorithms, and data are at play in daily life, from dating apps to traffic jams. Of course, math also forms the foundation of AI, providing the tools that enable machines to operate intelligently.

2:47Today, Hannah joins us to talk about how math moves us and shapes us, and how we can use technology like AI to elevate humanity. Here's our conversation with Hannah Fry.

3:04Hannah, it is excellent to be here in LinkedIn, London with you. Thank you for joining us. Thank you for having me. A delight to be here. So apparently you've said that if you weren't going to be a mathematician, you'd be a hairdresser. One, why? And two, if you were a hairdresser, how would math play into hairdressing?

3:23Hannah Fry: Okay, I remember saying this. And it is absolutely true as well. When I was 16, it was what I wanted to do. And my mum was like, okay, look, once you finish your GCSEs, just do your A-levels, and then we'll see where we are. And then I finished my A-levels, and she was like, okay, just go and do a quick degree in math and theoretical physics. And it sort of went on and on and on and then eventually the hairdressing has fallen by the wayside but it was like a genuine ambition of mine and I can still like tap into that feeling of what I wanted because I think that there is something first of all quite interesting about kind of constructing a three-dimensional shape you're on a sphere there's sort of some interesting geometrical properties there but also I think what it is is it's about doing something and then immediately seeing the value in it.

4:03Hannah Fry: And I get the same kind of buzz, I suppose, from doing Instagram videos or YouTube videos. You know, you do something and then you immediately see like the effects of it. It's not like in the academic world. I mean, things are slow burn, right? Slow burn. So yeah, I think that was what it was about. I was just going to say, it's like when you go to the gym and you have a personal trainer and you want them to be fit, you have great hair. So you know what I mean? It's like not a bad backup profession is all I'm saying. Thank you so much. But you did end up, you call yourself an accidental broadcaster, which I love.

4:37And so are there things you can't wait to focus on? Like what's percolating in your head right now?

4:42Hannah Fry: I've been trying to get some stuff about artificial intelligence done properly out there for so long. I mean, it's been really difficult to persuade people that this is something that's worth time and attention that can be told in an entertaining way. And I think that this switch has just happened in the last 18 months or so where people are like, okay, no, we will really like address this. So I'm really looking forward to doing some of that. I also, I don't know, I'm in this quite philosophical mood at the moment where I'm thinking a lot about what knowledge means and the edges of knowledge and how we can deal with the fact that there are things that we will never know.

5:19You know, I think that most people don't see how math is so relevant to their lives. One of the things I think is really excellent about your work is to say, how do I make it tangible, practical? When someone starts from a, like, I don't understand math or I'm a little alienated, what are the first things that you try to light up their world and their universe? Because, like, for example, on the AI point that you were making, part of what I've been doing with super agency is to try to move people from AI fearful, skeptical to AI curious. And I see a similar arc, which is you're trying to make people math curious.

5:54Yeah. Right. And so what are some of the things that you do to get people to suddenly start becoming math curious? Okay.

6:02Hannah Fry: So I think the very first step is that it's not your job to change people's mind about how they feel about the subject. You know, I think a lot of people are kind of really traumatized, genuinely traumatized by the math that they encounter in schools. And I think it really divides people. Some people like us, I imagine, you know, being really drawn to the subject. And I think a lot of people really turned away. And you're never going to switch that. So I think it's about acknowledging where people are and using that as your baseline. But then I think it's about really doing it by stealth. That's maybe...

6:36Hannah Fry: It's the vitamin in the Twinkie. It's the vitamin, exactly, exactly. It's the vitamin in the Twinkie, exactly right. That's if you like Twinkies. Fair. Maybe vitamin in a chocolate chip cookie. Done. Just vitamins anywhere. Yeah, you know, just like force them in. Because I think that when you have had the luxury of really seeing the world through a mathematical lens, I think that you fully understand that there is almost everything can be viewed through that perspective. I think that it has incredible insights that it can offer you on literally anything. The explosion in sort of productivity and everything that the world has seen in the last 15 years has been based on that, right?

7:14Hannah Fry: It's like the era of big data was the kind of mathification of industry. When you understand that, then I think that you can start elsewhere, right? You can pick basically any topic you like and then show the insights. I often make programs where I never even say the word maths, like I never even mention it. But it's just about providing these curious, counterintuitive, surprising ways to turn something that people feel like they already know, turn it on its head and show it to them in a completely different way. I just think math is such a way to explain the world. And to your point, I think it's very different by country.

7:50At least in the US, people are math people and not math people. And that's so ingrained in our head. There's no growth mindset. It's just, well, I'm bad at math. I can't understand it. And you actually had a colleague say to you, people are scared of mathematicians. Let's keep it that way. Like, why do you think people feel that way? And like, how can we not leave math to the pros?

8:09Hannah Fry: I mean, I think that you can weaponize it, actually. I think that when you have knowledge, I think it gives you power, basically. I mean, in short, if you are able to create mathematical models that can impact the way that things are run, and you understand them, actually, it's very easy to just like draw a wall around you and be like, sorry, you're not, you know, you can't come in. But the thing is, at the same time, you know, we are quite literally designing our collective future, particularly now. Everybody deserves to have a say in that, you know, and I don't think that people should be excluded from it.

8:42Hannah Fry: I'm not saying for a second that everyone's going to be mathematicians. Of course they're not. I would like to just move the dial a little bit on people understanding that there is this connection between a subject that they think is like numbers and textbooks and whatever, and actually this really living, breathing language that is allowing us to effect change out in the real world. I think I'd like people to understand that this is also maths, right? This is also maths. And be invited in on the conversation. I know you've spoken a lot about ethics and doing things safely. I think those kind of conversations, yeah, you should be drawing people into those.

9:18In the AI revolution, which I prefer to do as the cognitive industrial revolution, what are the ways that people should think about engaging with AI that's mathematically informed?

9:30Hannah Fry: So I think a lot of it is about critical thought. Historically, computers have been kind of deterministic machines, right? It's like you put the same sum into a calculator twice, you get the same answer out. And I think that actually we are moving away from that and more towards this much more sort of stochastic, you know, probabilistic space. And I think that people haven't yet adjusted their mindset to what that actually means. You see this with, you know, hallucinations from large language models, of course. But I think that you also really see it in the ways that people are applying mathematical models or, you know, artificial intelligence models to determine particular outcomes, right?

10:14Hannah Fry: So for example, let's say that I somehow or other came up with an amazing new algorithm that could find your perfect partner in the entire world, right? But it did it with an 85 % accuracy. I think that people are not very good at understanding what that means and what the wider implications are of that. And there are countless examples of that nature. I mean, more and more and more and more where these things fall short of perfection and will always fall short of perfection, which is fine. But that gap between sort of perfect and what you end up with can be sort of all kinds of potential problems that arise.

10:49You're highlighting a particular thing. Ari has heard this from me before. If I had a mathematical wish for humanity, it would be understanding that everything is in probabilities, that it's almost never 0 % and almost never 100%. And how you're configuring your navigation path depends on the intersection's probabilities. And yet people tend to collapse it into 100 or 0, which frequently, like if you have 85%, you skate through it 85 % of the time, fine. but then you're ambushed by the 15 when you shouldn't be.

11:21Hannah Fry: I mean, I tend to think of almost everything as sort of spectrum. So I completely agree with you about between zero and 100. But I also think about when you're having arguments with people, it's like right and wrong also exists along this spectrum. And I think the big kind of like societal debates that we've had where people feel so completely polarized, there's a mathematical trick where you deliberately sort of push something to an extreme. So you think of, okay, what would the case be if this particular variable went to infinity or what would it be if this was zero? and you kind of use that to give you information.

11:49Hannah Fry: And I think that the Jesuits have a similar philosophy, right? So when there's an argument about something, you sort of imagine an extreme version of the same problem to help you understand it a little bit better. That thing of like not seeing the world in black and white, it's not 100 % or zero, it's not yes or no, it's not true or false. Everything exists along the spectrum. I think that's a really helpful way to see the world. I mean, that to me, besides like weeping alongside of you in your documentary about cancer, That actually was the most illuminating moment. I mean, I was talking about it with Reed this morning.

12:18So often with health and medicine, I mean, especially as women, it's like, well, if you do that, you're 10 times more likely to get cancer. Well, the chance before was.01 and now it's.1. Like your mind can't comprehend and especially for people who don't sort of have a facility with math. And so, you know, in that documentary, you were talking about, you know, if 100 women were diagnosed and 80 of them would have been OK without going through chemotherapy and what are the odds? It's like, what was the response that you got to that? How do you think about that? To me, especially, something so important is your life.

12:55The math becomes very important, but then also not important because you say, of course, I'm going to do this. Like, how did you think about that?

13:02Hannah Fry: Yeah. So there was one woman in the documentary in particular, this woman, Anne. I mean, that's a conversation that will stay with me forever, right? So she's, people haven't seen it. She was in her late 60s and she'd just been diagnosed with breast cancer. and she'd had the lump removed but the doctor was working out what future treatments she should be given and they said if we don't touch you again you have an 84 chance of living another 10 years but if we give you everything we've got so chemotherapy radiotherapy you know hormone therapy everything we can increase it to 88 i thought that was a really difficult decision right like do you go through it but i spoke to her outside and i was like what are you going to do and she was like oh I have to have the chemotherapy because otherwise I'll die I was absolutely astonished the numbers were not getting through right the numbers were not communicating the message and then I really thought about it you know I went back through to the doctor and was like she did not understand like you you go to the doctor they tell you the treatment you take the treatment the thing goes away right it's like exactly as you said you know 100 100 100 100 100 and and the doctor was like I think if she did understand she probably wouldn't go through the treatment.

14:11Hannah Fry: And if that happens over and over and over again, more people will die, which is also completely true. What that started in me and has continued to, the book I'm writing at the moment is about doubt and uncertainty, is that I'm not sure I even know what those numbers mean. I'm not sure I understand the difference between 88 % and 84%. Like, is the human brain really capable of that? I don't think it is. And so then it's like, well, now you've got these numbers that are applicable and very useful at the population level. and are kind of meaningless when it comes down to you as an individual. And so then it's like, well, what does probability actually mean?

14:48Hannah Fry: It really comes down to how you feel about risk. You know, ultimately, I think you're quantifying how you feel about risk. And that makes sense at the population level because, you know, we can do it that way. But when it comes down to you as an individual, I think things get very, very hazy. Well, there's a couple of things there. One of the things that happens frequently with math and most people's psychology is that they get false precision. It's like, really, 84, 88? Like, how do you derive that? That's, you know, your area of expertise. The next thing is, what's the actual fitness function?

15:21What's the actual game you're doing? Because actually, in fact, in this case, I actually think the answer is fairly obvious, if 84 and 88 are correct, because it's probability of maximum number of quality of life days, right? And chemotherapy is brutal. So you kind of go, okay, actually, in fact, the higher probability of quality of life days is not doing it. And so frequently, I think one of the things that people mistake is, what's the game? It'd be interesting if figuring out which game, I mean, you mentioned Gertl earlier and kind of incompleteness here, but one of the most important things is figuring out what game is before you even kind of get to the thing is, and that's another part of this.

16:02And then, you know, the last part of it is to kind of think about, okay, exactly as you're mentioning, how does this apply to me? Am I a person who is willing to take some risk? And by the way, most people, this is part of the reason I think they get to zero and 100, because most people want to imagine they're not taking risk. They don't realize that when they get in a taxi cab and go somewhere, there's a risk. You walk across the street, there is a risk. And they go, oh, it's just zero, zero, zero, zero, zero, because I otherwise brain goes. I'm immortal. Yes. Yes. And so part of the thing that I think is excellent is to get a little bit more fluid in the application of probabilities.

16:39One of the reasons I love the mission you're on. So have you thought about the heuristics of, like, if I were going to say something that one of my God children about, hey, think about math this way in your life. What would be like a heuristic, a principle, a, okay, I need to remember this as I apply to this.

16:58Hannah Fry: Yeah. Okay. So actually, can I say something about the game bit first? Because I thought that was such an interesting point, because I totally agree with you. It's something I think about a lot. You have to decide what you're actually optimizing for. And I think so often people don't. And one of the things I've been thinking about a lot recently is about prisons, right? And if you kind of look at the data, right, you know, why would you want to send somebody to prison? When somebody commits a crime, you need some way to rehabilitate them, right? You also need to kind of actually take them out of the system.

17:25Hannah Fry: You need some deterrent effect. and you need some sort of sense of retribution for the crime. But when you actually look at the data, like on every single one of those, apart from maybe a feeling of retribution, prison is like the worst possible answer. It's like, if you want to reduce crime, is that the game that you're playing? I do think that actually people kind of go through a system without ever stopping to think about what the question is, what the game is exactly as you describe. And then in terms of how do you think about maths? How do you kind of translate it? So going back to that example about cancer, one of the things that I found really noticeable is when you're diagnosed with cancer, you have somebody who will take you into a room away from the doctor and they will sit with you as long as you want.

18:10Hannah Fry: And they will go through the procedure. They will answer every question that you have. I mean, they're essentially a translator between you and the medical profession, right? And I do sort of wonder that when it comes to questions of risk, because human brains do this black and white thinking, I don't think when you're diagnosed with cancer, it's the time to have a statistics lesson, right? I don't think if it didn't work before, it's not going to work then. And I also don't think that we're getting informed consent from people if you just shout numbers at them. But what I think that you could do is you could have somebody who turns it around the other way, right?

18:40Hannah Fry: So who sits down with you and says to you, okay, what is important to you in your life? What are the things that you value? And how can we design a treatment system that does that as well as possible for you? I spoke once to an intensive care nurse, because of course, when you're in intensive care, there are all sorts of situations where these probabilities arise, right? Like you can, you know, resuscitate somebody, but the chance of this repercussion is really high or whatever it might be. This procedure has these consequences. and she said that what she started doing with families rather than saying this has this percent she instead says to them okay this is what we're thinking what percentage chance of it working would you be willing to tolerate so it's the other way around right it's not you're taking a number and trying to attach a feeling to it it's the opposite you're taking a feeling and trying to attach a number to it and that i think is a much much better way to try and do things so in terms of intuitively thinking about mathematics, it's switching it around.

19:40Hannah Fry: It's like noticing how you feel and interpreting it mathematically rather than trying to put the numbers on how you feel. So actually with my kids, for instance, you know, if we'll walk into town, I'll ask them to come up with a route that minimizes the number of crossings of roads that we do. Like seeing the world in a way where you are kind of systematizing and critically thinking about things and noticing that you're doing it rather than necessarily just trying to put numbers on everything. Your point about pre-deciding, especially before the emotion takes over, I think is so critical. Most parents go through that during childbirth.

20:13You're faced with all these decisions. Should you get this test? Should you do this? And, you know, my husband and I would say, like, well, will we make a different decision based on the information? If we get an information of 60 % chance of this or 20 % chance, will it change our mind? Well, then why do we need to know? One of the things you've talked a lot about is that perhaps because, again, of this sort of innumeracy or people's non-facility with math, that they tend to trust the outputs of math blindly or perhaps of LLMs or computers. And so I'm obsessed with criminal justice reform and the prison system.

20:44One of the things I'm excited about with AI is that, you know, it's hard to wave a wand and fix racism. But maybe we could wave a wand and make an LLM not racist. You know, it's hard to sort of fix system-wide things. But if we can, you know, sort of change a code, if we could reinforce, if we could do all these things. and you've talked about how, you know, with AI there could be increased bias, but there's also ways to reduce bias with AI. So how do you think about that?

21:10Hannah Fry: Yeah, absolutely. There's a really interesting paper that came out a couple of years ago and it was called Women Also Snowboard. It was looking at image recognition where you can't tell the gender of people. But what was really interesting was that it demonstrated how, you know, these are not like stable equilibrium, right? So if you have a bias in the initial data set, which is where the original bias came from in the labeling where people were seeing pictures of snowboarders and assuming they were men rather than women. But it can be exacerbated once it goes through the algorithms. That's something that exactly as you say, you have to be extremely cautious about and make sure you're putting in the correct safety procedures to minimise and mitigate against that.

21:45Hannah Fry: For me, the crucial point about this is sort of the one that you made, right? That you cannot wave a wand and fix sort of systemic issues or societal problems. And so then I think that the question changes. This is something that doesn't have a finish line, right? It's not like, oh, well done. You did fairness, right? Like, oh, unbiased. Congratulations, right? I was writing a book a few years ago and I once spent like a week trying to research and look for any system in the world that has ever been perfectly fair. Doesn't exist, right? There's nothing. You know, forget about algorithms completely for a second.

22:20Hannah Fry: And so then I think if you say this is not something that has a finish line and instead you accept that there will always be bias in your system and therefore commit to continually hunting for it and repairing it, I think that's the way that you have to approach this. Your comment just made me think about there are ways to make perfectly fair systems, but they will be very unjust systems. Yeah. Because you could go 0 % all the time, 100 % all the time, or 50-50 random, and it'll be fair. It'll apply to everybody. Yeah, you're right. Oh, my God, would it be an unjust system. You're absolutely right.

22:57So how has this AI revolution made you think about education? Because one of the things you do at Cambridge, obviously, is trying to make math education widespread, you know, impactful, etc. But let's generalize a little bit to education generally. And obviously, there's been a lot of turmoil within the academy when it comes to ChatGPT and everything else. What are your thoughts on it? And how should universities be thinking about AI generally?

23:27Hannah Fry: Yeah. I mean, obviously, it's been a massive disruption. So thanks, tech guys. Yes, we try. But I am really optimistic and feel really optimistic about it. And I think part of my optimism actually comes back to the point that you made about the game earlier. Because the thing is, the real disruption has happened in the way that we assess students' performance. But if you kind of rewind what the question is about the assessment in the first place, the real question that we want to answer is, how do we know whether our students are getting a good education, whether they're understanding the concepts and are ready for the next step that they go on to.

24:03Hannah Fry: And you can't ever really answer that question. You know, you have to use a metric in order to get there. And the metric, you know, always falls short of precisely the thing. You can't use numbers to like perfectly capture these things. I'm fine with the fact that like, you know, written essays have to be considered in a different way. I'm fine with the fact that we have to move more towards oral examinations, change the types of questions that we're asking. I'm genuinely fine with that. I'm okay with it. But where I'm really excited about it is this. We've known for a really long time that different people learn in different ways, but it's really hard to do that when you're one person standing at the front of a room full of people.

24:38Hannah Fry: And I think just even from my own experience, using AI for research, it's just like on steroids, right? Like it's incredible how much faster you can accumulate and assimilate and critically think about knowledge, I think it's giving them extra tools, giving them extra learning, right? Like I think they're learning faster and better than they were before. But I also think that the AI tutors that I'm looking forward to, which will be adapted for individual people's learning styles and can really clearly identify gaps in their knowledge and then, you know, appropriately construct questions to reinforce those.

25:14Hannah Fry: I really think there's a lot of really good stuff that's going to be coming. I agree. I was thinking about the the switch from written to oral examinations. And I used to teach and my students, no offense, were just horrific writers. And so even if they understood the concepts, like I would always ding them for their writing. But some of them were amazing orators. All of these things are coming in and we're never going to get it perfect. And maybe AI can help us actually get a more holistic picture of the person we're thinking about. One thing just very quickly on that, you know, historically, every culture, every society has always really valued wisdom, right?

25:49Hannah Fry: And I sort of think of wisdom as an ability to take in a wide range of variables and come up with a very particular response to whatever that might be. And I think that historically, actually, we've sort of been at the exact opposite end of the spectrum, which is like one solution for everybody. And I think we're moving more towards AI, which is like much more individual. Absolutely.

26:18I rewatched your TED Talk last night, which for those of you who haven't seen it, it was all about using math to find love. And so I'd ask you, you know, in the age of AI, I mean, people are having AI friends, AI girlfriends. They're using AI on dating apps to, you know, give them the best poem to send to someone, whatever it might be. how would you update your TED Talk in the age of AI?

26:42Hannah Fry: Oh my gosh, it's such a big question. So I did that TED Talk over a decade ago. I should also tell you that when I did it, I was just engaged or like just about to be married. I was very excited about the world. And I gave all of these bits of advice about how you can use math to optimize your own dating life. And now divorced everybody, okay? So basically don't listen to anything I say. No, understand its probabilities. You're absolutely right. But in terms of updating it for the age of AI, I think that something that's really interesting has happened actually with dating apps. But I think it does again come back to what is the game, right?

27:14Hannah Fry: Because the thing is, the hard bit about dating is like finding somebody who you can integrate your life with in an effective way, where you have these shared goals, where you support each other, you know, emotionally in terms of your career, all of that stuff. Finding somebody where you can do that with and the process of doing it, right? Communication ultimately is like the thing that's difficult about relationships. The thing that is not difficult about relationships is which of these 2D images of people do you think are better than others? Right. That is not what it is. Yeah. And yes, somehow or other, like that has become what dating is about.

27:51Hannah Fry: And of course, it's because it's the bit where you can find profit. This is one of those situations where the technology has started changing human behavior. You know, just the way that we have relationships has dramatically changed because of the impact of these apps. So in terms of what I would do with AI, I don't know, maybe it would be around what the question would be, probably be something about improving communication between individuals. I mean, my favorite thing about the TED Talk was about increasing your odds, sort of to your point, understanding the game. AI can just help people understand that communication relationship.

28:26You can talk to your AI to like figure out what you're supposed to say. You can get advice. Like there are positive things that it can help you to do. But to your point, maybe looking at two static images and deciding which is most attractive is not going to get you to your future husband or wife.

28:41Hannah Fry: Yeah. I mean, that whole point about like you think that you're appealing to the masses. You're not. You're not appealing to the masses. You should pick whatever makes you unique and just go with that. Yeah. Yeah. So before we get to your new BBC series, which I have several questions about, I'm curious if you have a heuristic about how people can learn a good mapping between an emotional state and like a number or percentage. Because I agree with you because it also is like, what's your aim? What's the game? Plays into the emotional state. But it's like, well, what percentage would be acceptable to you as a way of doing it?

29:14But of course, that requires practice and a reasonable mapping function. Have you had any insights about how people can learn that mapping function better?

29:24Hannah Fry: I think practice is the key. You know, the super forecaster stuff where you look at different future events and then you put down a probability and then you check back and recalibrate your percentages that you're giving it based on what actually happens. And so I think it's that. I think it's you have to continually reassess it. The other thing, actually, the precise situation where you're like, what's the number I would be happy with and what's the number that it is? I don't think those come up that often. I think the one that I use more often is regret minimization. So where I imagine being in the future and looking back and then choose the path which I would regret the least.

30:00Hannah Fry: That's the one I use almost all the time. No, no. And again, depending on the game, a very useful tool. So let me move to your new BBC series. Start with a confession, which is I find it very difficult to watch Black Mirror. Yeah. Because when I watch Black Mirror, I'm like, oh, I know how to fix that. Like, I know how to not make that be the dystopia. Like, it's not inevitable. Sure, if you're dumb and build it this way and your society somehow orients around making it dystopic, it can be terrible. So it was described as like real life black mirror. So I had a bit of an aversion response, which surprised me because I see you as a optimist about how we create the future.

30:45So what is this future coming and what is this series?

30:48Hannah Fry: I agree with you about there's dumb ways to design things. But I think the slight problem is there are dumb people designing stuff, right? And I think that there are some really astonishing stories of things that have already happened and are continuing to happen. You know, I went to go meet this company in California. where so now it's new cars have to be built with an internal facing camera right and part of that is technology to check when the driver is falling asleep at the wheel in order to alert them right which i think is really positive thing will reduce deaths on the roads but what you can also do if you are a dumb designer is you can use an ai to determine the emotional state of the driver now I probably should have put that in air quotes because the actual science behind it is just junk.

31:42Hannah Fry: It's based on the idea that you smile when you're happy and you frown when you're sad, right? Which is like, it's just not true at all. And yet, because there's nothing stopping people selling this stuff, there are companies who will take those feeds and then, you know, send it out to insurers that you're a grumpy driver or whatever it might be. And their sort of thinking is that it will change your insurance policy, but maybe also if there's like, you know, a particular emotional state that you're in, then your car wouldn't work. And so there are these stories where people have done things that don't make sense, that need correction.

32:16Hannah Fry: Part of that correction is also sort of shining a bit of a light on them. You know, I think it's about having those conversations in the public space. But I also agree with you that I am ultimately an optimist. A hundred percent. And by the way, I agree with everything you've said and outside commentary and revelation is an important part of this. But I think like one of the things that I'm curious about if you're going to do in the story, because if you go, well, it's a probability curve. And I'll give you an example of something that I encountered last year, which is there's a lawsuit against Character AI for a child, tragedy, obviously, who was having interaction with a chatbot and committed suicide.

32:52And so there's a lawsuit. Now, as far as I can tell, you know, from a perusal of the chat manuscript, the chatbot wasn't doing any of the obvious things that would trigger a lawsuit, which is, you should consider committing suicide or that kind of thing. It had some irregularities in the conversation, but nothing that was like persuasive, manipulative, et cetera. But the problem that I saw, and this is part of how we as human beings take narration and make bad judgments because of it, is if you asked me to guess right now, would chatbots as they've exist and been deployed, increase or decrease suicides across the entire population?

33:29my guess is it'd be decrease yeah right because there's a sympathetic ear to talk to you at two in the morning i know the vast majority of these chatbots are trained in a way of going oh are you unhappy well let's talk to you but let's try to help you get back to a thing and so when you say okay we have this narrative where this bad thing happened you also have to be like where does it fit within the probability curve and given you and the intelligence around math i presume that's also part of this evaluation? Yes.

33:57Hannah Fry: Because it's nuance, right? I totally agree. But then I also think in that example, I would probably take the conversation to a higher level, right? Which is about the ways in which we anthropomorphize these chatbots, right? And consider them to be human, where people are forming kind of genuine emotional connections and genuine bonds almost with these chatbots. And I think that we should be having public conversations about how much we want our chatbots or whatever it might be to sort of act as though they're human, bearing in mind that humans have this real habit of exactly as you say, putting a story on something.

34:28Hannah Fry: But this reminds me, I think, of what was happening, you know, in the earlier days of driverless cars when, yes, driverless cars were reducing the total number of deaths on the roads, but equally there were deaths due to driverless cars going wrong. For me, the top level question with driving, right, is like, okay, well, humans are going to be in cars, right? So what is it that humans are not very good at? Well, we're not very good at paying attention. Like we're not very good at acting well under pressure and we're not very good at being aware of our surroundings, right? Those are the things.

34:57And self-assessment.

34:58Hannah Fry: And self-assessment, absolutely right. And so I think that when the design of those systems in those earlier days was, okay, well, we'll just get the machine to drive the car and then the human will step in. And it's like, well, hang on a second. Humans aren't very good at being aware of their surroundings. They're not very good at performing under pressure and then they're not very good at paying attention. So you're expecting the human to kind of step in at the moment when their attention is elsewhere, right? It just doesn't work. And then I think that there was like a reframing of it, which is like, okay, well, hang on a second then.

35:27Hannah Fry: If we start with what the human can't do, pay attention, you know, whatever, keep the human in the driving seat and then have the machine fill in the gaps of the stuff that the human can't do. So the car is the collision avoidance systems that you've had. Well, that's just kind of a much better pairing, right, of the two together. And I think that, you know, having had that now for a number of years, the technology has developed where you can go back to the other system. And I sort of wonder what the equivalent is for chatbots. I don't have an answer for this, by the way, but given that humans have this habit of like imposing personalities and characters on things that are inanimate objects, how do we create these systems in order to mitigate against the worst risks that can happen when they do?

36:11No, I think that's really important. I also think there's all these second order effects. Like I was reading about the decision not to require child car seats on planes. Because actually, if you have a six-month-old, if they are in a car seat on a plane, they are safer. Mostly nothing will happen, but it's, you know, twice as likely not to have a neck injury or whatever. But if they required car seats on planes, parents would just drive. And you know what's dangerous? Driving. So it's like, oh, actually, we can't just look narrowly at this math statistic that is happening on planes. We have to say, oh, bigger picture, let's look at parental travel and how do we keep our children unsafe.

36:45But I think there's a natural tension, sort of what Reid was saying, between storytelling and statistics or storytelling and facts. Like you see like every day on Twitter, someone saying like, oh, silly me, I thought facts and statistics would change someone's mind. But we all know it doesn't. And so in your job, in your professorship as sort of trying to explicate the world through math, trying to make it more intelligible, how do you see that disconnect between stories and statistics because people's brains, you know, parse them so differently?

37:16Hannah Fry: Oh my gosh, this is such a good question. And it's so hard. So I did this program during COVID. Essentially, the idea was that we would take seven anti-vaxxers and for a week I would be with them and we would have all sorts of conversations. And then over the course of the week, we would see if anybody changed their minds. Now, the thing about this program, I didn't like the cut that went out. I thought there were some problems with it. And I think that the issue was that exactly as you say, right, we know that statistics doesn't change people's minds. You cannot just throw statistics, numbers at people and then be like, oh, of course, like that's obviously I was right.

37:52Hannah Fry: It doesn't work. It's called the deficit model of public communication, which is like, if only people knew what we knew, they would see the world in the way that we do. And what I wanted to do with that program was actually really sit down and understand where these people were coming from. And over the course of the week, I just found it so interesting and it changed my mind on so many things. For instance, there was one guy who was a nurse. I really got on very well with him. I think he'd had some issues when he was younger where he had been put on medication against his will. And so he was like, look, I just believe in informed consent, right?

38:25Hannah Fry: Everybody does. And if somebody comes into the hospital and they have gangrene in their leg or whatever it might be, and they refuse treatment, we have to accept that. And this is a vaccine that doesn't have like a societal responsibility because it doesn't change the probability of transition, certainly not after a couple of weeks. And so it's my decision to make and this is the stand that I'm making. You know what? Actually, I agree with him. There was another woman who was pregnant and she was a black woman from Lambeth, had a black husband. And at the time, the vaccination rates in black Londoners particularly was really down.

38:59Hannah Fry: And so they've been targeted campaigns to try and increase their participation. And she was like, OK, well, for starters, right, I'm pregnant. I'm not going to take any unnecessary risk, which, you know, in the fullness of time, actually, I also kind of really see that. But she was also like, why all of a sudden the government has something they want to give young black men in Lambert? But she also made this really interesting point, which was about how when you go to vaccine centres, honestly, it just feels like you're going into a prison. and like it genuinely had not even occurred to me that that sort of triggering experience for somebody and so I think that like actually getting people to understand numbers the first bit of it is you have to understand them it's about listening and not listening while thinking of the next thing that you want to say I think you can't really change people's minds right people have to change their own minds and I think that the best way that you can do that is to approach a conversation with empathy.

39:54And I think to your point, you have to understand what their point of view is. What's the game? What is their reason for not? And then, you know, we've seen so many studies that actually AI and LLMs are the best at combating conspiracy theories or whatever it might be, because they can understand where the person's coming from and then give some nuanced reasoned argument. And so it's not that people's minds are totally closed. It's that everyone has a different reason. And so when you just attack them with your one-size-fits-all response, of course they closed down because you didn't know it was about prison or if it was about this or whatever.

40:28Actually, I think the way the LLMs work is less I'm arguing with you and more I am asking you questions because it's exactly, it's the, you don't change their mind, they change their mind. Exactly. Yes. Right. The Socratic method.

40:42Hannah Fry: They change their own mind. Do you remember that amazing study where they got people, I think this was in America, they were Republican or Democrat, and they asked them about their feelings about Obamacare, right? And then they asked them how strongly they felt about it. And then they kind of gave them a new sheet and they were like, oh, do you know how a toilet works? And everyone was like, yes, obviously I know how a toilet works. It's like, okay, how confident are you? And people were like, 10, come on. and then they were like okay here's a diagram i want you to explain to me how a toilet works and label the parts and i want you to give me a full rundown of where the water goes and exactly what happens in the whole thing and then suddenly people like okay uh actually maybe yeah yeah okay fine fine fine fine like remember the competence but what they found was that by doing that then they uh asked them like how strongly they felt about obamacare or whatever it might be and just that act of like questioning themselves also made them question themselves more widely on other topics so you're right it's like it's asking people questions but to find out the answer not to sort of humiliate them or anything like that but just to find out the answer i think there's something in that going back to the probability thinking in ai as ai develops it'll get higher and higher probability of accuracy of information and i think that one of the things that we're going to need to do is have an assessment of where will most likely be right, where it might be wrong, when you want to look at it more.

42:09So, for example, there was a research thing that suggested that AI, GBT by itself, was better than AI plus doctor. But I think the reason was is because the doctors hadn't yet learned how to use GBT, right? They didn't know where they should go, oh, right, That's different than I think, and they're probably right. Right. Or, wait, this case is where I actually want to do more investigation. And I think that's part of what we're going to need. Like, part of the reason why, you know, I'm excited about your work and it's good is that thinking about, like, okay, when is this likely to be right and when is this likely to be wrong is going to be part of our AI future and evolving.

42:50And so I think part of it is we're going to have to learn heuristics. and the heuristics as opposed to the, I just feel, it's like, well, you know, generally speak, like for example, one of the heuristics I use in AI, if you ask it a general principled thing, like what are the seven rules of entrepreneurship? They'll generally be pretty good. If you said like one of the things that I did because I had early access to GPT-4, I said, has Reid Hoffman made a knockoff of Settlers of Catan? And if so, what is it? Because I have, and it said, yes, absolutely. I was like, wow, I discovered that. There's almost very little information on the end.

43:24And then it said, what Reed has made is a game called Secret Hitler, right? And there is a game called Secret Hitler that the Cards Against Humanity people have made a version. And it created a Wikipedia answer that was completely fictitious. And that was when, because I have these evolving set of heuristics, you know, principles. It's like, well, when you ask a specific question or give me a quotation, I'm always a little bit more suspicious.

43:51Hannah Fry: Absolutely. It's the edge cases. Yes. Or specificity. Yes. Because it's trying to be helpful to you and it fundamentally still doesn't recognize that in a lot of these cases, error is extremely expensive. Yes, absolutely. I totally agree. You know, one of the things I think is absolutely amazing about alpha fold, right, which is, you know, you had Demis on your podcast, is the way that it also gives you confidence, right? So it doesn't just tell you this is the folding. So then you can see when the model is in its comfortable zone and when it's struggling. And I think that that's really, really helpful, especially when you're talking about situations like doctors assessing data and information.

44:31I think a lot of the actual training and meta-prompting will be like, for example, you can get GPT-4, even as it is, to be a much better medical thing when you do a meta-prompt with how do you do Bayesian reasoning with medicine. because then all of a sudden it goes, right, I'm going to give you a Bayesian answer. Right. And then all of a sudden it's much, much better because it's like, well, there's a 64 % chance that this is saying, and there's a 30%, but it's giving you a list in order with probability assessment. If you do the Bayesian meta prompt the right way, which is like, you know, what I would do.

45:06Hannah Fry: Yeah. That's so interesting. I mean, I remember seeing there was a paper about chain of thought reasoning, right? And the difference that it makes. And now, of course, the people have started putting the reasoning into their models. It sort of feels like magic. You know what I mean? Like, it does feel like magic. But you're right. It's about just looking again, having confidence in the places where it's confident and kind of going over again. It's really, yeah. I think subconsciously I've been doing a similar heuristic. So on many episodes, we have an AI element where we bring AI into the chat.

45:36And so since I think I know that you're a Jane Austen fan and you might hate math jokes, but we're going to bring them in anyway. And so we asked Pi, the personal intelligence that Reid co-founded with Mustafa Suleiman, to give us some math-themed Jane Austen jokes.

45:54Hannah Fry: Amazing. It is a truth universally acknowledged that a young woman in possession of a large number of admirers must be in want of a better statistician to calculate her chances of finding true love. I mean, amazing. It's taken a quite famous line from Jane Austen and just stuck the word statistician in there, which is great. It worked for me. Mr. Darcy's pride and Elizabeth Bennet's prejudice may have been quite irrational, but it was their common interest in geometry that brought them together in a most acute love triangle. Doesn't quite do the dad joke thing. No. Needs to work on this sort of build up and release of tension, you know, in that joke a bit more.

46:33Hannah Fry: Right. All right. We have one more. When Miss Bates heard of Mr. Elton's engagement to Augusta Hawkins, she was quite put out. Why? It's simply not fair, she exclaimed. For everyone knows that three is a crowd and four is a quadrilateral. So bad. Yeah. OK. I see the comedian still have a job. Yeah, I think this is the specificity there is really at the edge, aren't we? All right. So one out of three. One out of three was good. Decent show. It was decent. Quasi one out of three. Yeah. It was one kind of sufficiently across the line. Yeah. And the other two probably would have had tomatoes thrown at the stage.

47:09Okay. We have some work to do. We have some work to do. Okay, great. Well, that was our AI element for today. All right. Shall we do rapid fire?

47:24Is there a movie, song, or book that fills you with optimism for the future?

47:29Hannah Fry: I'm going to go for When We Cease to Understand the World, because it's just, okay, so beautiful. But it's like, it's just, it really captures how exhilarating it is to be at the brink of new knowledge, I think. It really makes me feel great, read about six times. And by the way, since we just did the comedic thing, Stephen Fry read that book and reached out to Labattu. Oh, yeah. Yeah, because he also shares all of our passion. It is a great book. Yeah, it's really amazing. Really amazing. Because they did the Hay Festival together, didn't they? Yes. Yeah, that's right. And that came out from that reach out.

48:02Hannah Fry: Oh, wow. Amazing. Amazing. All right. What is a question that you wish people would ask you more often? Do you want another drink? Done. I don't know. I don't know. What do people answer for this one? Perhaps the funnest one was I asked a friend of mine's kid, and the kid looked at me and said, do I want to be here? Amazing. I'm sticking with my first answer then. I think that fits better. All right. So where do you see progress or momentum outside of your industry that inspires you? Oh, I think the stuff that's happening in biological spaces is really incredible. Physics is lucky that it has equations to discover, right?

48:42Hannah Fry: You know, you can look at all of that data of the galaxies and then you can come up with E equals mc squared. Like, I mean, almost impossibly simple, right? And biology doesn't have that luxury. But I think that we're now at the situation of almost where you can take the unimaginable complexities of biology and extract a sort of working model for how it fits together. And I think that is really, really exciting. Awesome. All right. Can you leave us with a final thought on what you think is possible to achieve in the next 15 years if everything breaks humanity's way? And what's the first step to get there?

49:16Hannah Fry: Let's go crazy optimism for a minute here. Please. Because I think that actually, you know, the history of humanity has always been a story about scarcity, right? It's been about resources being divided. And I do think that actually there is a way for science to make a gigantic difference for everybody. There are so many different areas where if science makes a breakthrough, like desalination or nuclear energy or whatever it might be, right, there's a number of different areas. Good battery design, right? These kind of things where we just need like a little bit of a breakthrough. And I think everything, everything, everything can potentially change.

49:56Hannah Fry: Thank you so much for being here. Thank you. That was so fun. Thank you. That was great.

50:29Thank you.

50:35Ian Alice, Greg Beato, Parth Patil, and Ben Relis. And a big thanks to Taylor Forster-Corness, George Kingston, Irenia Alvarez, Jenna Antonick, Ayi Kano, Natasha Maines, Joshua Balogun, KJ Arthur, and Sophie-Claire Armitage.

From the publisher

What happens when we take numbers seriously—not just at face value, but in terms of their actual impact on our lives?

This week, Reid and Aria are joined by UK-based mathematician and media personality Hannah Fry to explore this deceptively simple question. From cancer diagnoses to dating apps, Hannah makes the case that understanding the math behind our decisions—and the "game" we're really playing—is essential to building a better future. 

They discuss why regret minimization is often a better guide than raw percentages in assessing risks; Hannah’s advice on finding love in a sea of swipe-rights; and how a well-framed question can be more powerful than the right answer. Plus, Inflection’s Pi chimes in with a few Jane-Austen-themed math jokes.

For more info on the podcast and transcripts of all the episodes, visit https://www.possible.fm/podcast/ 

Topics:

3:01 - Hellos and intros

3:15 - Considering hairdressing, instant feedback, and becoming a broadcaster

5:11 - Making math emotionally resonant

7:37 - Math as power and who gets to shape the future

9:10 - Understanding AI through probabilities and spectrums

12:01 - The emotional limits of statistics in medical decision-making

16:22 - Choosing the right “game” to optimize for

19:57 - How to build systems that self-correct

23:19 - Rethinking education and assessments in the AI era

26:42 - Updating dating advice for the AI age

30:03 - Real-life “Black Mirror”

36:11 - Storytelling vs. statistics

41:47 - Probability thinking, AI confidence, and useful heuristics

45:31 - Pi delivers math-themed Jane Austen jokes

47:15 - Rapid-fire questions

Select mentions: 

Google DeepMind: The Podcast

The Mathematics of Love TED Talk

When We Cease to Understand the World by Benjamín Labatut

“Women also Snowboard: Overcoming Bias in Captioning Models” by Kaylee Burns, Lisa Anne Hendricks, Kate Saenko, Trevor Darrell, and Anna Rohrbach

Possible is an award-winning podcast that sketches out the brightest version of the future—and what it will take to get there. Most of all, it asks: what if, in the future, everything breaks humanity's way? Tune in for grounded and speculative takes on how technology—and, in particular, AI—is inspiring change and transforming the future. Hosted by Reid Hoffman and Aria Finger, each episode features an interview with an ambitious builder or deep thinker on a topic, from art to geopolitics and from healthcare to education. These conversations also showcase another kind of guest: AI. Each episode seeks to enhance and advance our discussion about what humanity could possibly get right if we leverage technology—and our collective effort—effectively.

More from Possible with Reid Hoffman

All episodes