The mathematics of AI uncertainty

22:04Zoubin Ghahramani:Yeah. And that all plays into this too. Yeah. I mean, hallucinations are a symptom, right? And of course, sometimes we want our models to hallucinate. So there's a tension between, you know, not hallucinating at all and not being creative, right? For example, if I want my large language model to write me a short story of that time that Albert Einstein went to the moon in a rocket, That's clearly a hallucination, but it's an act of creative writing. So it should be able to do that. It should be able to infer that that's what you want, infer your intent. If I ask it a factual question, it should try to be grounded.

22:43Zoubin Ghahramani:And of course, at Google, we think a lot about grounding our models in what's available. And even information on the web is often contradictory, right? And so you want to hedge your bets. So actually what you would like is a system that's able to tell you for any statement some estimate of its belief or probability. I think that's what we want. We don't have it yet. No. How might you build it? If you wanted large language models to have uncertainty in there, how do you do it? Because there are some quite good ideas. I mean, I'm thinking about semantic entropy here, right, which is one of your students.

23:20Tell me a little bit about that. How might that work?

23:22Zoubin Ghahramani:Yeah, I mean, I think you can take the internals of a particular model and try to infer from that its degree of belief. So before it answers, before it produces a token in a large language model, you actually have a probability distribution over all possible next tokens. And that probability distribution, the entropy of that distribution tells you something about the uncertainty. A low entropy distribution is very spiky, is very certain. A high entropy distribution is very spread out, it's very uncertain. And so there are ways of sort of teasing from the internals of a model how certain it might be.

24:02Zoubin Ghahramani:Yeah, I guess I'm thinking here about an example. Let's say the Eiffel Tower in Paris, right? Like those two things would be an example of something that's quite spiky. Yeah, yeah. Yeah, sort of like if you ask it, you know, where is the Eiffel Tower? It should have a high probability over Paris, although I believe there's one in New York as well, isn't there? There's a little Eiffel Tower. Vegas as well, I think they've got one. Maybe there's one in Vegas as well. So basically, you know, depending on the context, you might have little bits of entropy on these other Vegas and New York as options.

24:38Zoubin Ghahramani:But Paris would have a big spike on it. Is that sort of the idea then that like in the data set, Paris and the Eiffel Tower appear near each other a lot? You have a lot of like a really big signal there. Yeah. Whereas, I don't know, the Eiffel Tower and sort of Marrakesh. Yeah, that might have very low probability. The problem with that is that's sort of faking it in the sense that you're relying on the data. I'll give you the analogy of a calculator. It's like trying to build the calculator just by showing it examples of addition and multiplication. But imagine you never show it a particular number, then it might not generalize that particular number.

25:18Zoubin Ghahramani:And you don't want to fake a calculator. You want a calculator that actually calculates. You want it to actually reason about the world that it's in. That's what we want from our AI systems. So why hasn't this been done yet? I mean, what is it that makes it so hard to do? It is genuinely hard because it's computationally hard. So essentially, I would argue we had all the ingredients of AI maybe 15 or 20 years ago. Like we kind of know how to build rational systems. We just thought like, well, representing probability distributions over every possible thing is computationally intractable. It would take giant supercomputers that we don't have.

26:03Zoubin Ghahramani:and it would be incredibly slow. You'd be waiting for millions of years before you get the answer. So people abandoned that for let's go with just training from data and I think we can revisit this idea. Right. And I have ideas for how to do this and I'm exploring them now but it's not sort of super well formed yet. Okay, so moving away from the large language model or the transformer type system then for a moment because I mean there are other examples of really cutting-edge artificial intelligence, which does handle uncertainty in more of this Bayesian way that you're describing. I'm thinking about weather forecasting here.

26:41Ah, yes. I mean, that's really important. Yes, absolutely. Tell me a bit about that.

26:45Zoubin Ghahramani:Yeah, so we've developed a whole series of state-of-the-art weather forecasting models at Google DeepMind. And if you look at the GenCast model, sort of very recent model, one of the key features it has, it can predict sort of weather over 15 days. And it can do it very fast, much faster. It can do it sort of like in eight minutes rather than on a giant supercomputer for hours. And it's obviously using neural networks and things like that. But a key ingredient for getting this to work is that it uses a diffusion model. So that already is like the image generation models that already is manipulating probability distributions over time.

27:26Zoubin Ghahramani:But then it generates an ensemble of forecasts. So if you're trying to track a tropical storm like the Hurricane Melissa, let's say, which we did with this model, you have the data so far. And then you want to be able to forecast the track of this into the future because your decisions depend on that, whether you evacuate a city or whether you call in emergency services and so on. And so it represents that with an ensemble of forecasts. It has a whole probability distribution over the possible tracks. You rerun the model over and over and over again. And then as you get more data, so a few hours later, you get more observations, you update that ensemble.

28:16Zoubin Ghahramani:And that is essentially applying these basic ideas from Bayesian updating to this sort of problem. Because, I mean, the original weather forecasting models that came out here did not have this. Yeah, that was sort of added on. And every time we add on these features, it makes the model better. Because you're essentially saying there is an inherent uncertainty in the way that weather works. Absolutely. You can't just sort of run the model once and be like, oh, that's going to be the weather tomorrow. Yeah. You have to do it lots of times and then work out a probability. Yeah, yeah. It's a combination of both that inherent randomness, a sort of, you know, classic butterfly effect in weather and that it's chaotic.

28:57Zoubin Ghahramani:It's very hard to predict. But also there is an uncertainty just because we have limited numbers of sensors, right? So, you know, it's a system that represents its beliefs about the weather, whether it's the, you know, the trajectory of this hurricane and its intensity. And then those beliefs get updated as you take in the sensor measurements and as time progresses. I think there's something quite delightfully counterintuitive about that, that you add in uncertainty to the system and it makes the predictions more accurate. Exactly. Yeah, I think it is definitely a key insight in AI. I mean, it's not that you're sort of making the system noisy in an arbitrary way.

29:42Zoubin Ghahramani:That's not going to help you. But what you're doing is you're being honest about the fact that, you know, your sensors are inaccurate. Sometimes you get faulty sensors, just like, you know, sometimes my ears are blocked and I can't hear very well. And, you know, sometimes your model assumptions are wrong, right? You know, that is also a form of uncertainty. And so all of those different forms of uncertainty have to be represented somehow, approximately. We can't do it all exactly so that we can get better calibrated forecasts. I think the other example that really manages to get this uncertainty idea right is alpha fold.

30:21Zoubin Ghahramani:Yeah. Where the protein prediction, I mean, is color coded by how sure the model is that that's the correct folding, right? Yeah, absolutely. And essentially, you're fundamentally trying to solve an uncertain problem. You're going from a sequence to the folded structure. and the physics itself means that some parts of the protein are going to wiggle around more, so you don't know exactly where they are. And you also have just uncertainty because you've used a model to predict that. It's not like experimental data, and so you need to be able to represent that uncertainty in the sort of cloud of forecast of where the molecules are.

31:01There is a danger here that you could go too far the other way. You could end up with a model that was so honest about its uncertainty that it just sort of didn't ever really give you an answer. It just always said, I don't know.

31:13Zoubin Ghahramani:Just hedging, hedging all the time. That would be pretty funny. But, you know, I think there's different degrees of I don't know. How do you get the balance right then? Yeah, I think the balance is if there is a repeatable event in the real world, then you want to be calibrated in that if I say the chance of rain is 0.7 or 70%, then for all days that I've said that, if I sum up over all those days, if I'm calibrated, then on 70 % of those days it actually rained, on 30 % it didn't rain. So that's a calibrated probability. Now, if you ask me a statement of uncertainty about something that is not a repeatable event, So, for example, we may be uncertain about the first day a human being reaches Mars.

32:04Zoubin Ghahramani:So that's a date. It's either going to happen or it's not going to happen. And that date, when it happens, you'll be certain of it. That's the sort of event that, you know, you can have probabilities over, you can have beliefs over, and it will resolve itself when it happens, if it happens. And Bayes also has a way of handling that, where it's not just 50%, right? It's not just like it happens or it doesn't. Yeah, absolutely. So what Bayesian statistics tells you is that it's perfectly valid and, in fact, the right thing to do to use probabilities to represent your degree of uncertainty about things that only happen once.

32:42How do you communicate that uncertainty in a way that actually means it adds value rather than just has a human sort of nodding along or kind of taking cognitive shortcuts when the machine says it's really confident?

32:54Zoubin Ghahramani:Yeah, I think people have different reactions to AI systems. Some people are just very skeptical and will kind of not believe anything the AI system tells it. Other people are going to end up being over-reliant, let's say. Let's imagine a future where we've got, not a very distant future, but imagine a future where we've got AI systems in a medical domain, for example, and you have doctors aided by an AI system looking at patients and symptoms and test results and so on. It's a great example of why we need uncertainty. If the AI system says something, you really want it to convey its uncertainty because that's literally what is going to determine your treatment plan or whether you take one decision or another.

33:42Zoubin Ghahramani:These could be life and death decisions, right? So it's absolutely essential that, first of all, we don't become over-reliant on overconfident AI systems. But to be able to do that, we need our AI systems to be honest about their uncertainty and bring that uncertainty in a visible form to the human users so we can understand it. And I mean, you know, my colleague at Cambridge, David Spiegelhalter, has done a lot of amazing work on how to convey uncertainties and probabilities in elegant ways to the general public. And I think there are ways you can do that. You can visualize the answer and so on.

34:26Zoubin Ghahramani:So, I mean, I really believe that it's important to have AI systems that are doing complementary things that are additive to humans, that are helping people solve problems that we care about. And in order to do that, you want them to be honest about what they know and what they don't know. It feels like this is really very critical that we get this right. It's like such an important part of designing our collective future with AI. Yeah, I think it is really important. I mean, I don't want to take away from the fact that AI systems have been incredibly useful already. I think they're pretty good at some of these things and we can do better.

35:05Zoubin Ghahramani:And there are open problems along the way. And that's one of the reasons we actually need more research, actually. The thing is, I mean, I'm sort of sitting here agreeing with you. You're also a Vajian thinker, so this is very much my philosophy. But not everybody does. No, no. Not everyone agrees with you. I mean, there are some people who sort of say, look, you just put in more data and then you don't need to worry about uncertainty because the model will know everything. Yeah, I think that is a view that a lot of people have in the field. And they're not completely wrong, just like I'm not completely right, in that the models are actually pretty good at a lot of useful things.

35:49Zoubin Ghahramani:The problem is that when you stretch them in the long tail of sort of unusual things, then you can uncover some gaps. And also, I think when the decisions, if you're just interacting with a chatbot, the decisions may not be so consequential. But if we're trying to build self-driving cars that are reliable or, you know, medical AI systems that are helping us make, you know, diagnosis decisions, then we really do care about getting those probabilities right. Because I guess when it comes to building AGI, there's sort of, to oversimplify it, two camps, really. One which says you just need scale, you just need more data, more compute, off you go.

36:32And then the other that sort of we actually need a new architecture to be able to do things better that we can't do at the moment. It sounds like you're very much in the second camp.

36:42Zoubin Ghahramani:Yeah, I think we've made a lot of progress in the first camp. So, of course, our systems are incredibly useful. They're used by billions of people every day. But that doesn't mean that we've run out of interesting things to discover. So I'll give you a few examples of things that I think are important areas of research. One example is continual learning. So the way we currently train our models, and we means everybody in the field, we train a giant model and then we use it in products or we release it in various forms. and then a few months later we train another giant model and so on. If you compare that to how humans and animals learn, we learn continuously.

37:25Zoubin Ghahramani:We're basically constantly getting a stream of data and constantly adapting our connections between our neurons and so on. And our AI systems are not really able to do that very well. They suffer from things like catastrophic forgetting and so on and so forth. Which absolutely links back to the idea of Bayesian thinking that we were talking about earlier Because the reason why humans, animals are able to do that is because we have this updating system in our minds of incorporating evidence with existing knowledge. Yeah. So it turns out if you think of learning from a strictly Bayesian point of view, you have your prior beliefs, you get a data point, you update them, you get a posterior and you get another data point, you update them.

38:10Zoubin Ghahramani:It turns out that that sort of Bayesian updating can do continual learning and does not suffer from catastrophic forgetting and all these things in theory. Actually, many of our attempts at doing continual learning in large general networks are approximations of that Bayesian updating. So that's one area of research is continual learning. Another area where I think we may need breakthroughs is energy efficiency. So if you look at the power consumption of a human brain, it's about 20 watts. I don't want to make an equivalence. It's a light bulb. Yeah, it's a light bulb. It's a rubbish light bulb.

38:49Zoubin Ghahramani:Yeah, it's a nice, very energy-efficient light bulb, let's say. You know, if you compare that to training a large language model in a big data center, it's orders of magnitude off. I don't want to compare a single brain to a large language model because single brains involve, you know, kind of the single lived experience of a human being. Large language models are basically giant soups of, you know, all of world knowledge of some kind. But they're definitely less efficient. But they're way less efficient, right? So we can certainly afford to do more research in more energy efficient learning. I also think that there may be new architectures that we need to discover.

39:30Zoubin Ghahramani:Basically, the two workhorses of modern AI are transformers and diffusion models. They're great, but there may be completely other co-evolutions of software and hardware, like novel hardware architectures that may involve very sparse neural networks of various kinds. And, oh, I'll give you another one. Our learning systems are incredibly data inefficient compared to human and animal learning. So, again, I think Bayesian ideas can help us there. It sort of sounds a bit like you've discovered a magic trick, you know, like this magic trick, which like embeds humility and uncertainty, offers the opportunity for continual learning and makes data more efficient.

40:15Yeah, I'd like to think. And you can write it in a single line of Bayesville. So it's like, is this a little bit too good to be true?

40:20Zoubin Ghahramani:It's too good to be true, right? Obviously, we've known this for a long time. I think it's important to understand these ideas. So, you know, I think all students of machine learning should at least understand that it's possible to do these things. The magic trick comes with a big curse. The curse here is that to do all of this is computationally very slow. So essentially, you know, if you look at textbooks in AI, they will explain to you how to do some of these things. But they will say, we can't do these things exactly because they're computationally hard problems. They're like, you know, kind of NP-complete or NP-hard problems to solve.

41:07Zoubin Ghahramani:And so we need to approximate them somehow. And you could argue, well, we know how to build ideal AI systems using these concepts. Are modern AI systems our approximations to that? Can we have our cake and eat it too? Can we have the best of both worlds here? Because the thing is, what you just said there about, well, it's very computationally expensive, it's very slow. They were saying that about neural networks in the 80s. Yeah. You're not going to make that mistake twice. Yeah, yeah. I think we can revisit some of these ideas with the compute power that we have now. You know, the giant state-of-the-art supercomputer, parallel connection machine computer that I used in my undergraduate years is actually slower than the Pixel phone that I have in my pocket.

41:55Zoubin Ghahramani:Computation is getting better, faster. And also, we have decades of ideas on how to approximate these methods really efficiently. And so we have the tools. We just need to put them together and maybe come up with a few new ideas. What really strikes me is everything we've discussed they feel like very human-like qualities, right? Yeah. These sort of ineffable characteristics of humility and honesty and uncertainty and doubt. Yeah. I mean, did you imagine when you started all these years ago that this would be the thing that our systems are lacking rather than just computational power for analysis?

42:42Zoubin Ghahramani:I mean, certainly I didn't really imagine we would be where we are now, right? Because I think if you talk to Eddie, AI researcher, they will be saying that they're stunned by the rate of progress. But I also think that although these are human qualities we want to add, they're also kind of fundamental qualities of intelligence systems. So I think we need to have a concept of intelligence that transcends humans. Because, as I mentioned before, we have flaws in our reasoning. We're actually quite bad at making good rational decisions under uncertainty. in the real world, we will miscalculate probabilities or misestimate probabilities and so on.

43:25Zoubin Ghahramani:So I think if we build human-centric AI systems, we work backwards from, well, what do humans need? What are sort of society's biggest problems? What are humanity's biggest problems? And what are the AI systems that we need to solve those things? And for, I think, all problems that matter, I would rather have an AI system that knows when it doesn't know than an AI system that is arrogant and overconfident. What an amazing point to end on. Stephen, thank you so much. That was brilliant. Thank you, Hannah. For years, we tried to build AI that was focused on accuracy, right? Building systems that crunch through enormous amounts of data to land an answer.

44:07And these things, they're astonishingly capable, of course, but they're also quite brittle in some ways. when they fail, they fail with total confidence. But by teaching AI to embrace uncertainty, it gives it something more human, humility and honesty and the wisdom to doubt. And this isn't a vulnerability. It doesn't make AI weaker. It grounds it in reality, making it a collaborator we can actually rely on.