309. Why we’re wasting our time predicting the future of AI

0:01Steph McGovern:Experts are rubbish at predicting the future. In fact, it's been proven that monkeys throwing darts at a board is just as effective as asking specialists to tell us what they think is going to happen next. So instead of making predictions about AI, today we're going to talk about why that's dangerous and what we should do instead. Dr Laura Gilbert is Senior Director of AI at the Tony Blair Institute for Global Change. Now, she previously founded and led the data science team at 10 Downing Street, as well as the government's incubator for AI. She's going to explain why open source AI is what the UK government should be focusing on.

0:39Steph McGovern:But is it safe to give our valuable information over about things like the NHS to the public and even more importantly, to US tech firms? Plus, the story of how she, as a civil servant, saved over£40 million in half an hour. Here's my interview with Dr Laura Gilbert. Support for this episode comes from Octopus Energy and the founder and CEO Greg Jackson is with us now. So, Greg, talk to me about Kraken. What is it and why is it spinning off from Octopus Energy? When we built Octopus, we also built a software platform, like an operating system like iOS or Android on your phone, to enable energy companies, utilities, starting with Octopus, but frankly, anyone in the world, to be much more efficient, to use vast amounts of data, to be able to become more innovative and to serve their customers better.

1:38Steph McGovern:That piece of software is cracking. We've used it not only for Octopus, but companies in the UK like EDF and Eon have used it to improve their business too. But you know what? It turns out over time, companies realized that they were licensing from a competitor. And so we've had to spin it off so it can reach its full potential. Yeah, makes sense, Craig. Thank you. Right, we're going to go to the episode.

2:09Steph McGovern:with the right skills, certifications, and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this show will get a$75 sponsored job credit at Indeed.com slash podcast. That's Indeed.com slash podcast. Terms and conditions apply. Need a hiring hero? This is a job for Indeed Sponsored Jobs. This episode is brought to you by Facebook. So you were scrolling on Marketplace, and there it was, the bike you'd been searching for. You sent a message, and it turned out the seller was super chatty, kind of funny, and an avid cyclist. The next thing you know, you're in a cycling crew.

2:48Well, a community cycling group.

2:50Steph McGovern:The thing about Facebook, you might find more than what you're looking for. From a browse to a bike ride, this summer, find more on Facebook.

3:09Steph McGovern:Laura, thanks for joining me. Now, you've had a really interesting career. You know, it spans particle physics, quantitative finance, you know, you've done stuff in business with the government. So I imagine you come at tech and the adoption of AI from lots of different perspectives. Where have you seen the most change? Yeah, I mean, I've used technology in a lot of different ways. And obviously, the way that we can use it now has been radically transformed in the past couple of years, you know, particularly in terms of actual development. The ability to use, you know, coding agents was really the realization of a bit of a dream.

3:53I literally remember probably 10 years ago thinking, why do I still have to manually type out a for loop every time I want to iterate over a data set? You know, how is how is this not something that can be done for me? So, you know, our ability to build at speed and particularly, I think, ideate at speed has been massively transformed. Now, if you have an idea, almost anyone can go into Claude and get, you know, a vibe coded dashboard or something. And I think, you know, some people view that as a bad thing, but I think it really helps us to innovate. It helps us to turn over ideas really quickly.

4:26It helps us if we want to and we should to do a much better job of meeting user needs. You know, you can throw something up quickly, figure out how people would use it, what they actually want. Does it do the thing that they need it to do before you invest huge amounts of engineering time? So I think it's been properly transformative in the way we actually deliver technology, the sort of recent advances in generative AI. And that's been hugely positive. Obviously, there's a number of risks that come with that. When you democratize something like software development, you also make it easier for people to build things that are unsafe and that can inadvertently cause harm and of course it's also democratize access to people that want to do bad things so there are pros and cons but in terms of you know the day-to-day job I think it's been massively positive

5:16Steph McGovern:yeah and it's interesting because you're talking there about how you know you use technology use AI I guess to try and predict what's what's going to happen next to you know to work out what people are going to need next. But you're a real, I guess, you're really passionate that we are rubbish at predicting the future, aren't you, when it comes to things like what's going to happen with AI and the future of the workforce. We don't get it right, do we? No, we don't. And we've got a lot of evidence that says that we don't get it right and we shouldn't expect to. And I think it's really interesting watching this space.

5:49I do get asked an awful lot what's next for AI. And I don't know, I didn't see ChatDBT coming. The people that built it didn't really understand what it was going to do, didn't see DeepSea coming. There's a whole raft of new technologies that I've seen hints of, but it's going to be very difficult to predict at exactly what happens next. And if you'd asked me five years ago, what jobs will be changed by technology, I would have said, you know, shelf stackers, not software engineers and photographers. So, and, you know, more widely than that, we've got this very interesting academic research. My favourite study, and I talk about it probably at least once a week, is the Tetlock study in the 1980s.

6:33And Philip Tetlock got nearly 300 policy professionals, and he asked them to, you know, predict the future sort of 20 years down the line. You know, is this, you know, piece of world politics, will it be happening more or less or about the same as now in 100 different domains? And I waited 20 years and he counted up the results. And he said they've done about as well at predicting the future as monkeys throwing darts at a dartboard. So, you know, slightly better than random chance, very slightly, much worse than what the paper calls, and I love this, the minimally sophisticated statistical model, which literally means go and find out what happened before, draw a ruler through it and carry on into the future.

7:16And that's your most basic, you know, model prediction you can ever do. So they did much worse than that. And then there was some research that followed that showed actually the more eminent, not only in that research, was there a hint that people were worse at predicting the future in the areas that they're expert in. But there was a follow up piece of research that showed us that actually the more eminent you are, so the more of the platform you have, the more you speak in public on something, the more respected you are, the less good you are at predicting the future, which is not like experts expertise is useless.

7:48It's not. I mean, clearly, if you if you're going to have a teeth removed, you should have an expert dentist. But what it does say is that our expertise does not confer the ability to more accurately predict what's going to come next. So it's a real risk, I think that we're seeing across the world, you know, for everyone from political pundits to sort of business leaders who think that they know a little bit more than everybody else and they'll be able to predict where this technology is going to go. And we have evidence that they won't.

8:23Steph McGovern:Yeah. So in terms of what they know, what they're good at is telling you about what has happened rather than what's going to happen. Yeah. And it's I mean, there's expertise is good for a lot of things. If you want somebody to apply expert knowledge to a thing I'm doing now, expertise is incredibly, incredibly helpful. But yet it doesn't confer on its own a better ability to predict the future. And in some ways, it can also harm your decision-making process. So there have been really interesting studies in hospitals. There's one I enjoyed, which showed in America that if you had a serious heart attack, this is less fun actually on the heart attack side, but the statistic is interesting.

9:06If you have a serious heart attack and you go into hospital, you are less likely to die when there's a major cardiology conference on and all of the consultant cardiologists are not in the hospital. What? Yeah. So again, you want to be seen by a good cardiologist, but actually when it comes to the kind of crises, there's an interesting side effect of expertise where people who are very expert, they've done something many, many times and they start to sort of lose the ability to see points are different. So they just go, well, this is the same thing I've seen before, we'll do the same thing. And you can find that, say, a more junior doctor is more worried about missing something, checks more carefully, spots that your case is a little bit different and they should try something different.

9:55And, you know, you sort of came out with higher survival rates. So it's very, very interesting. And you can actually, you know, another thing that Tetlock sort of discovered when he ran a very big competition was that there are ways to get better prediction. But actually, it doesn't have much to do with expertise in the topic. The people that do really well at predicting the future are very good at what we call Bayesian updating. When you learn new information, those people are very good at ignoring any biases they've got, absorbing that new information and changing their position. And I find that really fascinating because it's a skill you can learn.

10:36And people who are good at that, they outperformed even national security experts who had access to secret data that nobody else had, you know, members of the sort of security service, etc. Those people that were able to change their the way they thought and their beliefs based on the information coming in, they were the best at predicting the future. And what's really upsetting about this, if you're in the sort of space I work in, is that politicians are uniquely disincentivised from ever changing their minds. So if politicians seem to change their minds in public, it can cost them about 14 percentage points over politicians that stick to their view.

11:17Yeah. So the people who are the very best at predicting the future are exactly the people that we will never elect.

11:23Steph McGovern:So hang on a minute. The people doing U-turns are the ones that we should be keeping in. Yeah, pretty much. Yeah. God, this is fascinating. It's crazy, isn't it? But, you know, the public will not elect politicians that seem to change their minds. But across the sphere of leadership, the people that are the best leaders are quite good at changing their minds when they see that they've learned something new or something changes. And it's one of the reasons, actually, that women make very good leaders because they're slightly better at that. Yeah, yeah, this is fascinating. So then what you're saying is, should we abandon predicting stuff?

12:02Steph McGovern:So should we stop predicting what is going to happen next with AI? It's an interesting question. I mean, it's great fun. So I think people are going to carry on doing it anyway. AI is such a difficult area of prediction. So, you know, what I told you about the ruler is that you can do prediction really well. Generally, using a model to do it is much more successful. So, you know, taking out the guesswork, the instinct, the kind of, well, I had a bad day and therefore I'm looking at things differently. And picking all the data about what happened before and then saying, well, this is everything that happened before.

12:34So now I'm going to infer from that data what might happen next. That can be hugely successful. And it's particularly successful in specific areas, which is when we have data that is predictive. So, you know, if you are trying to figure out what to do with somebody with a heart attack, you have all of the other people before that have had heart attacks, you maybe measure everything about them and say, well, with this kind of person, with this kind of heart attack, with this type of data, it usually does this and therefore we should do that. So you can do a really good job of predicting the future with models.

13:07But with this kind of technology, we don't have a data set to work on. A lot of people have come in and said, oh, well, it's like the industrial revolutions and therefore our society will adapt to work differently in a similar way.

13:22Steph McGovern:Yeah, Robert and I have said that. I will own up to that. We keep talking about it as the industrial revolution. Well, yes, I think there are really major differences. Also, we don't have many industrial revolutions. It's not a big data set, right? So what we don't have basically in the AI space is we don't have any predictive data that says this will be what happens next. So broadly, yeah, I think we should stop. trying to predict slash guess what happens next. But what we do need to do, and we need to do it now, and it's really crucial, is we do need to make decisions about what comes next, which is a very different thing.

13:58So rather than waiting, let's actually choose.

14:02Steph McGovern:So let me ask you about when you do go into government to talk to them, what's your vibe in terms of their understanding of AI? And you know, what's the reception you get for them? Yeah, well, I mean, I go when I'm asked. And it's generally, you know, because of previous working relationships. There's a lot, there's still a lot of ambition, I think. You know, I've advised a couple of departments recently who are really seriously looking at how can we use our very limited resources, combine it with technologies that we can afford and that people will be able to use to do a much better job? How can we streamline the way we operate?

14:43How can we do a better job of pulling our data together? How can we improve our analytics? How can we improve our delivery at the point at which we reach the public? And I think they are really limited by sort of resources. And it's not easy to get headcount to suddenly go off and hire a load of engineers, you know, just because you want a tool. And it's not easy to go and get money to do it either. So they've been quite inventive in a number of places about, you know, how much they can do themselves. And the ones I really admire do manage to get people to come and do it for them for free, which was a skill of mine when I was in government.

15:22I ran a training program called Evidence House and we managed to get 17 ,500 hours of free technical training for civil servants. We spent 500 pounds of taxpayer money over sort of a year and a half. So I love it when I see that sort of thing. People are working really hard on it. But you do also see because of the way the civil service is structured and there's still really very low levels of sort of technical expertise.

15:52Steph McGovern:it's a low percentage, you know, people, they don't necessarily have a good mental model of what's possible. And so you can find with the conversations that you're having that, you know, either they think it's much easier to kind of pick up AI and just do absolutely anything and that this should be a solved problem. Or they think it's much harder and they end up spending too much money on it, you know, really big procurements for things that could have been really quite cheap. And that's where it goes a little bit wrong, I think. So I think the ambition's there. The willingness is there. A lot of the fear that we saw in the early days of generative AI has died down.

16:30People are no longer sort of terrified it's going to take all of their jobs or that, you know, it's really, really threatening if people use it. But there's still, you know, a little way to go as well.

16:39Steph McGovern:And just on procurement, then, we've talked about the the government being a bit rubbish at procurement in terms of supporting British businesses. You know, we've interviewed quite a lot of British businesses on the shore, some of them in the AI space who say, you know, often on a local level, they can get things done, you know, with councils, with regional authorities or whatever. But on a national level, it's really impossible. Yeah, but we do hear that a lot. I think the specialism that a large, successful company would bring to procurement of technology is not widely available in the civil service.

17:19It's not something where they've hugely prioritised hiring and making that available across departments. And of course, it comes under a huge level of scrutiny and a huge level of accountability. So there is a real incentive on this. there's a couple of sort of incentives that get in the way. There's the incentive to make sure that you're probably not taking bets on small companies because, you know, if it goes wrong, you're very much in the public eye for that. There's also too much procurement in some spaces in that as an individual civil servant who maybe doesn't have a tech background but yet is responsible for delivering a tech programme, you're not going to get a headcount.

18:02You're not going to be able to hire an engineering team, but also you don't know how. You don't know also how you would build it yourself. So I will have built most systems, at least in some form myself. I understand how to do it. So if somebody comes to me and says, well, it's going to cost you 40 million pounds. And I go, this is a real example. There was a quote for a system for 40 million pounds. Is this when you were in government? Yeah, I had one of our engineers go and build in half an hour.

18:29Steph McGovern:So you got a quote from a company saying this will cost 40 million. but then you got one of your engineers to build it in half an hour. Well, yes. The head of another team was procuring this system and he called me and said, we've got this growth of 40 million, you know, seems a bit high. So I asked him what it was exactly he needed and then, you know, had one of the developers do it in half an hour. It's a really extreme example. Was that a British firm? No, it was a multinational consultancy. Right, because that's the other thing here, isn't it? They can have your eyes out. Well, yeah. Yeah.

19:01And there is something there where we do need a bit more responsibility from companies that are coming in and pitching to government. You need the responsibility for them to give the government good advice and not try and gouge them, particularly to this kind of extent. There's a big difference between companies that will come and work with government and build the thing they need and deploy it responsibly. And other types of companies that will come in, and I've seen it time and time again, sell you something they've already built that in no way meets your needs. It takes a few years to deploy.

19:32And by the time, you know, the government realises they've got the wrong solution, you know, they're 50 million quid down. And it should be something that we name and shame for. I think, you know, there should be a perverse incentive because you are taking taxpayer money. It's not, you know, this isn't a business deal in the same way as if you're ripping off your competitor. So, yeah, we do. You know, it's a really big problem, I think. And procuring well is difficult. There's not an incentive on them, especially to do that well. they don't get a bonus or a promotion if they manage to spend less money and get it delivered faster and it does really good things.

20:05The incentive structure in terms of getting a promotion is much more about how many people can you get under you and are you sucking up to the right people above you and how good are you at that. The system isn't really set up to do procurement well and it's something that needs a massive overhaul and again I know the government's looking at.

20:22Steph McGovern:What needs to change for us to it sounds like we need more people who know what they're doing like yourself they understand tech in the civil service we need to take more risks on british businesses i think so yeah so so what in conclusion then what what are your thoughts on what needs to happen next what do you want to see happen next with the government in terms of at least having an attempt at catching up on the ai race yeah i mean if i was um you know running the world in this space the two things that i do think are the most important to fix are procurement doing that really well. I think they need to bring in specialists who really understand how to do this.

21:01And the other thing that I would fix with urgency is the incentive structure. The technology is almost a knock-on effect of some of that. It's really hard to hire experts into a system where they can't see how they're going to benefit. They can't see how they're going to get a promotion and build a career, which is why we do a good job of bringing them with these individual teams like the Safety Institute or the, you know, incubator, where they know what the mission is, and they're going to build a thing they're really interested in. But it's quite hard to do that in departments, where, you know, you're never going to get a bonus, you're never going to get promoted, you know, and it's going to be really, really, really hard.

21:41Incentivising people to do a good job with recognition, with ways to sort of move up the organisation. And by doing a good job, it is to make sure that the actual outcome is delivered. And giving them a lot more freedom in the way, you know, you should hire the right people and give them freedom in the way that they actually do that. Do they need more people? Do they need, you know, more contracts? Like how are you actually going to do this is something that you should hand over to people that really know what they're doing. And at the moment, the incentivization is completely perverse. It really you're attracting far too many generalists and not enough specialists.