#201 Danny Halawi: The Science Behind AI Predictions - Accuracy, Challenges, and Innovations

0:00We release a paper showing that an end to end language model system can forecast future events almost at the level of human crowd aggregates almost at the level of these prediction markets. And in some specific settings, they're actually better than the prediction market. We wait till the question resolves. And if the model made an accurate prediction, which basically means it gave a reasoning that was effective at predicting the event. Like we collect this reasoning and then we train the model on these reasonings because we found them to be effective for making correct predictions. Hi, I'm Craig Smith and this is Eye on AI.

0:38In this episode, I speak with Danny Halawi, a PhD student at UC Berkeley working on AI forecasting. Danny shares his research on using large language models to predict future events, such as election outcomes or sports results, at a level comparable to human forecasters and prediction markets. We discuss the potential of AI forecasting to help individuals and organizations make better informed decisions, the architecture of the AI system Danny and his colleagues have developed, and the challenges of improving its accuracy further. Danny also provides insights into the world of prediction markets and the wisdom of crowds.

1:23I hope you find the conversation as fascinating as I did. AI might be the most important new computer technology ever. It's storming every industry and literally billions of dollars are being invested. So buckle up. The problem is that AI needs a lot of speed and processing power. So how do you compete without costs spiraling out of control? It's time to upgrade to the next generation of the cloud, Oracle Cloud Infrastructure, or OCI. OCI is a single platform for your infrastructure, database, application, development, and AI needs. OCI has four to eight times the bandwidth of other clouds, offers one consistent price instead of variable regional pricing, and of course, nobody does data better than Oracle.

2:15So now you can train your AI models at twice the speed and less than half the cost of other clouds. If you want to do more and spend less, like Uber, 8x8 and Databricks Mosaic. Take a free test drive of OCI at oracle.com slash IonAI. That's E-Y-E-O-N-A-I, all run together. Oracle.com slash IonAI. That's oracle.com slash IonAI. Yeah, I did my undergrad at UC Berkeley in computer science. At the time, I was very interested in computer security. So, you know, after graduation, I worked at Google and Robin Hood, where I was doing machine learned fraud detection. And then, you know, around the time that GPT-3 came out, I thought it was a big turning point for AI, as a lot of people did, both from an advancement standpoint, and also just as a, you know, what are the implications for broader society?

3:21you know, as a for the labor force, just from a safety perspective. And so I felt compelled to sort of like go back to school and do work in this space. So I started working with Jacob Steinhardt at UC Berkeley. And, you know, I was really interested in understanding how models work. So, you know, we know that models work very well. We know neural networks are strong, but we don't necessarily know how they work. And so this is a problem that really gripped me because these things are deployed in high-stakes situations, but we have no idea what the underlying algorithm that they're implementing is to make decisions and do various tasks.

4:05So this is one area of research that I was very interested in. And then recently, we've done work on automated AI for automated forecasting with AI. So this is like forecasting future geopolitical events with AI systems. So, you know, we believe that this has implications for AI safety and also just implications for, you know, the broader public, how they could potentially use this technology if it were to work. So you can imagine that if we had a system that could forecast you know, future events better than humans, we would be able to understand the results of taking certain actions and making certain decisions.

4:51And if we could understand those things, we could, you know, make better informed decisions on what actions to take. So currently, you know, we have forecasters that are humans that, you know, forecast different events. So the really good ones are called super forecasters. And the thing with superforecasters is that there's not a lot of them and they're expensive. So if you can have a machine that could forecast at the level of humans, this would be very valuable because you could distribute this technology, you know, widely and people could use this to make more informed decisions. And then also you can imagine as AI gets really advanced, using AI to forecast AI itself, meaning using AI to forecast when AI will be able to do a certain thing or when it will get to this level of capability is really important so we can understand, you know, what is the world going to look like in two years?

5:54So these were the two things that got us interested and using AI for forecasting. And yeah, that's what I've been up to recently. Yeah. And so tell me about the system that you've built in the paper that you wrote. I guess start with the prediction markets, which I wasn't really aware of, and how those function, and also how the super forecasters function. Are they doing deep research around an event? Yeah, absolutely. So I guess the thing I'll start with is some context, which is the thing we're doing is referred to as judgmental forecasting versus time series forecasting. So time series forecasting typically excels when data is abundant and when it doesn't change much.

6:49So it's under minimal distributional shift. And here in this field, you take tools from time series modeling. So like you can imagine if you want to forecast the weather, you just collect a lot of data that you have currently in the environment and use some time series modeling to make a forecast about the weather. Judgmental forecasting, on the other hand, is where people assign probabilities to future events based off historical data, their intuition, their Fermi estimates and their domain knowledge. So it's like very much intuition based and it's very much just thinking about, you know, what are the possible outcomes?

7:25What are the factors that I should consider and what are the probability of the different factors? So we are interested in the field of judgmental forecasting. OK, and to study this, you can take a look at prediction markets. So prediction markets are like essentially these platforms where people put probabilities on different events happening. So anybody can go onto these prediction platforms, take a look at a question that interests them, like who's going to win the next U.S. election, and they can put a probability on each candidate. And you have many people doing this. And based off what people think, this will determine what the platform or prediction market thinks is the true probability of this event.

8:06So if you have like 500 people putting a probability on, you know, Trump or Biden or some other candidate winning and you take a look and you aggregate all this information of what all these different people think, the prediction market puts a probability on the event. So like one natural concern is like, OK, well, some humans are probably better at forecasting than others. So like, why should the platform consider everyone's probability? And the thing that these platforms do is they weight people's probabilities based off their past performance. So if you've been on these platforms for a long time and you've done very well at predicting the future, then the platform puts a lot of weight on your probability because you've done a good job in the past.

8:55So they have reason to believe you'll do a good job in the future. And another interesting thing I'll say about these prediction markets is that if you take a look at the aggregate forecasts of forecasters, so you take a look at what the prediction market says is the probability, which is based off of the individual forecasters, this probability is usually more accurate than the probability reported by the expert forecasters. And the reason why this happens is because every individual has their own biases. They base their forecast off the information that they find compelling. And when you average the probabilities across many different individuals, you're sort of getting all the different pieces of information that each individual is considering.

9:50and you're also sort of averaging out this bias factor that each individual has. So this is referred to as the wisdom of the crowd. And that's why prediction markets are a very valuable tool and are very successful. And it's because by taking what everyone thinks and averaging them in some way, you can get a more accurate forecast than you would have. So that's basically the field of judgmental forecasting. And this is sort of the state of prediction markets. I guess I'll also add that there's been, I mean, prediction markets have been around for a bit, but there's been a lot more excitement about them recently.

10:31So like just a lot more user participation. This has almost become like a trend or a fad where people go on these like prediction markets and just, you know, start, you know, making probability, start submitting probabilities for different things. and people will even like, you know, start like propose their own questions. So there's been a lot of excitement in this space and a lot of attention. And which I, which I think is a good thing for this field. You, let me just ask before we go on to, from that, the, what's the accuracy? is there sort of an aggregate accuracy for the different platforms?

11:16And I would imagine that when you break that down to different domains, they're more accurate in some domains than others. Yeah. And then if they are 70 % or above accurate, are people using them to gamble? because that seems like an obvious use case. Yeah, so I think certainly, so these prediction markets don't actually use real money. So that's important to say. You're not actually going on and placing a real bet. Although that would probably increase the accuracy of some of these questions because if you're actually using real money, maybe people are more incentivized to really think through the question.

12:02Um, people do like rely on the prediction markets for making their own decisions. Um, like people do use the wisdom of the crowd to, uh, actually like incorporate into their life when considering different things, whether it, uh, to, you know, maybe take on this major in college, like maybe, maybe like people who are entering college right now are trying to understand like, okay, if I go into this field of medicine, we'll just be completely automated by AI in two years. And so I think there are people who really like take information from these prediction markets and use it practically as far as like betting goes.

12:47So there's lots of questions on sports on these platforms. And I wouldn't be surprised if people do use them to make sports bets, but I would be surprised if anyone has the use them successfully to make money. And I think that's because when you think about the things that we can bet on, like sports events or maybe elections, these things are extremely hard to predict and very chaotic. So even if you get a more accurate probability from the crowd, which you will, there's just a level of noise that you can't really beat, which still allows basically the house to win almost every time. So, yeah, that's the current state to answer that.

13:33Yeah. But what's the reliability or the success rate on these platforms? Yeah, yeah. So up until 2022, I could give you some stats on some of these platforms. So an earlier paper by Andy Zao and colleagues looked at three platforms, Metaculous, Good Judgment Open, and CSET Fortel, which are three prediction markets. And across all questions, the crowd got, I believe, an accuracy of 92%. But if you look at the same exact platforms in the last two years, which is what we studied, their accuracy falls to 77%. This isn't to say that they're getting worse. This is just to say that the types of questions that they're forecasting are getting more and more difficult.

14:38So I would say, I mean, maybe one thing that's helpful. as a statistic is like these prediction markets, if you were to compare them, like, let's say you were to do a competition with like individual forecasters that are, you know, really, you know, have a lot of experience and are very smart and are have known to be accurate. These prediction markets on average tend to be even better than them. So the accuracy will completely just depend on the difficulty of the questions you're considering. But as far as how powerful are these markets, I would say that they're quite strong. And I think that they will only continue to get more strong as you get increased participation.

15:21Yeah. And before, again, we go into your paper and your system, just aggregating the predictions of the prediction market seems would be one step towards a stronger prediction. If you have, I don't know how many there are, five or ten markets uh probably um the the questions are phrased slightly differently and that would create a some noise but uh but yeah can you can you boost the prediction simply by aggregating the prediction markets themselves yeah absolutely i mean sort of like one of the key takeaways from the forecasting literature is like the more sources of information you have, the more signals you have, the better your forecast is.

16:13So there are actually currently platforms. I can't remember the name of the one I'm thinking of, but there is a current platform that simply aggregates the prediction from all the prediction markets for a particular question. So this is, yeah, this is done in practice. And I think one thing that you pointed out is you have to be careful because the questions might have different wordings on different platforms, which would create different probabilities for the event. Or they might have different resolution criteria. So like one platform, like two platforms will have the same question, but maybe one platform will say like, this will resolve as yes, under this criteria.

16:53And then another platform will say this will resolve as yes, under this criteria. For example, let's say the question is, will the Starship launch be successful. You know, people have different criteria for what successful is in one platform might outline, you know, certain criteria for a successful launch and the other one will have different, different criteria. And so, uh, yeah, you just have to be careful with respect to that. Um, but overall this, this would be a very intuitive and natural thing to do. Yeah. Uh, okay. And then, uh, let's talk about what you have been doing with, uh, large, large foundation models.

17:30Cool. Yeah. So recently our group at UC Berkeley, with Fred Zane, Yuhan Chen, and Jacob Steinhardt, we released a paper showing that an end-to-end language model system can forecast future events almost at the level of human crowd aggregates, almost at the level of these prediction markets. and in some specific settings, they're actually better than the prediction markets. So yeah, I could tell you about, we could start with the data set collection and I could tell you about what the task is and how, and the motivation for it and why we set it up this way. And then I could tell you how our system performs.

18:15So on the data collection front, there's all these prediction markets in the wild and most of them have APIs or it's easy to you know scrape the data online and they allow you to do this and you could scrape the questions you can scrape what the crowd thinks at different dates so for you know the crowd prediction is constantly changing over time as a new information is accrued it's not like there's a question they asked the crowd once and they use that as a probability like they asked the crowd at, you know, across a question's open date and close date. So the question might be open for six months or something.

18:53And the crowd prediction changes. So you can get all this data. And then given that you have these questions and this data, you can ask an AI system like a language model what it thinks the probability of this event happening. And you can compare it to what the crowd thinks. And then you could just see, I mean, eventually at some point the question is resolved, the outcome happens, and you can sort of compute the accuracy across many different questions over time and see how accurate your system is. So this is the setup. So we thought that language models would be like a really good place to start with getting, you know, superhuman machine forecasting.

19:37We thought language models was a good place to start for a couple different reasons. The first is that, like, for human forecasting, typically humans or expert forecasters or people who do this as a profession, you know, forecast in specific domains. So if you want really good forecasts about the environment, like, there are certain people who are really good at this. But this limits you because you have to rely on experts. And some questions, you know, call upon interdisciplinary thinking, which isn't always trivial to think about. So language models are trained on, you know, web scale data. So they're endowed with mass cross domain knowledge.

20:18So they're not, they don't, they don't succumb to this limitation. Furthermore, they can parse and produce texts extremely rapidly. So they're timely, you know, you can invoke them at any point in time, you know, during the day or at any point in time during, during your decision process. And lastly, they can be trained in a way that humans can't. So language models as they currently stand are trained on some very large dataset and then deployed and then used. But once they're deployed, they're not getting updated or retrained with recent knowledge. I think good evidence for this is if you go to ChatGPT, You ask it like who won, you know, the last NBA finals, I won't be able to tell you because it was trained up until 2021.

21:10Once it was released, it wasn't trained again. So why is this useful and interesting? It is because I can ask a language model about events that have already been resolved or have already happened and see what its prediction is and compare that to the crowd prediction. OK, in addition, if I let it look at information that was only available up into some date, I can sort of simulate forecasting. So like today, if I were to forecast the state of the election, like who what I who who I think is in the US election, you know, go online and I have information available up until today, March 8th. Right.

21:53And tomorrow I have information available up until March 9th. And so you could, you know, retroactively go to these, you go to these models and just give them information up into some date and then have them make a prediction. And this will effectively be like, okay, what did, like, what would this for, what would this model have said if it was really forecasting on this date? So you can set up this method of evaluation and method to train these models on information that has already been resolved in a way that's automatic and effective. So, like, one method you could even consider is having a language model that was deployed and trained up until, you know, with data up until 2023, to sort of teach and guide a model that was trained only up until 2021.

22:43The point here is that the 2023 model would have so much information about things that have happened that it could, you know, have the earlier language model make predictions and it could help you, you know, correct its predictions and learn from those learn like it could help it learn where its mistakes are and how it could have made a better prediction and improve accordingly. So you can have this, you know, completely automatic training between two systems where there's literally no human involved. So we thought that this would make language models like a really good place to start for forecasting.

23:18Interestingly, like this idea has been proposed and tried in the past. So like we're not the first people to like come up with this idea or try it. And, you know, in a study that was released up in 2022 by Andy Zao and his collaborators showed that, you know, their system achieved an accuracy around 65%, whereas the crowd accuracy was 92%. So their system was just like a lot worse than the crowd. But as language models have become more capable over the past two years, we thought to try this again, potentially with better, you know, sourcing even better information and, you know, training this system in more advanced ways.

24:08And what we observe is that, like, the model has gotten more accurate. So our accuracy was 71 % versus the crowd accuracy of 77%. And what we hope to show here is that language models are a good place to train, you know, machines to forecast. And we believe like Like this gap will only become more narrow over time and eventually machines will become better. And if this is the case, we hope to make, we hope that this technology is distributed across organizations and people so that they can use it to make more informed decisions. Yeah. Yeah. And the data. So let's say you want to predict, you know, a sports team, not that I'm particularly interested in gambling.

24:57But you have the Super Bowl. And are you relying on the information encoded in the weights of the models? Or are you supplying the models with reams of data on individual players, on historical outcomes? on what kind of data are you providing and how do you collect that data? Yeah, great question. So I'll answer that question. Then I'll also talk a little bit about how our system was trained since that'll give also insight on what sort of data the model is privy to. So yeah, so at test time, when we're actually having the model make forecasts, what we do is we give it access to essentially the internet.

25:52so up until some date, of course, because we don't want it to have information after a date. And what it has access to is what we let the system do is decide what it wants to search on the internet. So for forecasting, let's say the Super Bowl, the system will say, okay, I want to see what experts are saying. I want to see who's currently injured. I want to see what the weather is going to be like for the game. So the system itself will come up with everything that it wants, what types of information it'll want to see. And then we go and grab that information. And then the system looks through, let's say, 300 articles, and it determines which ones are relevant and which ones are irrelevant.

26:32So when you go try to look up the answer to a question on Google, you might have to look through a few articles before you actually get the answer to your question. So the system does the same thing. We grab a bunch of articles and we decide which ones are like most relevant to the questions it was asking. Okay. Let me just stop. When you say grab the articles, are you talking about physically downloading content and then uploading it to the model? Are you talking about copying URLs into the model? Or yeah, how does that happen? Yeah, so we grab the content of the article. And the way we do this is we use news APIs.

27:11So there's a bunch of news APIs out there. Google News is one we use. Another one, there's a startup called News Catcher, which we think sources like excellent information. I expect that to do very well. And they provide, you know, the body content of an article. So, you know, we load this into a data structure, all the content in each individual article. And then we have the model go through each article and, you know, look through the article end to end and see if the information is relevant. So, like, this is step one. Okay. And let me stop you there. When you say you, are you putting the content into, like, a vector database or are you?

27:57Yeah. Uploading PDFs to the model or how? Yes, we're feeding it into its context. So just like you prompt it, like just how you go on like some, you know, web app and prompt a language model a way that we typically do as, you know, just common users. This is exactly how it's being done. So we're not like embedding the model. We're not embedding the article in a special way or encoding it in a special way. We're just giving the plain text to the model within its context and then asking it some questions like, do you think this is relevant? given like what you wanted to know. Does that make sense?

28:34Yeah. Cool. Um, so yeah, so once, so once you determine which articles, once the, once the system determines which articles it finds relevant, the next step is to summarize this information. Why do we summarize this information? It's kind of for the reason you just said. So we're giving this information to the model within its context window, um, which might be 16 ,000 tokens or 32 ,000 tokens. So we have a limited space to work with. So we take like the 30 most relevant articles and we summarize them to condense the information. And then we give it to like a final language model and say, okay, look, these are all the retrieved articles based off the questions that you ask, based off what you found was relevant.

29:15Finally, like apply these like forecasting methods to make a forecast. These forecasting methods could be like, what do you think the reliability of each information is. What do you think, like for each piece of information, how much weight do you put on this for affecting the outcome? And like, what is the historical base rate of this event happening? You know, that's another thing it might consider. So it just applies all these tools from forecasting and then makes a prediction. So that's how the system works in deployment. I've been talking to journalist researcher and some data scientists about doing something similar, gathering information around a news event and seeing if a large language model or an ensemble of models could determine the probability of truth of any particular narrative around that event.

30:24And the point is that there are narratives that are not adopted by the dominant narrative. There are sub-narratives that are excluded. And, you know, you can think of a million examples in a conflict. There's, you know, eyewitness accounts on Twitter or social media that don't necessarily get picked up because they're considered too anecdotal or too biased or whatever. And in the same way in this prediction, as you add data, presumably the prediction will change. I mean, if you're using the news sources, that's one source of information. But if you use something noisier like Twitter or, I don't know, I'm not young enough to be using social media that much, but whatever platform is active on a question, you may get a different prediction.