AI Predictions for 2025: Geopolitics, Agents, and Data Scaling — With Alexandr Wang

28:20I mean, I think similarly, I think in everyone's personal lives, you're also juggling and navigating a whole set of various priorities. You know, I'm planning a trip with my friends over here and I need to, you know, get gifts for my family and figure out what they want for Christmas. And then I need to, I have all of these sort of personal projects which are still sort of like sitting there. And so I think in the same way, helping you sort of like level up on top of all of the projects that you're navigating and sort of like help you sort of coordinate between all of them more naturally. I think that's something that we're going to start seeing.

29:01Now, I don't know the perfect way that that happens, right? I think that the product experience is so important as a part of this and having a product experience where you don't expect it to be perfect, but you expect it to be pretty good. I think that's like 99 % of the challenge. And that's why we haven't seen it yet, despite the fact that the models already can do a lot of this stuff pretty well. My 2025 prediction is that guys use AI agents to use dating apps for them. And some get found out and some don't. And we're going to see some stories about how like some guy like set it on autopilot and ended up, you know, lining up more dates than he could ever hope for.

29:48Yeah, yeah, yeah. Well, hopefully. Maybe that's already happening. Hopefully there'll be good dates. Yeah, I don't know. What are you seeing? I know you had Benioff on the podcast a little bit ago. What are you seeing as the things that seem to make sense from an AI agent's perspective? Well, I think that Mark Benioff, the Salesforce CEO, when he came on, talked pretty convincingly that we'll have AI agents at work. And again, this is like the work or the enterprise use case because work has all this data. And there are all these tasks that we do all throughout the day at work that are just arduous and really quite annoying, preparing reports, making dashboards, going to meetings we don't need to be in, pulling out highlights from those meetings, sending them to our bosses, telling our bosses, you know, in the Salesforce instance, for instance, like how each conversation went and what our expected pipeline is to close that quarter.

30:43and all of this stuff can be used for AI. I think it can be used with AI. I think it's really interesting in the medical use case. I was just speaking with GE Healthcare about how they've now put in dashboards for doctors sort of summaries of cancer patients, medical histories, which have run thousands of pages. And the doctors never had a chance to read the whole history. And now the generative AI is summarizing it and going out and finding available treatments for them. and notifying them when they miss tests. And I think this is also an example that Benioff gave about the healthcare example where that can actually be proactive in scaling medical advice and medical treatment in a way that you'd never hear from like your doctor after you showed up to an appointment.

31:30And now can they create an agent that just kind of keeps you on your plan, you know, in terms of like follow-up stuff that you need to do. On the consumer side, like for everybody else, that's kind of where I wonder because all of our internet has been designed to effectively combat bots but if we have agents that work on our behalf on the internet like travel sites dating sites social media sites I'm very curious like whether they're going to come up against these bot protection systems like are they going to do captchas on our behalf are they going to get the text messages and throw put fill in those numbers so they're able to log into different systems because again, the whole internet has been built to defend against these things.

32:10So I'm curious what you think. I mean, is this vision of, you know, personal agents that act on our behalf to do things like book travel, keep up with our health, take action on internet services for us. Is it even a feasible thing to do given all of the protections to sort of guard against them up until this moment? We will have to sort of fundamentally reformat how the internet works to be able to support it. And I think that we're going to need, in some senses, there will be two webs. There will be the web that humans use when they need to navigate stuff on their own. And then there will be the web that agents use, which is under the surface and something that humans will never see, but allows them to conduct actions on their behalf more efficiently and easily.

33:00And that I think will be in the long run what ends up happening. And my honest take is I think that to the degree that most of us, you know, there's sort of like two kinds of usages of the internet today. There's sort of consumption, which is where we're seeking out content and, you know, we're curious about things. And then there's utility-based usage. And I think the sort of addressable market, so to speak, for the agents is all the utility work. Like everything where I'm using the internet just to like get something done, I want that to happen faster, easier, better. I would rather have to not have to do that actively at all.

33:45Let's say it's like booking an appointment and looking up a particular piece of information or, you know, figuring out how to like, you know, fill out my tax return or whatever it might be, like that stuff should all be handled by the agents. And we're still going to have to, you know, do a lot of consumption of content just to sort of like, you know, as part of our, as part of what we like to do. And so, yeah, I think it's a really good point. I mean, pretty, it'll feel like a toy, just like with any technology. So maybe, you know, we'll all start with like a language learning agent, or we'll start with a cooking aid agent, or it'll just be something that feels pretty innocuous.

34:34But then we'll start to realize we can really rely on it. And then, and also relying on it for a lot more. And that's kind of what happened, I think, with ChatGPT. Initially, it was sort of, we realized, you know, it was kind of a toy. and then people started doing a lot of homework with it. People started to code with it. And then now people do all sorts of stuff with ChatGPT and other chatbots. That'll be the thread. Let me ask you this question before we move off of agents. Do you think it's ethical for me to like have my AI agent, which can type and talk, go out and email and call a bunch of humans on our behalf, people working, you know, let's say in customer service or I don't know if I'm applying to schools and they're trying to find out like information about like whether I qualify and what I need to submit.

35:19I mean, these processes, maybe they've been designed as arduous to sort of filter out the people who aren't willing to do the work to sort of get in or pass that application threshold. So it's in some way it's combating these guardrails that companies and institutions have set up for us. On the other hand, it could end up wasting a lot of people's time. Like I'm, I really am anticipating like no agent policies from like certain schools or institutions being like, if you're going to reach out to us, it has to be a person versus an agent. What do you think? You know, I saw this thing on Reddit. There was this post of how an admissions officer, she sort of created all these ways in which they could track whether or not an essay was AI generated or not.

36:13And there were very detailed things. It was very specific. There were a list to maybe 20 or so criteria that they looked for. And I think that, you know, to your point, it was kind of heartbreaking to see because that means that, you know, if a student used an AI to generate an essay, you know, they have to spend way more time just figuring out whether or not it was AI generated to like sift through all the noise. And so, yeah, I think you're totally right. I think we're going to need almost in the same way that there will be an internet for humans and an internet for agents. There will be processes for humans, processes for agents.

36:56And a lot of things that are high intent or very expensive or otherwise special in some way are going to be reserved for humans only. and it'll sort of be the sort of like more transactional stuff that can be handed off to agents in mass. That's right. I mean, in some ways I'm looking forward to this future. On the other hand, I do sort of think like the more we talk about it, how much AI will take care of for us, I do sort of feel like we're cannonballing our way towards that Wally future where we're all fat and drinking big sodas and having Roombas take us around the world. it's uh yeah i think i think uh uh ease and convenience which definitely are the directions that technology has taken us uh you know uh clearly there should be limits at some point but uh but if we if they exist we don't know where they are exactly and this idea of like removing friction in some ways it's made the world great in other ways it sort of changes the brain chemistry of people where like we don't expect to go through hard things.

38:07And when we do, we lose our minds. And that's why you end up seeing the YouTube videos and the videos on X of people in the airport, because we've removed so much friction and companies have competed on the base of customer experience to the point where now if something goes wrong, we're fragile. And we think that, you know, we deserve better. And there is something to be said for friction, toughens people up a little bit. Totally. All right. We're here with Alexander Wing, CEO and co-founder of Scale AI,$14 billion company that works with others to help generate AI data for them and also help them scale their AI solutions.

38:51We're going to talk a little bit more about Alex's third prediction when we come back right after this. Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points and miles. Now, imagine a credit card with rewards that can grow in value. With the Gemini credit card, you can earn Bitcoin or one of over 50 other cryptos instantly with no annual fee. Every swipe at the store or gas pump earns you instant rewards deposited straight to your account. Plus, sign up now for a$200 Bitcoin bonus to kickstart your rewards.

39:27Visit Gemini.com slash card today. Check out the link in the description for more information on rates. Again, if you're looking to invest in Bitcoin but don't know where to start, the Gemini credit card makes it easy. The Gemini credit card is issued by WebBank. In order to qualify for the$200 crypto intro bonus, you must spend$3 ,000 in your first 90 days. Some exclusions apply to instant rewards in which rewards are deposited when the transaction posts. This content is not investment advice and trading crypto involves risk. The Gemini credit card cannot be used to make gambling related purchases.

40:23and wherever you find your podcasts.

40:32And we're back here on Big Technology Podcast with Alexander Wang, the CEO and co-founder of Scale AI. So Alex, I want to ask you about this interesting shift that we're seeing, right? So up until this point, we've talked entirely about AI models on the basis of how many GPUs or chips they're trained on, right? It used to be that you could train a model on like 16 chips, right? By the way, they're not cheap, like$20 ,000 to$40 ,000 each. Then I went to$1 ,000. And now towards the end of the year, we started hearing crazy numbers like$100 ,000,$200 ,000. I was just at Amazon's reInvent conference in Vegas, and Matt Garman, the CEO of AWS, told me that they're going to train the next anthropic model on hundreds of thousands of GPUs, GPUs or GPU equivalents.

41:25And then I was like, oh, that's a lot. And as he's saying that, Elon Musk came out and was like, well, we are going to train the next XAI model in Memphis on a million GPUs. So I think we're really hitting, like maybe we're hitting the limit, I don't know, of what you can do with chips. and so you believe that we're going to shift this conversation beyond chips in terms of what makes the most powerful model so i will tee you up for prediction number three yeah and so so um so much of the dialogue to your point over the past few years has really been around gpus and computational power and i think what's going to happen in 2025 is we're going to um we're not going to only be focused on who can create newer, better chips or bigger data centers with more chips, but also who can create newer and better data.

42:16And one of the things that I think we're going to see is a focus of the focus shift from just computational power to computational power plus data being sort of considered nearly equally. You know, data really is, at its core, the raw material for intelligence. So the conversations around data are going to be really interesting. And one of the big topics that's been bounced around for the past few months has been, you know, are we hitting a wall? Have we hit the data wall? And are we hitting a wall on progress overall? And I think the interesting thing that's been happening is, you know, this has come from an approach of scale up computational power at all costs.

43:01If we just scale up the number of GPUs and create huge, bigger and bigger data centers of GPUs without creating more and more data to train these models on, then we're going to hit issues and we're going to hit walls and barriers where we stop seeing the level of progress that we expect out of the models. So one of the big things that we see, especially in our work with a lot of the Frontier Labs, is it is true they're scaling up the GPU clusters, they're scaling up the number of chips. That's still a very aggressive path for them. But the in-parallel conversation is how do we scale up data? And there's two sides of that.

43:43One is obviously scaling up the volumes, but also scaling up the complexity. So they're seeing the need to go towards more of what we call frontier data. So go towards advanced reasoning capabilities, agentic data to support the agents that we were just talking about, advanced multimodal data. We just saw today, for example, that OpenAI released Sora. And so the needs for video data and more complex combinations of video, text, audio, imagery, etc. altogether is going to be really, really interesting going into the next year. And so I think one of the lessons that's really played out more recently with the models is that you can't just scale GPUs and expect to get the same levels of progress.

44:30You need to have a strategy by which you're going to scale up all three of the pillars. You need a strategy to scale up the compute. You need a strategy to scale up data. You need a strategy to continue improving the models. and it's only through the sort of concert of all three of those things that you're going to be able to get, keep pushing the boundaries and barriers on AI progress. But I'm curious what you think. I mean, you've talked to all these CEOs. What are they talking about? I mean, this is exactly the thing that they're talking about. We had Aiden Gomez from Cohere in a couple of weeks ago and he basically said that this has sort of been the path of training the models.

45:06Whereas in the early days, you could effectively bring anybody off the street, take down anything they had to say, and it would be new information for the models. And then you started to have to bring in grad students to talk about their, because that general knowledge base was built. So then you bring in grad students to talk about their area of discipline. Then you go to the PhDs. And he goes, where do we go next? Because we have all this general knowledge. And now we have all the specialized knowledge that we've used to train these models on. And by the way, it's just amazing the way that they've improved and been able to sort of handle some complexity.

45:39It's really crazy. And so the question is like, where to go next? And I think that's what you guys are working on now. And I'd be curious to hear what the process is like on your end for, you know, generating more data for these models to train on. Yeah. So it's exactly what you just mentioned. Like a lot of what we're focused on is how do we bring in expertise and really this sort of expertise from every field you might imagine, from medicine to law to math to physics to computer science to even knowing about really advanced systems of various kinds or being a great accountant or whatever field you might imagine.

46:24getting the sort of all of what are all the arcane knowledge what is all of the sort of really specific deep knowledge that exists in each of these areas and pull that into you know large scale data sets that we can use to help train these models to keep improving in a lot of these areas and a lot of the a lot of the effort for us has been something that we call hybrid data So one of the things that we've seen over the past year in particular is that synthetic data has not worked as well as I think everybody had hoped. Pure synthetic data, just using data generated from the models to try to train future models, that can sometimes cause real issues for the models.

47:09And so one of the things that we've been really pushing forward is this idea of hybrid data. So you have synthetic data, but you use human experts to mix in with the synthetic data to ensure that you're producing data that's really, really accurate and high quality and won't cause issues. But also you're able to do it very efficiently and at large scale. So you also have those PhDs that will sit down and kind of write what they know or dictate what they know. And then you feed that into the models. Yeah, exactly. And a lot of times it's even more targeting than that. You know, you run the model until you realize the model's making mistakes over and over again.

47:47And then, you know, you've hit sort of a limit of its knowledge or limits of its capability. And you have a PhD sort of come in and help, you know, set the model up on the right track, so to speak. What's the limit then in terms of where are we going to get to? Because if we let's say we have all these specialized fields input their knowledge, does that eventually make like AI complete? if it just kind of knows everything about every subject? Or does it have to hit like a new benchmark to really show that it has this like next level intelligence? Like does it have to start making discoveries of its own?

48:21What do you think the benchmark should be? Yeah, I think, well, to me, I think there's like clearly many more levels of improvement. So now it's sort of testing, okay, can it do each of these things right once? or how are there's sort of the first track was just reliability. So getting these models from doing something right once in five times to write 99.99 % of the time. And that requires a lot of development just to get to that, you know, increase the level of reliability of the systems. And then to your point, it's really about how can the model start taking more and more actions in a row?

49:01You know, one of the things that really is true in all the models today is that they're not that good at, you know, at taking multi-step actions. Whenever it has to take a few hops, whenever it has to take chain a few things together, it'll invariably make mistakes along the way. And so the next level of improving reliability is really enabling the models to do more and more multi-turn, more and more multi-step reasoning to be able to enable them to sort of do more and more complex tasks. And then the last piece as we go is, and this is the key to where you're going, is like eventually it'll be able to start making its own hypotheses, running those tests on its own, and sort of ultimately making its own sort of discoveries or realizations or sort of conduct its own research.

49:57and even then it's still going to get stuck sometimes and still going to need a human PhD to come in and sort of help it just in the same way that like you know a PhD student these days you know still needs an advisor to sort of still give it the right nudge and so um and so I don't think the sort of like the symbiosis so to speak between the humans and the AI will ever go away like I think we'll always be able to sort of will always be very important in helping the models get on the right track and ensure that they always are continuing to improve. But we're going to see the model sort of level up in terms of what is the degree to which they're able to be autonomous and the degree to which they're able to operate on their own.

50:39And on the multi-step thing, right, taking a bunch of different steps, I heard something interesting from Moody's last week, and I want to run it by you, where they said basically they've created 35 individual agents. So let's say they want to evaluate something for their portfolio, like a company for their portfolio. They'll have one that will look at one agent will look at the financial data. Another agent will look at the, let's say, weather risks. Another agent will look at the location that they're based in. Another one will look at the industry. And they have 35 different variables or whatever it is.

51:10And then they have all of them come back and they deliver their results to this compiler agent, which evaluates all of it and And then runs the results by voting agents, which ask, OK, is this reliable or not? I walked away from that impressed by the idea, but also like kind of my reporter brain went off and was like, I don't know if this is real or not. So I'm curious what you think. Is that a possible solution? And how feasible is that in terms of a way to get into these multi-step processes? So that's a very – in my work, it's a very sort of regimented way to try to enable the systems to do multi-step reasoning.

51:51Because ideally, what you want the model to do is to, just like how a human does, be able to sort of go through and figure out what are the bits of pieces it needs to know as it goes along and be able to do so on its own dynamically without having to sort of like predetermine and preset this entire regimen for the models to need to go through. So you're saying that might be something that a model can do entirely on its own. That's pretty cool. I think in the future, like, we're going to – the models will improve to be able to get there. And I think the real – on the multi-step side and the multi-step reasoning point, I do think that the – there's a lot of blockers because this is the kind of thing that humans learn how to do kind of from a lot of trial and error and experimentation.

52:44like we'll try to um do a complex task and then we'll realize we'll learn that oh we actually missed you know let's say you try to bake a cake for the first time you know a reasonably complex endeavor and then you realize you missed a b c and d and then the next time around you'll be like okay i'm definitely going to remember i just had a pan of flour that came out of the oven where did I go wrong?

53:11And, but yeah, exactly. I mean, like we learn a lot through trial and error. And right now the models are, the models are early in the process of doing the same thing, of going through and sort of, and being able to do these sort of dynamic processes where they learn through trial and error and they are able to continually learn from their mistakes. That's where we need to get to. Okay, great. I know we have just a couple of minutes left. so let me throw a couple um quick hits at you and then we can head out first of all i'm just curious we talked a lot about how data is going to matter a lot but i can't get my mind off the fact that elon's going to try to build this million gpu uh super cluster what's your prediction for what that spits out i honestly think right now at where we are in ai development today um we are more bottlenecked by data than we are compute so i think we'll just have a incremental improvement then was something like that yeah i think that i think the real step changes come from data okay um so just a quick follow-up to that if we um if we end up to like i just saw there was a news from google today about this breakthrough they had in quantum computing which we'll probably cover more on the friday show uh if we have working quantum computers which can process data much faster what do you think that does for ai um i i really think so i had the opportunity to to tour Google's quantum facility earlier this year.

54:38It's very impressive. I think quantum computing is on kind of like the way AI was back in 2018. It's on a few scaling laws where you can definitely sort of squint and see that in 5 to 10 years, this is going to be a really, really impactful technology. And ultimately, I think what it's going to enable is it's going to speed up AI's ability

55:05to do scientific discovery. And so whether it's, you know, I think a lot of the use cases that excite people are in biology or chemistry or fusion or a lot of these very chaotic and difficult to understand, you know, natural sciences. I think that's where quantum computing has the ability to be pretty transformational fundamentally. And I think AI will be able to use it as a tool to be able to enable it to do incredible research in those fields. That's crazy. Okay. So, all right. Last one for you. We're in the middle of like this race where it seems like every week the foundational model companies put out a new development, whether that's open AI, whether that's Anthropic or even XAI, Google, Amazon just released a set of new models last week.

55:53So who do you think is in the lead at the end of 2025? five? Oof, that's hard to say. I mean, I think that one thing that we see today with the models is that because all the benchmarks that were used today are what's called saturated, i.e., you know, in other words, like all the models do really well at the benchmarks, it's really hard to discern actually which models are fully on top versus not on top. You know, there's a lot of argument, for example, on the internet, at least in the Twitter feeds that I see, in terms of whether Claude is better or a 1 is better. And there's all these comparisons between the two of them.

56:36So one of the things that I think we're going to need in 2025 are much, much harder benchmarks and much, much harder evaluations that are going to be able to help us figure out, separate the wheat from the chaff a little bit. I don't know who's going to who's going to be in the lead, but I do think that I think that we need much better measurement to actually be able to discern between all of these incredible models that that labs are pushing out right now. OK, all right, we'll take it. No, no prediction on who's going to be the best, but a definite interesting perspective on evaluations. Alex, great to meet you.

57:12Thank you for coming on the show. I think these predictions have been fascinating, definitely stretched my mind in areas that I wasn't thinking about. So thank you. And we hope to have you back sometime soon. Yeah, this was a lot of fun. Thanks for having me. Thanks for being here. All right, everybody. Thank you so much for listening. We'll be back on Friday with Ranjan breaking down the news. We will see you then on Big Technology Podcast.