2024 In Review, 2025 Predictions — With Casey Newton

0:00Let's look back at 2024 and predict what's going to happen in 2025. in our annual episode with Casey Newton of Platformer and the Hard Fork Podcast. That's coming up right after this.

0:14You're used to hearing my voice on the world bringing you interviews from around the globe. And you hear me reporting environment and climate news. I'm Carolyn Buehler. And I'm Marco Werman. We're now with you hosting The World Together. More global journalism with a fresh new sound. Listen to the world on your local public radio station and wherever you find your podcasts.

0:59As always, to do it is Casey Newton of Platformer and Hard Fork. Casey, great to see you. Welcome to the show. Alex, let me be the first to wish you a happy new year. Happy new year, man. Here we are, 2025. Wonderful to be here in 2025. I know. I mean, we're recording this in 2024, but it is very cool. Don't tell them that. They don't need to know that. But we're manifesting the future together right now. We're manifesting. We are. Looking back at 2024, it is interesting because it was a year of tremendous progress. progress technologically. And we spoke about this a little bit last year about how we're starting to see some real tech progress, as opposed to like, repackaging or, you know, new things that we talk about on social media that aren't actually technological advances.

1:42So we saw big tech progress in 2024. And that is, of course, in the world of AI, first and foremost. But I think I've been surprised when I think about the big picture of 2024, how difficult it still was for people to make use of that technology. We're still talking about what the ROI is of this more than two years after ChatGPT was released. And towards the end of the year, you also did a piece talking about how the people who say AI is fake and it sucks aren't really correct. Look what it's doing. But actually that they have a leg to stand on sort of says something about where AI has gone, which is that the tech has been cool, but still don't really know exactly what to do with it when you think about the totality of dollars invested.

2:30So I want to get your perspective when you look back at 2024. Does that seem right? And why do you think that happened? Yeah, I mean, I agree with some stuff in there. You know, my starting point is that just over two years ago, ChatGPT didn't exist. Recently, OpenAI told us that ChatGPT had 300 million users a week. So to go from zero to 300 million is really impressive and almost never happens, right? And along the way, they've started to generate huge amounts of revenue. And there are those who say, well, but they're not profitable yet, Casey, and therefore they probably never will be. And I just find this answer very naive and sort of ahistorical.

3:18You know, you look at the history of Silicon Valley, and it is a story of companies that were deeply unprofitable for a long time that are now some of the biggest companies in the world, right? I never thought Uber was going to turn a profit. Uber is now a profitable company. Some people will say, well, they still haven't made back the money that they took in from investors. And like, fair point. But I think they're probably going to get there, right? So when I look at AI, I see something that, yes, is very expensive to train. But when you look at the fact that it's a general purpose technology, and that we are finding more and more things to do with it, I do believe that at least one of these companies is going to make a massive amount out of one of these large language models.

4:00Right. So I would say if I had to put a bow on what you're labeling 2024 as maybe the year of continuing to figure it out, I know it doesn't have a catchy slogan style ring to it. But that seems to be what you're saying. Yeah, I mean, I also think that we saw real progress on the technology side. What we did not see was the sort of leap in capabilities that we saw from GPT-2 to GPT-3, and then GPT-3 to GPT-4. And I think something that grabbed a lot of people's attention this year was the fact that what are sometimes called the scaling laws, which is the ability to improve a large language model just by increasing the size of the model, the amount of data that you run through it, and the amount of computing power that you put through it, that apparently started to see some diminishing returns.

4:56And some people said, aha, the whole thing is a house of cards. It's about to collapse. But along the way, people started finding all these new, what they call scaling vectors. So for example, you can take the model that you already had, and you could say, hey, why don't you think a little bit longer before you give me an answer? Turns out that greatly improves the result, right? So just because one of these sets of scaling laws has started to diminish doesn't mean that there aren't going to be others. And it's why over the long term, I'm bullish on the idea that, again, at least one of these companies is going to figure out the right combination of things to deliver something really powerful.

5:30And that's 2024. So let's go on to 2025 and start our predictions. And we're going to talk about AI, but we'll also talk about a bunch of different technology and we'll talk about antitrust. So if you want to hear about what's going on in those areas, stay tuned. But in context, thinking about the fact that, okay, we had that year in 2024, some progress, some questions. Do you think we're going to see a giant leap in 2025? Like you talked about those step changes when we thought about what happened last year. Is that coming? So based on the conversations that I've had with executives of the AI labs, it feels like people are trying to set expectations for more linear progress next year.

6:14I do think we are going to see one really notable set of new capabilities in what they call AI agents. So an agent is essentially a computer that is using itself. Anthropic was the first of the big labs to put out a demo of this. Google put out their own demo the week that we're recording this. And I expect that OpenAI is going to release their own pretty soon. If you look at the demos that have been released so far, they're pretty janky. The computer is still figuring out how to use a mouse. The computer is still figuring out how to navigate a web browser. It's kind of like watching an eight-year-old try to learn to use a computer.

6:54But our experience with these same sets of technologies in other contexts has been that they learn very quickly. And so my prediction about what is next year in AI, it is all of the big labs and platforms showing off what their agents can do. businesses trying to figure out, is there anything in there that is powerful and reliable enough that we can actually use it to start automating some processes inside our company? And that's going to lock some stuff. I don't think it is going to feel like AGI. I think it is going to feel like unnecessary, difficult, but ultimately impressive extension of what AI is already doing today.

7:37So I'm also putting together my predictions and I think my big technology story will also be out about this by the time this podcast goes live. I'm not fully committed to this, but one note that I've written down is that AI agent buzz fades. And I can't tell you why exactly I believe that. It probably goes counter to what all these research houses are saying. But I think in a nutshell, the reason why I would make that prediction, and I'm open to the fact that I'm going to be wrong, is that the use cases are tough to find. Sort of like, what do we trust the computer to do for us? And now in an area like a B2B software company, when it comes to like fetching things or moving one job from one system to another, maybe we see some Asian progress, but my hunch says the hype around them fades.

8:28What do you think? Well, I mean, let me give you some examples for what this thing could do, right? Let's say that you gave Gmail, you put some sort of agent inside Gmail and you said, okay, Gmail, find every invitation in my inbox from the last month and show me those. And then from there, I'm gonna tell you which of these things I wanna go to and which of these things I don't wanna go to and you write the acceptances and the replies and I just click send. That's like actually possible with the technology that we have today. Nobody has built the product, but like we already have that. That would be hugely useful to me.

9:07If I had that agent go through my inbox and say like, show me everyone who has asked for a response from me about something, that would be very powerful, right? So I just think that there are some like core productivity use cases that people are really going to like, you know, I already use AI as a kind of thinking partner. you know there's like a kind of classic thing that happens in tech reporting where maybe you're writing about a new paper that's coming out or there's a new product that's being released and the company that you're talking to will like send you some materials about it i now upload that directly into a chat bot i ask i essentially interview the pdf before i interview the person after i do that interview i'll say like hey i'm actually going to talk to these people you know in a couple hours what are some things to talk to them about some of the questions are awful and I would never ask.

9:58Some of them are good, but mostly it's just like a great first draft. Right now, I'm putting a lot of like intention and effort into doing that. But like you can imagine a world where it's a little bit more on rails. And I say like, hi, agent, I'm writing a story about this. Here's the PDF. You know, all the questions I usually ask, like, you know, the kind of stories that I write, give me a little bit of a template for this and like essentially prepare a briefing document for me about the story I'm about to write. again like we already have the technology for all this stuff like we just really need to build the product so i predict next year you're going to see technological progress but the question mark is like how effectively are these people able to productize these experiences yeah and i do something similar uh i should probably start where you start but oftentimes when i'm writing something i'll write the story and then i'll drop the finished story in with all the primary sources and be like what did i miss like here's the transcripts of the interviews i did yeah there must have been some points that I didn't include and it can pick it out pretty well.

10:57That's great. Yeah. It's getting really good at that sort of thing. And like the reason I really liked that example is, you know, we're talking at a time when I've been besieged by haters calling me a shill. And the thing that they'll often say is like this stuff, it just does not work. It constantly hallucinates. You can't rely on it for everything. And if the way that you want to use it is as a search engine to do all your fact checking, you're absolutely right. It's gonna be a terrible fact checker. You cannot rely on it. You have to go elsewhere to verify your facts. But most of it, particularly people like you and me, Alex, who are solo entrepreneurs, we don't have a newsroom full of colleagues to help us think through every angle.

11:38This stuff is actually really good at that, right? You're not relying on it for the facts. You're saying, help me think about this thing. And if you don't see the value in that, well, I don't know what to tell you, brother. Yeah, it's valuable. I also think it's pretty nice for grammar correction. Like I canceled my Grammarly subscription and now I use Claude. Oh, but can I give you a prediction? Grammarly dead by end of 2025. How does that company still exist? I don't know. I don't know. And honestly, it's also a company that makes its product worse every year, but we could go on and on about that.

12:10They had a great browser extension and I think it's always about what the company prioritizes. and I think they wanted you to go to grammarly.com. So instead of being able to work inside your docs, you copy and paste it for a more robust service. So they killed the browser extension that was really useful. Or they didn't kill it, they just diminished it. And then it's like, what am I doing here? You know, it's a step I don't need. But Claude, I'll be like, just tell me where my usage is weird. And I'll be like, you wrote exits instead of exist, which by the way, I do in every story I write. And I'm like, okay, great.

12:44Well, I'm gonna put exist now. So that's helpful. yeah i do that too now i will say i also ask claude to edit my column and um if it gives me six suggestions for like you know a spelling or grammatical mistakes three of them will be fake they will like it'll say it'll say you did this thing and then i go back and i read the copy i'm like no i didn't so i you know i always want to acknowledge like that we haven't finished building any of this yet some of it is like really crappy but i like it is just in my nature to look for the ways in which it works as opposed to the things that it is not good at.

13:18It's a true gas lighter. That's the truth. And I think that that might never change. Yeah. If I had a wish that I would put under the product building tree, I don't know if that actually works as a metaphor, but I'm going to roll with it. It would be that the people building AI built for more normal use cases and not for like Tim Cook's use case. Like that's my number one complaint with Apple intelligence or like I pick up my phone, it's summarizing my text messages and my emails as if I, I mean, I'd get a fair amount, but I'm not getting like Apple CEO level. And I could see Tim Cook being like, you summarized a hundred emails.

13:53That is amazing. That is incredible. Whereas for me, it's like, you know, it summarizes two or three and it takes all the personality out of a text message. So if you want a year in review, a thought for me on that, what I would say is Apple intelligence flop of the year. Oh my gosh. But honestly, I love it. This is not me being mad calling it a flop. This is me celebrating what a flop it is. Because I'm in a very active group chat and the boys are sending me the memes all day long. And it used to be in the little preview window, I would see the meme. I would see the tweet. I would see the joke.

14:27Now, four times a day, I get my little summary and it's like, link shared to x.com meme discussed and i'm like i truly cannot imagine a more worthless addition to my life but it makes me laugh every single time so apple intelligence you are a complete flop and i hope you never change i mean to me one of the funniest things about apple intelligence is just how mean it could be uh or like someone gets a breakup text it like goes this is uh something that people have been circulating on twitter it just goes well she's not interested or like yeah Yeah, even when I get an email where someone's like, sorry, I can't do the podcast.

15:02It's like, invitation rejected. I'm like, I'm on Apple Intelligence. Do you think there's a second life for this product in 2025? Or is it sort of - Oh, sure. I mean, they're not going to give up on it. Like, presumably they also know it sucks because they use it. You know, I think it was interesting. They did a big sort of sit down with Wired, like all the sort of Apple C-suite in December. and the the point of it was to convey the idea um that hey look we're not behind on this we were just very patient we were biding our time and when the moment was right we we struck right and like now we've like really nailed it and it's like that's like a lovely idea but it's not true like you can just use this stuff it's like you know so i think where they're right is that the mainstream is either like not ready for this doesn't care about it or hates it like some combination of those three things they're right about that and that's actually like a perfectly valid reason for them to be going slower than everyone else in ai but you know if you're like a tech nerd like we are and you want to live in the future and you want to like use tools to improve your life you would not be using any of the apple intelligence stuff because it like it truly is aimed at somebody who is two years behind like the state of the art yeah we talk about this on the show all the time it still can't do basic things like pull a flight out of Apple Mail and tell you when it's leaving.

16:28Yeah. Shame on them. Shame on them. I'm just kidding. All right. I'm not. I'm going to take it one level further. Take it further. Say something really offensive. Let's say disgrace after disgrace because— Disgrace. Yeah. Yeah. Well, I'm going to go for the second disgrace, which is Vision Pro. Oh, yeah. And, you know, you predicted last year that the Vision Pro was going to usher in a moment where people are going to take mixed reality a little bit more seriously. Did I say that or was my voice deepfaked? We're going to have to look into that. Let's go with the deepfake. What happened there? So this was an interesting one.

17:05I was wrong in my prediction. Basically, my prediction for this year was that Vision Pro would not be a runaway hit, but that they would sell enough to justify continued investment in the device. And the most recent projections are that they were going to sell something like 420 ,000 of them this year. That puts them at close to$1.5 billion in revenue. That is a rounding error for every other Apple product. And so you could see that as it being a complete flop. On the other hand, if any other company exploring a new category released a product that generated$1.5 billion in revenue for the first version, that would be enough to justify continued investment in it, right?

17:52So on one hand, yes, not a runaway hit. I think where they really failed was to nail a daily use case. But by the way, everyone else who has built a headset has failed in this if the use case is not playing a game. but the hardware is really cool, man. They did some cool stuff. The eye tracking, very cool. The visual fidelity of the display, very cool. Would I enjoy watching a movie in it? Like probably. So they did a lot right. And I think like the version of that thing that exists five years from now probably will be super compelling. I am gonna admit that every now and again, when I have a visitor that comes into Brooklyn, I say, why don't we go to the Apple store?

18:34There won't be a line. Let's try out these Vision Pros. And they're blown away every time, but they're just like, I would not buy that. Wow. I want to come visit Alex Kantrowitz. And he's like, what can we do? We're in New York City. What can we do? Take me to the top of the Empire State Building. You're like, no, we're going to the Apple Store. I need a lightning cable. We don't have fun here. We can go on a slack line in mixed reality. That's even more exciting than the building itself. This is great. I'm excited. They are planning smaller, cheaper devices next year. Do you think those hit or is it more of the same?

19:04So I'm somewhat skeptical about this because I don't actually think the problem with the Vision Pro was that it was too expensive. Apple customers are rich. Apple customers were ready to spend$3 ,500 on it. Obviously, yes, there are many people that just cannot and will not spend$3 ,500 on a device. But if you look for adjusted for inflation, how much an Apple laptop costs in the 80s, the Vision Pro is not actually that expensive. right? So the reason people didn't buy this thing was that there was just nothing to do with it, right? It's like the Oculus headsets. You buy it, you try it out, and then you're like, this thing's kind of a hassle.

19:41And there's really nothing it can't do that I can't do more easily on my laptop. Or to the extent that it can do things that I can't do on my laptop, I don't really care about those things that much. And so you put it in a drawer and never think about it again. So if you told me like, hey, great news, they're making a Vision Pro that is exactly as good as the one that came out in 2024, but now it's going to cost half as much. It's going to be $17.50 plus tax. I'd be like, that's actually not that compelling because it still just can't do that much stuff. So they really need to work on the let it do more stuff.

20:15And I would note, their relationships with developers are really bad right now. They can't even get freaking Netflix to make an app for this thing. So there's a lot they need to do on the business and partnership side. But you know, I'm somewhat skeptical that they're going to do that because they love their app store monopoly. And they would just rather just live off the fat of the app store than like rebuild those relationships with developers. Yeah, it's amazing what so many years of a 15 or 30 % tax just to exist will do to your relationship with developers. Yeah, yeah, it really is. And yeah, I don't know.

20:48I'm curious if we ever see a reckoning for that. Like I imagine at some point that we will and it'll be like very satisfying to observe. But man, is it taking a long time in coming. So there are some other AR platforms that we started to see this year. You and I both got a chance to try the meta Orion device, which is a pair of glasses that you put on your face and, you know, will take you into mixed reality without the big headset thing. I mean, this probably isn't a 2025 thing. Maybe it is. I don't know. Do you think that we're going to see these things get any traction this coming year? So my understanding is that the Orion headset is like multiple years out and that when it arrives, it won't actually even be as good as the glasses that we tried on this year, because in order to make the cost work, they're just going to have to like rip out some of the components.

21:37So that is super unfortunate. And you know, I mean, ar glasses has been like one of the first snapchat spectacles come out like a 2013 or something um that i truly feel like i've been waiting so long for these things to get good and like i'm going to be close to retirement before like orion actually like arrives but all that said and i said this to the meta folks when i was there orion is what i wanted the vision pro to be very high fidelity visual display, very easy to wear. It doesn't make me nauseous the way a VR headset does. Had had obvious utility. You could just sort of walk around the world wearing these things on your face.

22:19You could take a video call and sort of, you know, see the person in the lenses of your glasses. It was cool. Again, you know, the whole conversation we just had about the Vision Pro is about what it can't do. When I think about the Orion glasses, I think about what it can do and how enjoyable it was to use. So I think they've really got something there. The problem is just that the glasses are like extraordinarily expensive and they cannot deliver a consumer version of that for multiple years. Now, another one of your predictions last year was that Google catches up with OpenAI on the foundation model game.

22:52We're going to give that a ding. That worked out well and a good ding. And it has been interesting watching the way that Google has caught up and even surpassed OpenAI in a number of different fronts in the AI world in 2024. They've released Notebook LM, which is a great product. They had a bunch of releases towards the end of the year, including Gemini 2.0 and this research assistant that can look at dozens of different sources and compile research reports. And by the way, their foundational models are pretty good. They of course have search as a, I think, a continuing liability where people will try the perplexities of the world.

23:32Maybe they'll try a chat GPT search. And then you're like, well, what's going to happen to the bread and butter of the Google business? So I think that sets it up for a very interesting year in 2025, where it's the narrative coming in to last year might have been Google's behind, Google's screwed. Now it's like Google's competing. Google might be in the lead by the end of the year. But what exactly is going to happen to the company? So what do you think is going to happen this coming year? Yeah. So I'm like, I, of course, feel good that I actually managed to get a prediction, right? Like, if you look at Chatbot Arena, which is the site that measures the performance of the large language models as we record this, the Gemini models are at the top.

24:12So that's a sort of win for them. On the other hand, the quality of the models does not match my experience of using Gemini in Google products, right? The three big models that I use and pay for every single week, some of them every day, some of them many times a day, are ChattyPT, Claude, and Gemini. Of those, I would say Gemini has clearly been the worst. It had some really rocky launches this year. The launch of imagery where it was mostly refusing to draw white founding fathers was this huge black eye, and they had to remove image generation for months and months and months. They had the notorious launch of AI overviews where they were telling people to eat rocks and put glue on their pizza.

25:06And so there's just this disconnect where the models perform very well on benchmarks and the products kind of suck. So that is the circle that they need to square. And I don't, I mean, I think that they can do it. You know, it's really notable to me that Notebook LM, which is by far the best and most interesting of the AI products that I think that they have released so far, was made by a Skunk Works team in a forgotten corner of Google that struggled to get a lot of support and then went viral because they managed to create something really cool. And then as soon as it went viral, most of the founding team of Notebook LM quit to go do a startup.

25:51So it's like, that's like Google's real problem. You know, if I were them, I'm like, I would just go find the best product people in the world and just be like, go nuts in Gmail and Google Drive. Like make some stuff that is truly freaking useful to people because if they could do that, that is when I would feel like they were sort of recapturing the lead. Because as far as I'm concerned, all the cool stuff in AI is happening at OpenAI and Anthropic. And with the AI models being good, but the products being bad, is that just like a post-training thing? Or like, how do you end up in that situation?

26:25That's hard to do. Well, I have a conspiracy theory, which is, you know, kind of not great to be sharing on a podcast, but I'm going to do it anyway, which is I think Google is teaching to the test. You know what I mean? Yes. Like I think that they know what the benchmarks are, and they train the models to be really good at the benchmarks, and then they train the models, and they come out and they say, look, it does really well on the benchmarks, and then you just try to use it for other stuff, and it kind of sucks. You know, on my podcast, Hard Fork, in December, we had a conversation about what my co-host, Kevin Roos, calls the cult of Claude.

27:02And if you look in chatbot arena, Claude is not the number one model, but people have started to develop this very warm, humane relationship with it because of the way that it responds to questions, the things it notices, the probing questions that it asks you in return. And to me, that's the difference between teaching to the test and making a great product. Google's really good at the former, but just not at the latter. Yeah, I'll confess I have that sort of relationship with Claude. Yeah. What are you using it for? I use it for so many things. I mean, like all the writing applications that we spoke about.