2023 In Review, 2024 Predictions — With Casey Newton

0:00Let's predict 2024 with platformers Casey Newton. And that's coming up right after this. LinkedIn presents.

0:12Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Well, it's the end of the year, and that means it's time to recap what happened this year and predict what's going to happen next. For the third year running, we're joined by Casey Newton. He's the editor of Platformer, the co-host of Hard Fork, and a wonderful repeat guest here this time of year. So, Casey, great to have you back. Thanks for coming on. Nice to be back, Alex. Thank you for having me. Wasn't 2023, like, the most interesting year covering this stuff? For me, definitely. It's been the most interesting one I can remember.

0:50Yeah, I think every year has its own kind of flavor. you know, last year's flavor kind of feels like it was chaos. And I think of it mostly as the Elon Musk taking over Twitter year. This was a year where we saw people start to build a lot of new things. And I think for the first time in a long time in Silicon Valley, there was a sense that something truly new and momentous was gaining momentum. Exactly. And I feel like it had everything. There were business stories. There was like legal stories with the Sam Baikman-Fried story. And then of course, like the big technology moments with AI. Do you have a single moment that stands out for you this past year?

1:30I mean, I think the release of GPT-4 to the public and the beginnings of it proliferating throughout so many different industries, I think will be the moment that we look back on when we think about 2023. Like this was the year that large language models went from pretty good to like often great. And I think the implications of that, you know, we're just going to be dealing with for a long time. One of the interesting moments to me that stands out from this year, and we're going to move on to our predictions as opposed to spending too much time looking back, but it almost is easy to forget that this was like the quote unquote year of efficiency and all these big tech companies ended up laying off so many employees and dealing with some serious business problems at the beginning of the year, only to end the year in better shape financially than they've ever been, more dominant than they've ever been.

2:31A company like Meta up more than 150 % on the year. Even Alphabet, right, up 40%. I was looking like we have so many more trillion-dollar companies now than we started with in 2023. It's kind of an interesting footnote in a year that's been more interesting from a technology standpoint, but definitely something that I'm trying to wrap my head around. Yeah, I mean, I think that the year of efficiency was really like the year of pretext. And the pretext they were looking for was just get rid of a bunch of employees they didn't want around anymore for a lot of reasons. You know, I'm sure that some aspect of it was wanting to be more efficient.

3:11But I think for the most part, they really, really just wanted to get rid of a bunch of divisions and do some reorganizing. Do you think it worked for them? I mean, I don't know. You know, like I don't get too far into the bowels of company org charts for the most part. I think the proof is a culture thing, though, right? It's like, did it revitalize? I mean, that's what people said that sparked their culture. And it was like very lazy before. So just not from a broader standpoint. I mean, I don't know. I think that most of the people who got laid off in the past year do not feel like they were lazy at their jobs.

3:48And, you know, there's a lot of just kind of individual cases. And I think it's hard to speak to generally. But we do know that these companies have been wanting to get rid of, you know, whole divisions for a while. And this year they finally got their excuse. Okay, let's get to some of these predictions because you've written a handful of fascinating predictions in Platformer, which I'm a happy subscriber to. And I think we should talk about them. So one of the first ones that really caught my eye is that you say that Google mostly catches up to open AI in large language model quality and begins to neutralize GPT's lead.

4:23Effectively saying all this marketing shenanigans that we saw with Gemini. Don't read too much into that. the Google LLM is going to be on par with OpenAI's and it's going to be productized. Can you expand on that a bit? Yeah, I think for the past year, OpenAI has had this lead that has felt kind of, I don't know about insurmountable, but I just think they've been unquestionably the leader. And whenever anyone releases a new LLM, OpenAI is always the benchmark. And so Google comes out at the end of this year and they say, here's our new LLM and here are the benchmarks. and according to us, it's better than GPT-4 in a lot of, excuse me, in a lot of cases, not every case, but it's like better in a lot of cases.

5:10And I think that we should basically take them at their word. I mean, sure, let's wait and see what the thing looks like when it actually comes out. And it's easy to say this when you don't have to actually have to release the product. But I just think that Google has generally been pretty conservative in the way that it talks about its AI. I don't see them as people who've been doing a lot of hyping and hyperbole. So I'm just sort of inclined to believe them when they say, trust us, this thing is going to hit the benchmarks it needs to. But I think the bigger issue is that after that happens, these models are all roughly going to be at parity.

5:44And then it's no longer a pure technology game. It's no longer about who hit which benchmark, who's 3 % better at which test. it's all about product design and distribution, you know? And I think Google's in a really good position to just put its generative AI in front of a billion plus people. And I think OpenAI is too through Microsoft. And I think they're just gonna kind of fight it out there. Like obviously they're gonna continue to research these next generation models and there might be some big leaps ahead and big breakthroughs still to come that'll kind of reset those dynamics. But I see 2024 as a year where these two get kind of closer together in terms of quality and they have to find new ways to compete.

6:26Right, and that open AI, I've argued, is in the hits business, right? Which is kind of going along the way of what you're saying. Do you think they have another big hit in them? I mean, Sam Altman, you know, spoke with you guys about their research, was talking about how he recently saw like the veil of discovery pulled back. I mean, all this stuff around him getting fired, you know, there was a hint of maybe this was like in the wake of a big breakthrough. Do you think we see that next year? um i mean well first of all i think i would disagree that they're in a hits business why do you think they're in a hits business well if everything becomes commoditized then you have to i mean i'm not saying they go away if they can't make hits but they sort of are in a place where they're gonna get caught up to by the bigger players very quickly i mean thanks to their partnership with microsoft i think they just kind of are one of the big players.

7:17I think they have a really powerful API that is being used by a lot of different developers. And I think that API is going to see increasing use for the next year. I mean, the story with them so far is basically that they can't keep up with demand. Maybe sometime in the next year, demand levels out. But I think they have a lot of just kind of very basic stuff they can do to make their API more available, to continue to fine tune it, to continue to explore new applications for it. And I just think that gives them a lot of runway through the end of next year. You know, when I think hits based business, I think of like video games, you know, it's like you spend like 10 years making Elden Ring and you hope people like it.

7:52You know, OpenAI has built some pretty powerful underlying technology that I think they can kind of continue to refine and like put into new shapes and new products. And so I think that's going to set them up for success. Okay. Yeah. You know, I think they'll be successful. And the question is like to be the leader, right? What's it going to take? That's really, I think you're right to that. Maybe the hits business is too much into this like Hollywood video games motif, which might not be the proper way to compare it. You think we're going to get to AGI this year? You mean by the end of 2023? Let's go big.

8:28No, I think we have a few more weeks. No, probably not. I mean, I think OpenAI, one of the things I predicted for next year is that OpenAI will begin to train GPT-5. It seems like it's been about a couple years in between models for them. If they start it toward the end of next year, it'll still be a little early, which is why it's not my highest confidence prediction. But I think that after the whole Sam Altman drama, they are going to feel the pressure to show that they are still the leader. And I think that's going to lead them to push really hard in seeing whether they can get their next model into training.

9:08Nobody who I have spoken to thinks that GPT-5 is going to be AGI. And in fact, there are some technological hurdles that they still have to figure out to decide, like, what is GPT-5 even going to be and how do we get there? But, you know, this is part of the unknown that we are in. if you just continue pouring more compute into these models and just try to keep making them more general and feed them more data, do they become omniscient? And I think the answer is probably no, but ask me again at GPT-7. Right. So do you have a prediction as to how close we are to artificial general intelligence?

9:48No, I have no idea. I only pretend to be a smart person on podcasts. I don't actually know when they're going to get there. Okay, let me do it this way. Over or under 20 years? I'll go first. I'm going over. Let me be really annoying. I don't think we're close to it. Well, let me be really annoying and say, how do you define AGI? That is basically artificial intelligence that's on par with human level intelligence that can generalize. Well, some would say that we already have that today. Some would say that we already have that today. Some would say we've already hit that bar. You know, I think if you want to talk about something that is like some kind of software that is better than at every single human at everything humans can do, that's basically what I think of as AGI.

10:35And I to me, the odds that we get there in under 20 years are higher than 50 percent. 20 years is a long time. Do you look at the technology you were using in 2003? You didn't even have like an iPod Nano back then, right? So a lot can change in 20 years. And just look at the progress we've made with AI in the past four or five years. And it's clear that the rate of change is accelerating. So everyone on the subject always says, look, we could hit some ditch. We could hit some cliff where we can't actually figure out how to get to the next place from where we are. and I think a lot of people would actually be really excited if that happened, including people who work in AI.

11:21But it is also possible that we don't really hit that ditch or like to the extent that we hit that ditch, we figure out how to get out of it within like a few months or a year. So there was around the time of Sam Altman being fired and then rehired, there was these stories about this Qstar model that apparently was able to reason. What type of stock do you put into that? I mean, we're talking about path toward artificial intelligence. Like, what do you think about that model? I mean, is that what we see next year? It's hard to say. And I did a little reporting on it. And I was told, like, to the extent that that was happening at OpenAI, like, it's not something that had come to the board's attention.

11:57I do think they're working on experimental models all the time. It would not be surprising to me to learn that they had built something that had shown some kind of new capability that they got really excited about or they got really concerned about. but you know, we, we don't know for sure. I would like to know more about that subject than I do. The big thing that Qstar was supposed to be able to do was to be really good at math. And so far the LLMs that have been devised are really bad at math. Cause it's just, they're not really designed to understand it. So if an LLM got really good at math, then yeah, that would be something we'd want to pay attention to.

12:33Okay. One more follow-up. So you mentioned one of your predictions is GPT-5 starts training next year. It's always been surprising to me that they don't just start training the next GPT after they released their current GPT. What does GPT-5 look like that's different than GPT-4 and why haven't they started yet? So, I mean, I'm going to feel a little bit out of my depth here, but you think about what a large language model is. The first thing you do is you get a whole lot of data and then you run a process against that data that creates this model that is then able to make predictions based on various inputs and then you sort of go in and you fine-tune the model and like you build guardrails and my understanding is that like this is uh this is just a sort of massive project that you, you don't just do it over and over again.

13:35It's phenomenally expensive to do this, right? So it's like, yeah, you gotta go get a lot. You gotta raise a lot of capital to undertake this, this process, right? And I think GPT-4 was, was their best idea at the time for, for how they could do all of that. and I think in the years since they have put it out, they have not devised all of the next steps. Where are we going to get the rest of the data? How are we going to reach the sort of next, if the current model scored 86 % on the bar exam, what do you need to do so that the next one scores 100 % on the bar exam? And I think there's just like sort of a million questions to answer along the way.

14:20So it's not like you release Microsoft Word 1.0 and then you have a meeting that says like, what should be in Microsoft Word 2.0? It's like, we've completed this massive science project and solved a lot of problems. And in some amount of time, we will start to think about, okay, let's plan another massive science project. Casey, that was an excellent explanation. Thank you. If any of your listeners actually know anything about AI, I think that answer probably drove them insane, but forgive me. It's been a long year. No, I think it was great. So let's talk a little bit about AI created content now, which is going to be something that we really started to reckon with this year and is going to just explode next year.

15:04So one of your predictions is that the quality of search results degrades as Google proves unable to reliably detect AI generated content. Now, I have a question for you about this one, because if so, Google's aren't isn't the search engine that Google built all about basically identifying what a quality result is based off of the way that the rest of the Internet sort of relates to it. So, you know, if you get links in from from high quality pages, then your then your your page goes up in the results. And my thought on this is if the AI generated pages are good enough to rank high within Google search results, then are they going to actually be low quality?

15:47Like, should we really care all that much? And I'd love to hear you expand upon why you think that's the case. I mean, I think that framing sort of takes it for granted that Google search results are pretty good today. Most of the people I know feel like Google search results used to be better and that the results that you find at the top of a lot of Google search result pages today are not necessarily the best. They're people who paid the SEO consultants to game the system as hard as they possibly could so that they would show up at the top of results. And that is one reason why I'm not as inclined to believe that Google is prepared for what is coming for them.

16:32I think they have been able to get away with not actually caring all that much about what the quality of search results has been. They effectively have a monopoly. Almost nobody uses Bing, right? And so what they have is a product that is just basically good enough. They have a problem. You know, it is good enough that people don't lose their minds and not too many people try to build an alternative. But once they get flooded with this crap that they're already being flooded with, you know, you're already seeing mistakes. There have been some interesting and high profile ones this year. Well, then I think, you know, we could be in a little bit more trouble here because I think they don't have the ability to assess what was written by AI and what wasn't.

17:16The SEO demons are going to ensure that a lot of bad stuff rises to the top. And I'm just not all that convinced that Google is gonna have the tools to stop it. So, you know, we'll have to see. Would love to be wrong here. But in my experience, I'm just not as convinced that they're going to be able to figure it out. That's a great answer. So where do you stand on this idea that maybe generative AI responses and chatbots could replace the search? So I think it's going to happen. You know, one of the best things that I did for myself this year was I started using this app called Raycast, which is a kind of launcher application.

18:00so I can launch different programs from it via my little keyboard here. And I pay to upgrade Ratecast so I can type in any question that I have to, you know, about anything. Most recent thing I did this for, by the way, I was watching Jeff Bezos on the Lex Friedman podcast and I was like, how old is Jeff Bezos? And so I just hit command tab and I said, how old is Jeff Bezos? And I hit tab again and then it accessed GPT-4 to tell me that Jeff Bezos is 59 years old. And that was, you know, useful information. Why is any of that interesting? Well, I didn't use Google to get there. I just asked ChatGPT.

18:34Now, ChatGPT might have had to do a little web searching to figure that out. In fact, I think it did in this case, but I didn't go to Google. And according to my little Raycast end of year results, I've done this hundreds of times this year, that things that I would have used to do Google searches for, I now just ask the LLM directly. Now, maybe you're saying, Casey, isn't that super dangerous? Like, don't these models hallucinate? Like, aren't you sort of exposing yourself to a lot of bad information? and the answer might be maybe, but I only use this stuff for things where I don't care too much if the answer is wrong.

19:06And it turns out that is most of the things that you ask a search engine. You know, we always talk about search like, you know, every search is life or death. A lot of search is like, hey, what movies was Adam Sandler in? And like what you basically want is a list of eight movies. And if one of the movies is wrong, it doesn't matter. You basically got what you needed, right? And this is what the LLMs are actually pretty good already at doing is giving you a good enough answer. And so now for the first time in my adult life, for the first time, like basically since web search existed, I'm now using a Google alternative all of the time.

19:42And I'm here to tell you it's fine. I get an answer directly to my question. I do not need to see 15 blue pages of links. I just get what I was looking for and nothing else. And the user experience is better than Google. The information quality is just as good. And this is an area where I feel like I am like three to five years ahead of most people, but I'm telling you other people are going to get there. And what Google is going to realize is they're going to actually just let the web decline because their future is in building the experience that I'm using today with OpenAI. They're going to build it with Google products and the web, let God sort them out.

20:15So predictions episode. I mean, I've been trying out this Google generative AI, generative search results. Do you think in 2024 that rolls out to everyone? and that's where they just basically answer your search queries as if they're a chatbot with a long text block i think they'll keep pushing i don't know if they'll roll it out everywhere there'll be regulatory concerns you know google because it has this monopoly um people are quite sensitive to what those search results look like they get in trouble all the time for you know what they are and aren't showing in search results and how are they ranking things wall street journal had a great story today where they said that publishers have looked into this and they believe that if the search generative experience rolls out broadly, most publishers will lose between 20 % and 40 % of their traffic.

21:00Why is that? It's because, well, the AI is just scanning the webpage, giving you the answer, and now no one has any reason to click anything. So Google might look at that and say, well, this is the best user experience and we're going to do it. Publishers be damned. Or it might say, it's actually too risky to do this because we don't want to have to go sit in a regulatory hearing in every country in the world. So we're going to create some licensing deals with publishers to maybe allow us to not have to do that. So again, I don't know what's going to happen by the end of 2024, but I do believe that Google will make the sort of bare minimum of licensing deals that it needs to so that it no longer has to care about whether it's sending traffic to publishers.

21:39And now we get to the most depressing of all of your predictions, which is that scaled up AI generated sludge outcompetes many digital media companies for advertising and affiliate link dollars, sparking further waves of job losses and consolidation. And you note the fact that we've lost BuzzFeed News, My Former Employer, Protocol, and Vice have all gone away. And I mean, if we have AI outcompeting, basically, you know, publishers putting these AI stories on their sites and realizing it's making them more money than the human writers, that's a serious problem. I mean, we've already seen a few examples this year with SI and invest.com and it's happened to me where I've been plagiarized with AI.

22:21How prevalent do you think this is going to be? Like, I mean, the New York Times recently hired someone whose job it is to do generative AI. Where do you see this going? I mean, so I would put what the New York Times is doing in a very separate category is everything else that you just said. Like basically they hired somebody to figure out like, is there any useful thing that we could do with generative AI? I think that's actually a fair and good question for newsrooms to ask. Everything else? Yes, we are in a lot of trouble here. The reason that I believe this is because when you think about a digital media company like Vox Media, where I used to work, BuzzFeed, where you used to work, their core competency was really in the journalism.

22:57It was in the writing. It was in coming up with creative, entertaining, informing stuff to show people, right? And they're so good at it. And I want them to be able to do it forever. But there's this other dark side of this world. And the core competency that people have is not in the writing. it is in the SEO and in the spamming. And I am just making the bet that the spammers are going to do better in this world than the digital media companies. Because the things that you would have to do to outcompete a spammer, I'm just not sure anybody really has the stomach for it, right? These spammers are going to take just to name one example.

23:36A lot of the internet is just driven by people asking how to do something, right? How do I change a tire? How do I replace my kitchen sink? Okay. And we now probably have good answers to like 95 % plus questions of that variety that anyone would ask. And they're all available for free, like on the internet. And I know that a lot of, you know, some of the publishers that we've been naming in this conversation, they also do their own how-to content and they reap a lot of advertising revenue off it. Often you could put affiliate link revenue in there because, oh, you need to replace a sink. Maybe you need to hire a plumber, click this link and we'll get a little spiff if you book a plumber that way, right?

24:12So that is just powering a huge portion of the good publisher's revenues. But then along come the spammers and they say, we no longer even need to hire humans to do any of this. We can just have an AI, monitor search trends in real times, spin up a new article about this subject every day and sort of promote it as being even fresher than the last thing, and just sort of endlessly win this game. This is their core competency. They do not care about the writing. All they care about is that you click on the site. They don't care if you replace your thing. They don't care if you change your tire. They just want you to click on the thing, right?

24:49And that is the sad thing, is that they are the ones who are in the best position to win here unless Google actually goes in and says, we're going to pick some winners. Or well, even more. I mean, you might not even need these type of websites. If you get these how queries to chatbots and they can walk you through step by step, that whole class of publisher goes away. Exactly. Step one is the spammers win. Step two is Google wins by replacing the spammers. And the distance in between those two steps might not even be that big. But like those are the next two turns of the screw that I see coming.

25:24Yeah. I mean, I know personally that if I need to go for like a how to at this point, like number one is YouTube. But if YouTube doesn't get the job done, I'm going directly to these chatbots. Yeah, sure. Why wouldn't you? I mean, I use them to cook meals now, like give me recipes I can make with these ingredients type of thing. Again, it is not perfect. Sometimes they will lead you astray, but it's good enough most of the time that you do just wind up using it. Casey Newton is here with us. He is the editor of Platformer and the co-host of Hard Fork, which you can get in your podcast app of choice.

25:56On the other side of this break, we're going to talk about things that are not AI, but will also be very interesting in 2024. including Apple's Vision Pro, which is going to be released next year. And we're going to ask, how's it going to do? And also, we're going to talk about whether people are going to get into more romantic relationships with AI. And of course, the threads versus Twitter battle. Back right after this. Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points and miles. Now, imagine a credit card with rewards that can grow in value.