8 Predictions for AI in 2024

0:00Today on the AI Breakdown, we're looking at eight predictions for AI in 2024 and assessing how accurate we think they are, and what additional predictions we might have. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our Discord, our YouTube, and our newsletter.

0:24Welcome back to the AI Breakdown. Well, I really liked how yesterday's episode worked, by which I mean taking someone else's content, in that case, their ranked list of the top 10 most important stories in AI in 2023 and reviewing it and using it as a basis for a larger conversation. And so I decided I would do the same looking forward now, since yesterday we looked backward, today we'll look forward. And so we head over to a TechCrunch piece called 8 Predictions for AI in 2024. Just like last time, I will start by reviewing each of these 8 predictions, talking about whether I agree or disagree, and maybe what some larger implications are.

0:59And then to the extent that there is anything missing, we will talk about those at the end as well. The author Devin starts, This last year was a banger for AI as the technology went from niche to mainstream about as fast as anything ever has. 2024, however, will be the year when the hype runs full steam into reality as people reckon with the capabilities and limitations of AI at large. Here are a few ways we think that's going to play out. Prediction 1. OpenAI becomes a product company. After the leadership shakeup in November, OpenAI is going to be a changed company. Perhaps not outwardly, but the trickle-down effect of Sam Altman being more fully in charge will be felt at every level.

1:34And one of the ways we expect that to manifest is in a ship it mindset. We'll see that with the GPT store originally planned for launch in December, but understandably delayed due to the C-suite fracas. The App Store for AI will be pushed hard as the platform to get your AI tools and tools from, and never mind Hugging Face or any open source models. They have an excellent model to work from, Apple's, and will follow it all the way to the bank. Expect more moves like that from 2024's OpenAI, as the caution and academic reserve that the previous board exerted gives way to an unseemly lust for markets and customers.

2:04Other major companies with AI efforts will also follow this trend. For instance, expect Gemini and Bard to horn in on a ton of Google products, but I suspect it will be more pronounced in this case. This one is super interesting. On the one hand, I think it's pretty undeniable that there is going to be more pressure than ever in this AI arms race. The more that there becomes a tight band around GPT-4 level LLMs and state-of-the-art image generators and all these sort of things, the more all these labs will be scrapping and crawling for any type of advantage. Now, I think that if OpenAI jumps out ahead again with 4.5 or even 5, that will put even more pressure on the Googles of the world to catch up.

2:42But when it comes to OpenAI itself, I'm not totally sure about this idea of Sam Altman being more fully in charge. On the one hand, it is obviously true in some ways, right? Sam Altman was fired and a rebellion of the company led him to come back in. It's hard for him not to be more powerful in some ways because of that. We saw that OpenAI is Sam Altman, and Sam Altman is OpenAI. At the same time, he didn't fully win, did he? He's a little bit neutered in the context of the board. Adam D 'Angelo was able to stick around. They just instituted a new policy where the board explicitly has the ability to reverse decisions from the executives when it comes to safety questions.

3:18And that in particular could create some tension around exactly this sort of theory. It is going to be super interesting to see one way or another. Next up, agents, generated video, and generated music graduate from quaint to experimental. Some niche applications of AI models will grow beyond, eh, status in 2024, including agent-based models and generative multimedia. If AI is going to help you do more than summarize or make lists of things, it'll need access to things like your spreadsheets, ticket-buying interfaces, transportation apps, and so on. 2023 saw a few tentative attempts at this agent approach, but none really caught on.

3:52We don't really expect any to really take off in 2024 either, but agent-based models will show their stuff a little more convincingly than they did last year, and a few clutch use cases will show up for famously tedious processes like submitting insurance claims. Video and audio will also find niches where their shortcomings aren't quite so visible. In the hands of skilled creators, a lack of photorealism isn't a problem, and we'll see AI video used in fun and interesting ways. Likewise, generative music models will likely make it into a few major productions like games, again where professional musicians can leverage the tools to create an unending soundtrack.

4:22So this is two fairly different categories of things. but a decent combination, so I'll take them each in turn. I actually very much share some of these sentiments around agents. Obviously, if you've watched my content for any period of time this year, you will know that AI agents, AutoGPT, Baby AGI, all these things have been such a hot area of developer interest throughout the year, but haven't bridged into something actually all that useful. Lord knows Multion and other companies are trying, but I tend to think, which it seems like TechCrunch is saying as well, that the application is not going to be general sort of AI individual personal assistants, and instead highly specific use cases.

5:00Agents that deal with very specific processes, probably inside dense workflows, where we really first get value from this new world of AI agents. I am quite bearish in general, actually, on AI personal assistants. I think that they're going to cause more problems than they solve, and most people aren't going to be interested. But I think that specific use cases designed in the context of particular professions or roles, Whole different story. I even kind of agree that I don't necessarily see any fully taking off next year, but I think we're going to see the first hits, to use baseball analogies, singles and doubles, in those specific use case areas in ways that could pick up steam really quickly.

5:35Now, on video and music, I think it's super clear that generative video is going to have a big year next year. Between Pika and Runway, those tools are getting so much more advanced, so much easier to use, and already people are doing fairly incredible things with them. That said, to me, the biggest question is, if these tools are effectively democratizing access to video creation, who are the new people on the margins who come in and start creating stuff because they can? Image creation has a lot more use cases for normies than video creation does. So will it just be existing content creators who are using generative video?

6:09Or will the availability of generative video actually create its own new demand? When it comes to music, we've seen leaps and bounds this year. but it feels unlikely to me that people just generating full songs is the actual use case, and so I'm not sure exactly where we'll find it even though the models themselves will definitely have huge upgrades next year. Still, I wouldn't be surprised if we see a lot more experimentation, maybe the first hit singles that involve AI in some way. I just don't know exactly where in the creative process it's going to fit. Next on the list, the limits of monolithic LLMs become clearer.

6:42So far, there has been great optimism about the capabilities of large language models, which have indeed proved more capable than anyone expected, and have grown correspondingly more so as more compute is added. But 2024 will be the year that something gives. Where exactly, it is impossible to predict, as research is active at the frontiers of this field. The seemingly magical emergent capabilities of LLMs will be better studied and understood in 2024, and things like their inability to multiply large numbers will make more sense. In parallel, we will begin to see diminishing returns on parameter counts, to the point where training a 500 billion parameter model may technically produce better results, but the compute required to do so could provably be deployed more effectively.

7:19A single monolithic model is unwieldy and expensive, while a mixture of experts, a collection of smaller, more specific models and likely multimodal ones, may prove almost as effective while being much easier to update piecemeal. Now, here is one where I will insert something that is not in this list that I think is going to be a major theme, which is a transformation in our understanding of models' capacity, particularly small model capacity, because of the open source field. Already, we are seeing some of the parts of what they're talking about here. People are now speculating, given that it's been nine months with no one being able to beat GPT-4 as the state of the art, that maybe we're running up against some serious actual limits.

7:54Maybe we're running up against these diminishing returns, in other words. In the meantime, smaller models than ever, from everyone from Microsoft to Mistral, are transforming what we thought was possible and pushing to an area where models could be deployed in much different contexts, such as on-device. So it's not that I disagree with this on a fundamental level, it's just that I think that simultaneous to this happening, there is going to be an absolute flourishing and Cambrian explosion of alternative approaches that yield some pretty incredible results. Next on the list is marketing meets reality.

8:22The simple fact is that the hype buildup in 2023 is going to be very hard for companies to follow through on. Marketing claims made for machine learning systems that companies adopted in order to not fall behind will receive their quarterly and yearly reviews, and it's very likely they will be found wanting. Expect a considerable customer withdrawal from AI tools as the benefits fail to justify the costs and risks. On the far end of the spectrum, we are likely to see lawsuits and regulatory action with AI service providers that fail to back up their claims. While capabilities will continue to grow in advance, 2023's products will not all survive by a long shot, and there will be a round of consolidation as the wobblier riders of the wave fall and are consumed.

8:57Lots of different pieces to pull on with this one. One, my firmest agreement is with that last piece that 2023's products will not I'll survive by a long shot and that there will be a round of consolidation. All it takes is going through any of the various tool aggregators to see how many repeat versions of specific use case tools that can't possibly have enough of a market to really justify themselves. Either the best among those is going to rise up and become the basis for a larger suite of things, or they're going to get subsumed into another company, either by acquisition or just by a repeat of functionality.

9:27So yes, there will be massive consolidation. There's no way for that not to be the case. Now, when it comes to this idea that there will be a considerable customer withdrawal from AI tools, I'm not so sure, and here's why. It's not that I disagree that the integration of these tools into the workplace is going to be a lot harder than just understanding that they're going to be big, important things. It's more that I think that adoption has already been hampered by those same factors. I think that companies who are insecure about their data policies or don't feel that they have their heads around the risk have, by and large, not opted to actually dig very deep in this space.

9:59Accenture, for example, has seen increasing AI-related bookings, but they're still not up to a billion dollars a year. In other words, while yes, there may be some customer withdrawal, I think that it'll be vastly outstripped by the number of new companies that are finally actually starting to wade their feet in, and perhaps doing so with more realistic expectations. Now, speaking of wading in, the next prediction from TechCrunch is that Apple jumps in. They write, Apple has an established pattern of waiting, watching, and learning from other companies' failures, then blowing in with a refined and polished take that puts others to shame.

10:31The timing is right for Apple to do this in AI, not just because if it waits too long, its competition may eat up the market, but because the tech is ripe for their kind of improvement. I would expect an AI that focuses on practical applications of users' own data, using Apple's increasingly central position in their lives to integrate the many signals and ecosystems the company is privy to. There will likely also be a clever and elegant way to handle problematic or dangerous prompts, and although it will almost certainly have multimodal understanding, primarily to handle user images, I imagine they'll totally skip media generation.

10:59Expect some narrowly tailored but impressive agent capabilities as well. Siri, get a table for four at a sushi place downtown around 7 and book a car to take us sort of thing. What's hard to say is whether they will bill it as an improved Siri or as a whole new service. Apple AI with a name you can choose yourself. They may feel the old brand is freighted with years of being comparatively incapable. But millions already say, hey Siri, every 10 seconds, so it's more likely they'll opt to keep that momentum. Alright, one, definitely agree Apple's going to get in next year. To me, the M3 Max chip, which just came out in the new MacBook Pros, makes that super clear.

11:32They are racing towards a future in which they can run LLMs on device with the same sort of consideration of data privacy and on-premise rather than in the cloud that they have across all their other products as well. Now, what they use that for might be highly specific in the beginning. We're seeing a little bit of that already with the way that they are integrating AI-ish features into future iterations of their health software on Apple Watches. And I think that everyone believes that an upgraded Siri is an obvious place for them to start as well. Now again, I have real skepticism around the AI personal assistant use case.

12:04However, to the extent that people are already using Siri for similar things, this is an area where I could see that being incorrect, at least for a portion of the public. As to whether keep or retire the Siri brand, I think they'll keep it if only because Apple AI, whatever that becomes, will be much more comprehensive and much bigger than just that one feature. Next on TechCrunch's list, legal cases build and break. We saw a fair number of lawsuits filed in 2023, but few saw any real movement, let alone success. Most suits over copyright and other missteps in the AI industry are still pending.

12:342024 will see a lot of them fall by the wayside as companies stonewall critical information like training data and methods, making allegations like the use of thousands of copyrighted books difficult to prove in court. This was only the beginning, however, and many of these lawsuits were filed essentially on principle. Though they may not succeed, they may crack the process open far enough during testimony and discovery that companies would rather settle than have certain information come to light. 2024 will bring new lawsuits as well, ones pertaining to misuse and abuse of AI such as wrongful termination, bias in hiring and lending, and other areas where AI is being put to work without a lot of thought.

13:06But while a few egregious examples of misuse will be punished, a lack of relevant law specific to it means that it will necessarily only haphazardly brought to court. Hard agree that we will see many more legal actions. And in fact, I think in some ways it'll be a race to see how much new policy and new laws determine the shape of the rules around AI versus court decisions. I would not at all be surprised to see a copyright related case make it all the way to the Supreme Court. In fact, I'm kind of expecting it and I kind of think it has to. The reality is it's too complex and there's going to be too many varying opinions for it not to make it all the way up to the highest court in the land.

13:40And what they decide, by the way, will be hugely impactful to how the AI field develops, one way or another. The second-to-last prediction, early adopters take new rules by the horns. Big moves like the EU's AI Act could change how the industry works, but they tend to be slow to take effect. That's by design, so companies don't have to adjust to new rules overnight. But it also means we won't see the effect of these big laws for a good while, except among those willing to make changes preemptively and voluntarily. There will be a lot of we-are-beginning-the-process-of talk. Also expect a few quiet lawsuits challenging various parts of laws.

14:11To that end, we can expect a newly flourishing AI compliance industry, as the billions going into the technology prompt matching investments, at a smaller scale but still considerable, and making sure the tools and processes meet international and local standards. Unfortunately for anyone hoping for substantive federal regulation in the US, 2024 is not the year to expect movement on that front. Though it will be a year for AI and everyone will be asking for new laws, the US government and electorate will be too busy with the trash fire that will be the 2024 election. Two things here. One, agree, I don't think it's likely that we will see comprehensive AI legislation next year.

14:43I don't think it's likely that we'll see comprehensive anything legislation next year. The only thing that could change that is some major crazy event, some unanticipated technological advance that is so serious and so seemingly serious in the eyes of the public that everyone has to pause re-election and all the battles therein to get together and make some rules around this. Count me as skeptical on that front. Now, when it comes to the EUAI Act, I don't know, man. I ascribe good faith and good intentions to many of the people involved in that rulemaking process. And remember, it's gone on a lot longer than just the generative AI boom of the last year.

15:16But I think unfortunately, Europe is probably putting itself in a position where the two years between now and when it's fully implemented will be two years where most companies get themselves out of Europe. That is, of course, unless other jurisdictions all follow suit, which is why it's going to be important for the Europeans to convince their US counterparts and regulators elsewhere to implement similar rules. Otherwise, Europe will basically just be cutting itself off of the industry. Last prediction, the 2024 election is a trash fire and AI makes it worse. How the 2024 presidential election will play out is really anyone's guess right now.

15:48Too many things are up in the air to make any real predictions except that, as before, the influence mongers will use every tool in the box to move the needle, including AI in whatever form is convenient. For instance, expect bot accounts and fake blogs to sprout generated nonsense 24-7. A few people working full-time with the text and image generator can cover a lot of ground, generating hundreds of social media and blog posts with totally fabricated images and news. Flooding the zone has always been an effective tactic, and now AI acts as a labor multiplier, allowing more voluminous yet also targeted campaigns.

16:17Expect both false positives and false negatives in a concerted effort to confuse the narrative and make people distrust everything they see and read. That's a win state for those politicians who thrive in chaos. Organizations will tout AI-powered analysis to back up purges of voter rolls, challenges to vote courts, and other efforts to suppress and interfere with existing processes. Generated video and audio will join the fray, and though neither are perfect, they're good enough to be believable given a bit of fuzzing. The clip doesn't have to be perfect because it will be presented as a grainy zoomed-in cell phone capture in a dark room, or a hot mic at a private event, or what have you.

16:47Then it becomes a matter of who are you going to believe, me or him, and that's all some people need. Likely there will be some half-hearted efforts to block generated content from being used in this way, but these posts can't be taken down fast enough by the likes of Meta and Google, and the idea that X can or will effectively monitor and take down such content is implausible. It's going to be a bad time. I think that it's very likely that all of these things will be tried. The stakes are too high for them not to be tried. We're already seeing AI find its way into the elections right now. The big question to me, which will be hugely reflective of where the general public is, is whether any of us believe anything.

17:23Will we so have internalized the idea that we simply don't believe anything that we see or hear. And if that is the case, how much does that make things better or worse? Certainly better in the sense that people won't be as tricked, but worse in the sense that also real information won't make it through the filter either. It will be a very live action test, a chance to see where some of the biggest concerns that people have actually fall, and although it will be chaotic, it will also certainly be interesting. Overall, I think this is a good list, especially if you view it as implicitly a non-technical list that's not getting too deep into speculation around actual scientific or research advances.

17:58I'm really interested to see what you guys think. Come join us in the Discord. Come have a conversation about it. All that information is, of course, at breakdown.network. What are your big predictions for next year? Can't wait to see what happens and be here to chat about it with you all the entire time. Until next time, peace.

18:24You