Beyond Chatbots: Marc Andreessen and Ben Horowitz on AI's Future

0:00I think we don't yet know the shape and form of the ultimate products. It's one just obvious historical analogy is, you know, the personal computer from sort of invention in 1975 through to, you know, basically 1992 was a text prompt system. 17 years in, you know, the whole industry took a left turn into GUIs and never looked back. And then, by the way, you know, five years after that, the industry took a left turn into web browsers and never looked back, right? And, you know, look, I'm sure there will be chatbots 20 years from now, but I'm pretty confident that both the current chatbot companies and many new companies are going to figure out many kinds of user experiences that are radically different that we don't even know yet.

0:32Every major technology shift brings new capabilities, new pressures, and new questions about how progress unfolds. At A16Z's Runtime Conference, I sat down with Marc Andreessen and Ben Horowitz to discuss the current state of AI, how reasoning and creativity are evolving, how markets adjust to new technology, and what this moment means for founders and institutions shaping what comes next. Now, to Marc and Ben. Please join me in welcoming Mark Andreessen and Ben Horowitz with general partner Eric Tornberg. Follow me into a solo, get in the flow, and you can picture like a photo. Music makes mellow, maintains to make melodies for MCs, motivates to point some everlasting.

1:14Thank you for the rock Kim who did that. Ben picked the music. Mark, there's been a lot of talk lately about the limitations of LLMs, that they can't do true invention. of, say, new science, that they can't do true creative genius, that it's just combining or packaging. You have thoughts here. What's a you? Yeah, so for me, it's, yeah, so you get all these questions, and yeah, they usually come in either sort of, are language models intelligent in the sense of, can they actually process information and have sort of conceptual breakthroughs the way that people can? And then there's our language models, our video models creative.

1:47Can they create new art, actually have genuine creative breakthroughs? And of course, my answer to both of those is, well, can people do those things? And I think there's two questions there, which is, okay, even if some people are quote unquote intelligent as in having original conceptual breakthroughs and not just, let's just say regurgitating the training set or following scripts, what percentage of people can actually do that? I've only met a few. Some of them are here in the room, but not that many. Most people never do. And then creativity. I mean, how many people are actually genuinely creative, right?

2:11And so you kind of point to a Beethoven or a Van Gogh or something like that. You're like, okay, that's creativity. And yeah, that's creativity. And then how many Beethovens and Van Goghs are there? Obviously not very many. So one is just like, okay, like if these things clear the bar of 99.99 % of humanity, then that's pretty interesting just in and of itself. But then you dig into it further and you're like, okay, like how many actual real conceptual breakthroughs have there ever been actually ever in human history as compared to sort of remixing ideas? If you look at the history of technology, it's almost always the case that the big breakthroughs are the result of usually at least 40 years of sort of work ahead of time, four decades.

2:45Right. In fact, language models themselves are the culmination of eight decades, of previous work. And so there's remixing. And then in the arts, it's the exact same thing. Novels and music and everything. There are clearly creative leaps, but there's just tremendous amounts of influence from people who came before. And even if you think about somebody with the creativity of Beethoven, there was a lot of Beethoven in Mozart and Haydn and in the composers that came before. And so there's just tremendous amounts of remixing and combination. And so it's a little bit of an angel's dancing on the head of a pin question, which is like, if you can get within 0.01 % of kind of world-beating generational creativity and intelligence, and it's like, you're probably all the way there.

3:19So emotionally, I want to like hold out hope that there is still something special about human creativity. And I certainly believe that and I very much want to believe that. But I don't know, when I use these things, I'm like, wow, they seem to be awfully smart and awfully creative. So I'm pretty convinced that they're going to clear the bar. Yeah, I think that seems to be a common theme in your analysis when people talk about the limitations of LMs. Can they do transfer learning, just learning in general? You seem to ask, can people do this? Yes, can people do these things? Well, it's like lateral thinking, right?

3:44So yeah, so it's like reasoning in or out of distribution, right? And so it's okay. I know a lot of people who are very good at reasoning inside distribution. How many people do I actually know who are good at reasoning outside of distribution and doing transfer learning? And the answer is like, I know a handful. I know a few people where whenever you ask them a question, you get an extremely original answer. And usually that answer involves bringing in some idea from some adjacent space and basically being able to bridge domains. And so you'll ask them a question about, I don't know, finance, and they'll bring you an answer from psychology.

4:09Or you ask them a question about psychology, and they'll bring you an answer from biology, right, or whatever it is. And so I know, I don't know, sitting here today, probably three. I probably know three people who can do that real widely. I've got 10 ,000 in my address book. And so three out of 10 ,000 is not that high a percentage. By the way, I find this very encouraging. Yeah, immediately the mood in the room has gone completely to hell. I find this very encouraging because look at what humanity has been able to build, right, despite all of our limitations, right? I look at all the creativity that we've been able to exhibit and all the amazing art and all the amazing movies and all the amazing novels and all the amazing technical inventions and scientific breakthroughs.

4:42And so we've been able to do everything we've been able to do with the limitations that we have. And so I think that, do you need to get to the thing where you are 100 % positive that's actually doing original thinking? I don't think so. I think it'd be great if you did, and I think ultimately we'll probably conclude that that's what's happening. But it's not even necessary for just tremendous amounts of improvement. Ben, we were just celebrating some hip-hop legends at your paid-in-full event last week, and so you think a lot about creative genius. How do you think about this question? Yeah, I mean, I think that I agree with Mark that it's, whatever it is, it's very useful, even if it isn't all the way that level.

5:15I think that there's something about the actual real-time human experience that humans are very into, at least in art, where with the current state of the technology, the pre-training doesn't have quite the right data to get to what you really want to see. But it's pretty good. One of Ben's nonprofit activities is something called the Pain and Fall Foundation, which is honoring and actually providing essentially a pension for sort of the great innovators in rap and hip hop. And so he knows and has many of, we were just at the event, and he has many of the kind of leading lights of that field for the last 50 years perform, and it's really fun to meet them and talk to them.

5:56But like how many people in that entire field over the course of the last 50 years would you classify as like a true conceptual innovator? Yeah, well, you know, it's interesting. It depends how broadly you define it, but there were several of them there on Saturday. So Rakim, I think, yeah, Rakim, you'd certainly put in that category. Dr. Dre, you'd certainly put in that category. George Clinton, you'd certainly put in that category. In a narrower sense, like Cool G Rap certainly had a new idea. But, you know, it depends. Like a fundamental kind of musical breakthrough, you'd probably just say Rakim and George Clinton.

6:33Are they excited? So two out of? Well, I mean, those are the guys who were there. Oh, yeah, yeah. Yeah, but yeah, it's a tiny percentage. Tiny, tiny, tiny, tiny, tiny. We had the fireside last night with Jared Leto. He was talking about how many people in Hollywood are really scared or against what's happening here. What do you see in, you know, when you talk to the Dr. J's, the Nas, the Kanye's, are they excited? Are they using it? So everybody who I speak to, there are definitely people who are scared in music, but there are a lot of people who are very, very interested in it. And particularly the hip hop guys are interested because it's almost like a replay of what they did, right?

7:09They just took other music and they kind of built new music out of it. And I think that AI is a fantastic creative tool for them. It like way opens up the palette. And then for a lot of what hip hop is, is it's kind of telling a very specific story of a specific time and place, which having intimate knowledge and being trained just on that thing is actually an advantage as opposed to being like a generally smart music model. At the same time, people also use the same logic of, hey, whatever is more intelligent will rule whatever is less intelligent. And Mark, you recently - Not said by anybody who owns a cat.

7:49Mark, you recently tweeted, a supreme shape rotator can only rotate shapes, but a supreme word cell can rotate shape rotators. And also, someone's clapping here. And also, high IQ experts work for mid-IQ generalists. What means? What means? Yeah, so the PhDs all work for MBAs, right? So it's, okay. So, yeah, well, I just take it up a level. It's just like, when you look at the world today, do you think we're being ruled by the smart ones? Right? Is that your big conclusion from like current events, current affairs, right? Okay, we put the geniuses in charge. You mean Kamala and Trump aren't the best?

8:25Well, let's not even be specific towards the U.S. Let's just look all over the world. Yeah, and so I think two things are true. One is we probably all kind of underrate the importance of intelligence. And actually there's a whole kind of backstory here of intelligence actually turns out to be this like incredibly inflammatory kind of topic for lots of reasons over the last hundred years, which we could talk about in great detail. And even the just very idea that like some people are smarter than other people, it just like really freaks people out and people don't like to talk about it. We really struggle with that as a society.

8:50And so, and then it is true that intelligence is like, in humans, intelligence is correlated to almost every kind of positive life outcome, right? And so intelligence, generally in the social sciences, what they'll tell you is what they call fluid intelligence. The G factor or IQ is sort of 0.4 correlated to basically everything. And so it has 0.4 correlation to like educational outcomes and professional outcomes and income. And by the way, also like life satisfaction. And by the way, nonviolence, being able to solve problems without physical violence and so forth. And so like on the one hand, like we probably all underrate intelligence.

9:21On the other hand, the people who are in the fields that involve intelligence probably overrate intelligence. And you might even coin a term like maybe like intelligence supremacist or something like that, where it's just like, oh, like intelligence is very important. And so therefore, maybe it's like the most important thing or the only thing. But then you look at reality and you're like, okay, that's clearly not the case. Yeah, it's still only 0.4, right? Yeah. Well, so to start with, it's only 0.4. And, you know, in the social sciences, 0.4 is a giant correlation factor, right? Like most things where you can correlate, whether it's, you know, genes or observed behavior or whatever, anything in the social sciences, the correlations are much smaller than that.

9:55So 0.4 is tiny, but it's still only 0.4. So even if you're like a full-on genetic determinist and you're just like, you know, genetic IQ just like drives all these outcomes, like it still doesn't explain, you know, 0.6 of the correlation. And so that leaves it. But that's just on the individual level. Then you just look at the collective level. Well, you just look at the collective level and it's like a famous observation is you take any group of people, you put them in a mob and the mob is dumber, right? Than the average. and you put a bunch of smart people in a mob and they definitely turn dumber.

10:27And you see that all the time, right? And so you put people in groups and they behave very differently. And then you create these questions around like who's in charge, whether who's in charge at a company or who's in charge of a country. And like, whatever the filtration process, it's clearly not, it's not, it's not, it's certainly not only on IQ and it may not even be primarily on IQ. And so therefore it's just like this assumption that you kind of hear in some of the AI circles, which is like inevitably the smart, you know, kind of thing is going to govern the dumb thing. Like, I just think that's like very easily.

10:59It's just sort of very easily and obviously falsified. Like intelligence isn't sufficient. And then you just convey it. You know, we're all in this room lucky enough to know a lot of smart people and you just kind of observe smart people. And like some smart people, you know, really figure out how to have their stuff together and become very successful. And a lot of smart people never do. And so there must be, there obviously are, and there in fact must be many other factors that have to do with success. and have to do with who's in charge than just raw intelligence. It begs the follow-up question of what are some examples of what that might be, skills outside of intelligence, and more specifically, why couldn't AI systems learn them?

11:36Yeah, so Ben, other than intelligence, what, in your experience, determines, for example, success in leadership or in entrepreneurship or in solving complex problems or organizing people? Yeah, there are many things. You know, like a lot of it is, being able to have a confrontation in the correct way. And a bit like there's some intelligence in that, but a lot of it is just under really understanding who you're talking to, you know, being able to interpret everything about how they're thinking about it and just kind of generally seeing decisions through the eyes of the people working in the company, not through your eyes is a skill that, you know, you develop by talking to people all the time, I'm understanding what they're saying, so forth, these kinds of things.

12:23And it's just, you know, it's certainly not an IQ thing. And not that, like I could imagine an AI training on any individual and like figuring it all out and knowing what to say and so forth. But then you also need that integrated with, you know, like whatever the business ought to be doing. So you're not trying to do what's popular. or you're trying to get people to do what's correct, even if they don't like it. And, you know, that's a lot of management. So it's not a problem anybody's working on currently, but maybe they will. Right, some combination of like courage, some combination of motivation, some combination of emotional understanding, theory of mind.

13:11Yeah, you know, what do people want? Like, you know, married to, you know, what needs to be done? And then like, how talented are they? Like, which ones can you afford? Like, if they jump out the window, it's fine. You know, which one's not fine? You know, this kind of thing. There's a lot of like weird subtleties to it. And it's very situational. I think the hardest thing about it and why management books are so bad is because it's situational. You know, like your company, your product, your people, your org chart is very, very different than, And, you know, here are the five steps to building a strategy.

13:48It's like, well, that's the most useless fucking thing I ever read because it has nothing to do with you. So one of the interesting things on this, like the concept of theory of mind is really important, right? So the theory of mind is, can you and your head model what's happening in the person's head, right? And you would think that maybe that, you know, maybe obviously people who are smarter should be better at that. It turns out that that may not be true. And there's a reason to believe that that's not true, which is as follows. So the U.S. military was the early adopter and has continued to be sort of the leading adopter in U.S.

14:18society of actually IQ testing. And they basically launder it through something called the ASVAB, which is their vocational aptitude battery test. But it's basically, it's essentially an IQ test. And so they still use basically explicit IQ tests and they slot people into different specialties and roles, you know, in part with according to IQ, including into leadership roles. And so they know what everybody's IQ is and they kind of organize around that. And one of the things that they've found over the years is if the leader is more than one standard deviation of IQ away from the followers, it's a real problem.

14:52And that's true in both directions, right? If the leader is not smart enough to be able to manage, you know, to be able to, you know, for somebody who is less smart to model the mental behavior of somebody who's more smart is, of course, inherently very challenging and maybe impossible. But it turns out the reverse is also true, which is if the leader is two standard deviations above the norm of the organization that he's running, he also loses theory of mind, right? It's actually very hard for very smart people to model the internal thought processes of even moderately smart people. And so there's actually a real need to have a level of connection there that's not just, and therefore by inference, if you had a person or a machine that had a thousand IQ or something like it, it may just be, it would be so alien.

15:38And its understanding of reality would be so alien to the people or the things that it was managing that it wouldn't even be able to connect in any sort of realistic way. So, again, this is a very good argument that, like, yeah, the world is going to be far from organized by IQ for centuries to come. Yeah, and Zuckerberg had a great line, which is intelligence is not life. And life has a lot of dimensionality to it that is independent of intelligence, I think, that, you know, if you spend all your time working on intelligence, you lose track of that. We sometimes say about some specific people that they're too smart to properly model or, you know, they sort of assume too much rationality on other people or they just overthink things or over-rationalize them.

16:22Yeah, just to your point that it's on everything. Yeah, yeah. People often, people seldom do what's in their best interest, I should say. You know, I also suspect this kind of gets more into the biology side of things. You know, there's more and more scientific evidence that basically also that like human cognition or human, I don't know, whatever you want to call it, self-awareness, information processing, decision-making sort of experience is not purely a brain. Like the basically sort of famous mind-body dualism is just not correct. Like, and again, this is an argument against sort of IQ supremacism or intelligent supremacism is not, you know, human beings didn't experience existence just through the rational thought.

17:02and specifically not through just the rational thought of the brain, but rather it's a whole body experience, right? And there's aspects of our nervous system and there's aspects of everything from our gut biome to smells, to olfactory senses and hormones and like all kinds of like biochemical kind of aspects to life. If you just kind of track the research, I suspect we're going to find as human cognition as a full body experience, much more than people thought. And so therefore to actually, and this is one of the kind of big fundamental challenges in the AI field right now, which is the form of AI that we have working is the fully mind-body dual version of it, which is it's just like a disembodied brain.

17:45The robotics revolution for sure is coming. When that happens, when we put AI in physical objects that move around the world, you're going to be able to get closer to having that kind of integrated, intellectual, physical experience. You're going to have sensors in the robots they're going to be able to gather a lot more real-world data. And so maybe you can start to actually think about synthesizing a more advanced model of cognition. And maybe we're going to actually discover more both about how the human version of that works and also how the machine version of that works. But to me, at least reading the research like that, all those ideas feel very nascent, and we have a lot of work to do to try to figure that out.

18:16Do you have a sense for how they are, how I'm sorry, at Theory of Mind today? Or do you have a sense where the limitations are? You like to talk to them a lot. Are there any particular things that are particularly surprising to you as you do? Yeah, I would say generally they're really good. Yeah, and so I find one of the more fascinating ways to work with language models is actually have them create personas and then basically have them. Well, actually, I like Socratic dialogues. I like when things are argued out and like a Socratic dialogue. And so tell any advanced LLM today to create a Socratic dialogue and it'll either make up the personas, you can tell what it is, it does a good job.

18:52it has this very, very annoying property, which is it wants everybody to be happy. And so it wants all of its personas to agree. And so by default, it will have a briefly interesting discussion and then it will sort of figure out, you know, basically like you're watching, I don't know, PBS special or something. It'll kind of figure out how to bring everybody in agreement and everybody's happy at the end of the discussion. And of course, I fucking hate that. Like it drives me nuts. I don't want that. So instead I tell it, I'm like, make the conversation more tense, right? and like fraught with like anger and like, you know, people, you know, they're going to get like increasingly upset throughout the conversation.

19:26And then it starts to get really interesting. And then I tell it, you know, bring it, you know, introduce a lot more cursing. You know, really have them go at it. Like all the gloves come off. They're going for full, you know, reputational destruction of each other. You do a lot of these skits. Yeah, skits. And then I get carried away. And then I'm like, it turns out they're all like secret ninjas. And then I'll start fighting. And you've got Einstein, you know, you know, hitting, you know, Neil's board with nunchucks. And by the way, it's happy to do that too. So you do have to control yourself.

19:53But it is very good at theory of mind. And then I'll give you another example. There's a startup actually in the UK in the world of politics. And what they've found is that they've found that language models now are good enough. So specifically for politics, which is sort of a subcategory where this idea matters. So, you know, in politics, people do focus groups of voters all the time. And by the way, many businesses also do that. So you get a bunch of people together from different backgrounds in a room and you kind of guide them through discussion and try to get their points of view on things.

20:23And focus groups are often surprising. Like politicians, if you talk to politicians who do focus groups, they're often surprising. They're often surprised by the things that they thought voters cared about is actually not the things that voters care about. And so you can actually learn a lot by doing this. But focus groups are very expensive to run. And then there's a long lag time because they have to be actually physically organized and you have to recruit people and vet people and so forth. And so it turns out that the state-of-the-art models now are good enough at this so they can actually, they can correctly, accurately reproduce a focus group of real people inside the model.

20:54So they're good enough to clear that bar. In other words, you can basically have a focus group actually happening in the model where you create personas in the model and then it actually accurately represents, you know, a college student from, you know, Kentucky is contrasted to a housewife from Tennessee is contrasted to a, you know, whatever, whatever, you just like specify this. And so, you know, they're good enough to clear, they're good enough to clear that bar and, you know, we'll see how far they get. I want to segue to the bubble conversation. Amin and G2, Jensen and Matt spoke about the enormous scale of physical infrastructure being built out.

21:26AI CapEx is 1 % of GDP. How should we understand and think about this bubble question? Well, I think the fact that it's a question means we're not in a bubble. That's the first thing to understand. I mean, a bubble is a psychological phenomenon as much as anything. and in order to get to a bubble, everybody has to believe it's not a bubble. That's sort of the core mechanic of it. And we call that capitulation. Everybody just gives up like, okay, I'm not going to short these stocks anymore. I'm tired of losing all my money. I'm going to go long. And we saw that actually and a little bit of question like really what was the tech bubble?

22:06But in the kind of dot-com era, right as the prices went through the roof, Warren Buffett started investing in tech. So like, and he swore he would never invest in tech because he didn't understand it. And so if he capitulated, nobody was saying it was a bubble when it became like a quote unquote bubble. Now, if you look at that phenomenon, the internet clearly was not a bubble, you know, it was a real thing. It was in the short term, there was a kind of price dislocation that happened because the market, there were just not enough people on the network to make those products go at the time. And then the prices kind of outran the market.

22:51In AI, it's much harder to see that because there's so much demand in the short term. We don't have a demand problem right now. And the idea that we're going to have a demand problem five years from now, to me, seems quite absurd. you know could there be like weird bottlenecks that that appear you know like we just at some point we just don't have enough cooling or something like that you know maybe but like like right now if you look at demand and supply and what's going on and multiples against growth it doesn't look like a bubble at all to me but I don't know do you think it's a bubble mark Yeah, look, I would just say this.