Marc Andreessen and Amjad Masad: English As the New Programming Language

24:17So the neural network would generate, would like generate a list of potential moves. And then you had a more discrete algorithm sort those moves and find the best based on just tree search, based on just trying to verify. Again, this is sort of a verifier in the loop, trying to verify which move might yield the best based on more classical way of doing algorithms. And so that's a resurgence of that movement where we have this amazing generative neural network that is the LLM. And now let's layer on more discrete ways of trying to verify whether it's doing the right thing or not. And let's put that in a training loop.

24:59And once you do that, the LLM will start gaining new capabilities, such as reasoning over math and code and things like that. Exactly right. Okay. And then that's great. And then the key thing there, though, for our elder work, for LLMs to reason, the key is that it be a problem statement that there is a defined and verifiable answer. That's right. Is that right? And so, and you might think about this as like, let's give a bunch of examples. Like in medicine, this might be like, you know, a diagnosis that like a panel of human doctors agrees with. or, by the way, or a diagnosis that actually, you know, solves the condition.

25:30In law, this would be a, you know, an argument that in front of a jury actually results in an acquittal or something like that. In math, it's an equation that actually solves properly. In physics, it's a result that actually works in the real world. I don't know, in civil engineering, it's a bridge that doesn't collapse, right? So there's always some test of practice. The first two do not work very well just yet. Like, I would say law and healthcare, they're still a little too squishy, a little too soft. It's unlike math or code. Like, the way they're training on math, they're using this sort of like a program language, provable language called Lean for proofs, right?

26:11So you can run a Lean statement. You can run a computer code. Perhaps you can run a physics simulation or a civil engineering sort of physics simulation. But you can't run a diagnosis. Okay. So I would say that... But you can verify it with human answers or not. Yeah, so that's a more HF in a way. So it is not the sort of autonomous RL, like fully scalable autonomous, which is why coding is moving faster than any other domain, is because we can generate these problems and verify them on the fly. But there's two, with coding, as anybody who's coded knows, there's coding, there's two tests, which is one is, does it code compile?

26:50Right. And then the other is, does it produce the right output? And just because it compiles doesn't mean it produces the right output. You tell me, but verifying that it's the correct output is harder. Yeah, so Sweebench is a collection of verified pull requests end states. So it is not just about compiling. So Sweebench is the main benchmark used to test whether AI is good at software engineering tasks. And we're almost saturating that. So last year, we're at like maybe 5 % early 24 or less. And now we're like 82 % or something like that with cloths on at 4.5. That's state of the art. And that's like a really nice hell climb that's happening right now.

27:36And basically, they went and looked on GitHub. They found the most complex repositories. They found bug statements that are very clear. And they found ProQuest that actually solved those bug statements with unit tests and everything. So there is an existing corpus on GitHub of tasks that the AIs can solve. And you can also generate them. Those are not too hard to generate what's called synthetic data. But you're right. It's not infinitely scalable because some human verifiers still need to kind of look at the task. But maybe the foundation models have found a way to have the synthetic training go all the way.

28:18Right. And then what's happening, I think, because what's happening is the foundation model companies are, in some cases, they're actually hiring human experts to generate new training data. Yes. So they're actually hiring mathematicians and physicists and coders to basically sit, and they're hiring human programmers, putting them on the cocaine. Yes. And having them, probably coffee. Yeah. And having them actually write code, and then write code in a way where there's a known result of the code running such that this RL loop can be trained properly. That's right. And then the other thing these companies are doing is they're building systems where the software itself generates the training data, generates the tests, generates the validated results, and that so-called synthetic training data.

28:58That's right. But again, those work in the very hard domains. It works to some extent in the software domains. And I think there's some transfer learning. You can see the reasoning work when it comes to tools like deep research and things like that. But we're not making as fast as progress in the more soft domains. So, softer domains, meaning domains in which it's harder or even impossible to actually verify correctness of result in a sort of a deterministic, factual, grounded, non-controversial way. Like if you have a chronic disease, you could have, you know, you have a POTS or, you know, whatever, EDS syndrome.

29:40And they're all clusters. Because it is the domain of abstraction. It is not as concrete as code and math and things like that. So I think there's still a long ways to go there. Right. So sort of the more concrete the problem. Like it's the concreteness of the problem that is the key variable, not the difficulty of the problem. Would that be a way to think about it? Yeah, I think the concreteness in a sense of can you get a true or false verifiable, I put it. Right. But like in any domain of human effort in which there's a verifiable answer, we should expect extremely rapid progress? Yes. Right.

30:11Okay. Absolutely. And I think that's what we're seeing. Right. And that for sure includes math. That for sure includes physics. For sure includes chemistry. For sure includes large areas of code. That's right. Right. What else does that include, do you think? Bio, like we're seeing with a protein folding. Like genomic, genomic. Yeah, yeah. Things like that. I think some areas of robotics, there's a clear outcome. But it's not that many, I mean, surprisingly. Well, it depends on your point of view. Some people might say that's a lot. And then you mentioned the pace of improvement. So what would you expect from the pace of improvement going forward for this?

30:48I think we're ripping on coding. I think it's just, it's going. I think it's going to be like what we're working on with Agent 4 right now is by next year, we think you're going to be sitting in front of Replit and you're shooting off multiple agents at a time. You're like planning a new feature. So I want a social network on top of my storefront. And another one is like, hey, refactor the database in your running parallel agents. So you have about five, 10 agents kind of working in the background and they're merging the code and taking care of all of that. But you also have a really nice interface on top of that, that you're doing design and you're interacting with AI in a more creative way, maybe using visuals and charts and things like that.

31:36So there's a multimodal angle of that interaction. So I think, you know, creating software is going to be such an exciting area. And I think that the lay person will be as good as what a senior software engineer that works at Google is today. So I think that's happening very soon. But, you know, I don't see them, and I'd be curious about your point of view, but like my experience between as a sort of a, you know, on the, let's say, healthcare side or more, you know, write me an essay side or more creative side, I haven't seen as much of a rapid improvement as what we're seeing in code. So I think code is going to go to the moon.

32:18Math is probably as well. Some, you know, scientific domains, bio, things like that, those are going to move really fast. Yeah. So there's this weird dynamic. Let's see if you agree with this. And Eric also, I'm curious your point of view on this. Like, there's this weird dynamic that we have, and we have this in the office here a lot, and I also have this with, like, the leading entrepreneurs a lot, which is this thing of, like, like, wow, this is the most amazing technology ever, and it's moving really fast, and yet we're still, like, really disappointed. And, like, it's not moving fast enough, and, like, it's, like, maybe right on the verge of stalling out.

32:48And like, you know, we should both be like hyper excited, but also on the verge of like slitting our wrists. Because like, you know, the gravy train is coming to an end. And I always wonder, it's like, you know, on the one hand, it's like, okay, like, you know, not all, I don't know, ladders go to the moon. Like, just because something, you know, looks like it works or, you know, doesn't mean it's going to, you know, be able to scale it up and have it work, you know, to the fullest extent. You know, so like, it's important to like recognize practical limits and not just extrapolate everything to infinity.

33:13On the other hand, like, you know, we're dealing with magic here that we, I think, probably all would have thought was impossible five years ago or certainly 10 years ago. Like, I didn't, you know, like, I, you know, I got my CS degree in the late 80s, early 90s. I never, I didn't think I would live to see any of this, right? Like, this is just amazing that this is actually happening in my lifetime. But there's a huge bet on AGI, right? Like, whether it's the foundation models, I think, you know, now the entire U.S. economy is sort of a bet on AGI. and there are crucial questions to ask whether are we on track to AGI or not.

33:43Because there are some ways that I can tell you it doesn't seem like we're on track to AGI because there doesn't seem to be transfer learning across these domains that are, you know, significant, right? So if we get a lot better at code, we're not immediately getting better at like generalized reasoning. We need to go also, you know, get training data and create RL environment for bio or chemistry or physics or math or law. And this has been the sort of point of discussion now in the AI community after the Dworkish and Richard Sutton interview where Richard Sutton kind of poured this cold water on the bitter lesson.

34:25So everyone was using this essay that he wrote called The Bitter Lesson. The idea is that there are infinitely scalable ways of doing AI research. And any time you can pour more compute and more data and get more performance out, that's the ultimate way of getting to AGI. And some people interpreted that interview that perhaps he's doubtful that we're even on a better lesson path here. And perhaps the current training regime is actually very much the opposite in which we are so dependent on human data and human annotation and all of that stuff. So I think that I agree with you. I mean, as a company, we're excited about where things are headed.

35:15But there's a question of like, are we on track to AGI or not? And be curious what you think. So, and, you know, I think, you know, Ilya Sescover makes a specific form of this argument, which is basically like we're just literally running out of training data. It's a fossil fuel argument. Right. Like we've slurped all the training data. Fundamentally, we've slurped all the data off the internet. That is where almost all the data is at this point. There's a little bit more data that's in like, you know, private dark pools somewhere that we're going to go get. But we have it all. And then, right, we're in this business now trying to generate new data.

35:42But generating new data is hard and expensive, you know, compared to just like slurping things off the internet. So there are these arguments. You know, having said that, you know, you get a definitional question here really quick, which are kind of a rabbit hole. But having said that, like you mentioned transfer learning. So transfer learning is the ability of the machine to be an expert in one domain and then generalize that into another domain. My answer to that is like, have you met people? And how many people do you know are able to do transfer learning? Not many. Not many, right? Well, because there's quite the opposite, actually.

36:09The nerdier they are in a certain domain, they kind of, you know, often they have blind spots. We joke about how everyone's just retarded in one area, or they make some massive mistake, and don't trust them on this, but on this other topic. Right. Yeah, well, and this is a well-known thing among, for example, public intellectuals. So this happens, there's actually been whole books written about this on so-called public intellectuals. So you get these people who show up on TV, and they're experts. And what happens is, they're like an expert in economics, right? And then they show up on TV, and they talk about politics, and they don't know anything about politics.

36:36Right? Or they don't know anything about medicine, or they don't know anything about the law, or they don't know anything about computers. You know, this is the Paul Gruckman talking about how the internet is going to be no more significant than the fax machine. Fax, yeah. He's a brilliant economist. He has no idea how a computer works. Is he a brilliant economist? Well, at one point. At one point. At one point. Let's get, even if, even if he's a brilliant, well, this is the thing, like, what does that mean? Should a brilliant economist be able to extrapolate, you know, the internet is a good question.

37:02But the point being, like, even if he is a brilliant, you know, or take anybody, oh, by the way, or like Einstein's like actually my favorite example. I think you'd agree Einstein was a brilliant physicist. He was a Stalinist. Like, he was a socialist and he was a Stalinist. And he was like, he thought, like, Stalin was fantastic. Well, he's already out, so. Yeah, okay, all right. True socialism. All right, Einstein. You know, I'll take your word for it. But, like, once he got into politics, he was just, like, totally loopy. Or, you know, even right or wrong, he just sounded like all of a sudden like an undergraduate lunatic, like somebody in a dorm room.

37:36Like, there was no transfer learning from physics into politics. Like, he was right or wrong. He didn't, there was no, there was clearly, there was nothing new in his political analysis. It was the same routine bullshit you get out of, you know. Yeah, so in a way, the argument you're making is like, we may be already at human level AI. I mean, perhaps the definition of AGI is something totally different. It's like above the human level that something that truly generalizes across domains. It's not something that we've seen. Yeah, like we've idealized, yeah, as I said, we've, we've, and in your look, we should, we should shoot big, But we've idealized a goal that may be idealized in a way that, like, number one, it's just, it's like so far beyond what people can do that it's no longer a relevant comparison to people.

38:16And usually AGI is defined as, you know, able to do everything better than a person can. And it's like, well, okay, so if doing everything better than a person can, it's like if a person can't do any transfer learning at all, doing even a little bit, a marginal bit might actually be better. Or it might not matter just because no human can do it. And so, therefore, you just stack up the domains. There's also this well-known phenomenon in AI. Typically, this works the other way, which is a phenomenon in AI. AI engineers always complain about, scientists always complain about, which is the definition of AI is always the next thing that the machine can't do.

38:45And so the definition of AI for a long time was kind of beat humans at chess. And then the minute it could beat humans at chess, that was no longer AI. That was just like, oh, that's just boring. That's computer chess. It became an entire decision. Yeah, it's computer chess. It's just like boring, and now it's an app on your iPhone, and nobody cares, right? And it's immediately... The Turing test was the next one. The Turing test. And then we passed it and nobody... This is a really big deal. There was no celebration. There was no parties. That's exactly right. There was no party. For 80 years, the Turing test, I mean, they made a movie about it, like the whole thing.

39:11That was the thing. And like, we blew right through it. And nobody even registered it. Nobody cares. It gets no credit for it. We're just like, ah, it's still a complete piece of shit. Like I said, yeah. Right. And so there's this thing where, so the AI scientists are used to complaining, basically, that they're always being judged against the next thing as opposed to all the things they've already solved. But that's maybe the other side of it, which is they're also putting out for themselves an unreasonable goal, and then doing this sort of self-flagellation kind of along the way. And I kind of wonder, yeah, I wonder kind of which way that cuts.

39:42Yeah, yeah. It's an interesting question. Like, I started thinking about this idea of, like, it doesn't matter whether it's truly AGI, and the way I define AGI is that you put in an AI system in any environment and efficiently learns. You know, it doesn't have to have that much prior knowledge in order to kind of learn something, but also can transfer that knowledge across different domains. But, you know, we can get to like functional AGI. And what functional AGI is, is just, yeah, collect data on every useful economic activity in the world today and train an LLM on top of that or train the same foundation model on top of that.

40:21And we'll go, we'll target every sector economy and you can automate a big part of labor that way. So I think, yeah, I think we're on that track. For sure. You tweeted after GPT-5 came out that you were feeling the diminishing returns. What were you expecting and what needs to be done? Do we need another breakthrough to get back to the pace of growth? What are your thoughts? I mean, this whole discussion is sort of about that. And my feeling is that, you know, GPT-5 got good at verifiable domains. It didn't feel that much better at anything else. The more human angle of it, it felt like it regressed.

40:57and like you had this sort of Reddit pitchfork sort of movement against Sam and OpenAI because they felt like they lost a friend. GPT-4.0 felt a lot more human and closer, whereas GPT-5 felt a lot more robotic, very in its head, kind of trying to think through everything. And so I would have just expected like when we went from GPT-2 to 3, it was clear it was getting a lot more human. It was a lot closer to our experience. You can feel like it's actually all it gets me. Like there's something about it that understands the world better. Similarly, three to four, four to five didn't feel like it was a better overall being as it were.

41:46But is that, is that, is that, is that a, is the question there like, is it emotionality? Is it... Partly emotionality, but again, partly, like, I like to ask models, like, very controversial things. Can it reason through, I don't know how deep we want to go here, but, like, what happened with World Trade 7? Right, right. Sure. It's an interesting question, right? Like, I'm not putting out a theory, but, like, it's interesting. Like, how did it, you know, and can it think through controversial questions in the same way that it can go think through a coding problem? And there hasn't been any movement there.

42:30Like, all the reasoning and all of that stuff. And not just that, you know, that's a cute example, but like COVID, right? Like, you know, the origins of COVID. You know, go, you know, dig up GPT-4 or other models and go to GPT-5. you're not going to find that much difference of, okay, let's reason together. Let's try to figure out what was the origins of COVID. Because it's still an unanswered question, you know? And I don't see them making progress on that. I mean, you play a lot with them. Do you feel like... Yeah, I use it differently. I don't know. Maybe I have different expectations. The way I...

43:01My main use case actually is sort of PhD and everything at my beck and call. And so I'm trying to get it to explain things to me more than I'm trying to like, you know, have conversations with it maybe. Maybe I'm just unusual with that. And that gets better. Well, so what I found specifically is a combination of like GPT-5 Pro plus deep reasoning or like Rock 4 Heavy, like the highest end models like that. You know, they now basically generate 30 to 40 page, you know, essentially books on demand on any topic. And so anytime I get curious about something, maybe it's my version of it, but it's something like, I don't know.

43:35Here's a good example. When an advanced economy puts a tariff on a raw material or on a finished good, like who pays? You know, is it the consumer? Is it the importer? Is it the exporter? Or is it the producer? And this is actually a very complicated, it turns out a very complicated question. It's a big, big, big thing that economists study a lot. And it's just like, okay, you know, who pays? And what I found, like, for that kind of thing is it's outstanding. Well, but it's outstanding at sort of going out of the web, getting information, synthesizing it. Correct. It gives me a synthesized 20, 30, 40 page.

44:05It basically tops out of 40 pages of PDF. But I can get up to 40 pages of PDF, but it's a completely coherent, and as far as I can tell, for everything I've cross-checked, a completely, like, world-class... Like, if I hired, you know, for a question like that, if I hired, like, a great, you know, econ PhD postdoc at Stanford who just, like, went out and did that work, like, it would maybe be that good. But then, of course, the significance is it's like, you know, at least for, this is true for many domains, you know, kind of PhD and everything, and so... But this is synthesizing knowledge, not trying to create new knowledge.

44:36Well, but this gets to the sort of, you know, of course, you get into the angels dancing on the head of a pen thing, which is like, what's the difference? How much new knowledge ever actually is there anyway? What do you actually expect from people when you ask them questions? And so what I'm looking for is like, yes, explain this to me in like the clearest, most sophisticated, most complex, most like complete way that it's possible for somebody, you know, for a real expert to be able to explain things to me. And that's what I use it for. And again, as far as I can tell from the cross-checking, like I'm getting, you know, like almost like basically a hundred out of a hundred, like I don't even think I've had an issue in months where it's like had a problem in it.

45:11And it's like, yeah, you can say, yeah, synthesizing is supposed to create new information, but like it's generating a 40-page, it's basically generating a 40-page book. That's amazing. That's like incredibly like fluid. It's, you know, it's, you know, the logical coherence of the entire, like it's a great, like if you evaluated a human author on it, you would say, wow, that's a great author. Yeah. You know, are people who write books, you know, creating new knowledge? Well, yeah. Well, sort of not because a lot of what they're doing is building on everything that came before them as synthesizing and combining.

45:41But also like a book is a creative accomplishment, right? And so, yeah, one of the things I'm interested in, I'm hoping AI could help us solve this, just like how confusing the information ecosystem right now. You know, everything feels like propaganda. Like it doesn't feel like you're getting real information from anywhere. So I really want an AI that could help me reason from first principles about what's happening in the world for me to actually get real information. And maybe that's an unreasonable sort of ask of the AI researchers, but I don't think we have made any progress there. So maybe I'm over-focused, maybe I'm over-focused on arguing with people as opposed to trying to get down to the line truce.

46:22But, well, here's the thing I do a lot with this is I just say, like, take a provocative point of view and then steel man the position. Take your COVID thing. So I often pair these. Steel man the position that it was a lab leak and then steel man the position that it was natural origins. And again, is this creativity or not? I don't know. But what comes back is 30 pages each of like, wow. That is the most compelling case in the world I can imagine with everything marshaled against it. The argument is structured in the most possible way. Part of the reason that's not happening is because it stopped being taboo to talk about human origin.

46:52When it was taboo, the AIs would talk down to you. It's like, oh, you're a conspiracy theorist. And so there's a, you know, a period of time. And so it takes something truly controversial. And they actually, they can't reason about it because of all the RLHF and all sorts of limitations. And as you know, I won't pick those specific ones here, but like there are certain big models that will still lecture you. That you're a bad person for asking that question. But, you know, like I was just, some of them are just like really, really open now to, you know, being able to do these things. And then, yeah, so, okay.

47:28Yeah, so, okay. So, yeah, so there's this, yeah, so basically, like, ultimately what you're looking for, like, the ultimate thing would be, if there's something that's like, I don't think anybody's really defined this really well, because it's not, because, again, it's like, conventional, all the conventional definitions of AGI are, like, basically comparing to people. Yeah. And there it's always like, you know, the conventional explanations of AGI always, for me, strike me a lot, like the debate around like whether a self-driving car works or not, which is, does a self-driving car work because it's a perfect driver or does it work because it is better than the human driver?

48:00And better than the human driver, I think, is actually quite, you know, just like with the chess thing and the go thing. I actually think like that, that's like a real thing. And then there's the like, is it a perfect driver, which is, you know, what they're obviously the self-driving car companies are working for. But then I think you're looking for something beyond the perfect driver. You're looking for the car who knows where to go. So I'm of two minds. So one mind is the sort of practical entrepreneur. And I just, I have so many toys to play with, to build. Like stop AI progress today and Repl.o will continue to get better for the next five years.

48:31There's so much we can do just on the app layer and the infrastructure layer. So, you know, but I think that it will, you know, the foundation models will continue to get better as well. And so it's a very exciting time in our industry. The other mind is more academic because as a kid, I've always been interested in the nature of consciousness, nature of intelligence. I was always interested in AI and reading the literature there. And I would point to the RL literature. So Richard Sutton, there's another guy, I think co-founder of DeepMind, Shane Lagg, wrote a paper trying to define what AGI is.

49:07and in there I think that the definition of HI I think is the original perhaps correct one which is efficient continual learning like if you truly want to build an artificial general intelligence that you can drop in any domain you can drop in a car without that much prior knowledge about cars and within you know how long does it take a human to learn how to drive within months being able to drive a car really well. Generalized skill acquisition, generalized understanding acquisition, generalized reasoning acquisition. And I think that's the thing that would truly change the world. That's the thing that would give us a better understanding of the human mind, of human consciousness.

49:54And that's the thing that will propel us to the next level of human civilization, I think. So on a civilization level, I think that's a really deep question. question, but separate from the economy and the industry, which is all exciting. But there's an academic aspect of it that I'm really... And so what odds, if we're on CalSheet today, what odds do we place on that? I'm kind of bearish on true AGI breakthrough because what we built is so useful and economically valuable. So in a way... Oh, good enough. Good enough is the enemy. Yeah, yeah. Do you remember that essay? Worse is better. Worst is better.

50:33Worst is better. So there's like a local, there's like a trap. There's like a local maximum trap. That's right. We're in a local maximum trap. Local maximum trap where it's, because it's good enough for so much economically productive work, it relieves the pressure in the system to create the generalized answer. Yes. And then you have the weirdos like Rich Sutton and others that are still trying to go down that path and maybe they'll succeed. But there's enormous optimization energy behind the current thing that we're hell climbing on this like local maximum. Right, right, right. Right. And the irony of it is everybody's worried about like the, you know, the gazillions of dollars going into building out all this stuff.

51:06And so the most ironic thing in the world would be if the gazillions of dollars are going into the local maximum. That's right. As opposed to a counterfactual world in which they're going into solving the general problem. But it's also potentially irrational. Like maybe the general problem is actually, you know, not within our lifetimes. Who knows? How much further do you think, like, do you think we squeeze most of the juice out of LLMs in general then? Or are there any other research directions that you're particularly excited about? Well, that's the thing. I think the problem is there aren't that many.

51:37I think the breakthroughs in RL are incredibly exciting, but we also knew about them now for like over 10 years where you marry generative systems with TreeSearch and things like that. But there's a lot more to go there. And I think, again, the original minds behind reinforcement learning are trying to go down that path and try to kind of bootstrap intelligence from scratch. Carmack is going down that path. As far as I understand, Carmack, you guys may be invested, but they're not trying to go down the LLM path. So there are people that are trying to do that, but I'm not seeing a lot of progress or outcome there.

52:15But I watch it kind of from far. Although, you know, for all we know, there's already a bot on X somewhere. Maybe. You never know. It might not be a big announcement. It might just be, you know, one day there's just like a bot on X that starts winning all the arguments. Yeah, it could be. Or a user of Reddit and all of a sudden it's generating incredible software. Okay, let's spend our remaining minutes. Let's talk about you. So, yeah, take us and start from the beginning with your life. And how did you get from being born and being in Silicon Valley? Okay. In two minutes. Yeah. I'm joking. Yeah.

52:51I got introduced to computers very, very early on. And so for whatever reason, so I was born in Amman, Jordan. And for whatever reason, my dad, who was just a government engineer at the time, decided that computers were important. And he didn't have a lot of money, took out a dad, bought a computer. It was the first computer in our neighborhood. First computer of anyone I know. and I just, one of my earliest memories, I was six years old, just watching my dad unpack this machine and sort of open up this huge manual and kind of finger type CD, LS, MKDIR and like I would, you know, be behind his shoulder and just like watching him, you know, type these commands and seeing the sort of machine kind of respond and do exactly what he's asked it to do.

53:41Popping Tylenol. Popping Tylenol. Exactly. autism activated. Of course. You have to. You have to. Exactly. What kind of what kind of computer was it? It was an IBM as far as I remember. IBM PC. It was IBM PC. So what year was this about? 1993. 1993. Okay. So did it have Windows at that point? No. It didn't have Windows. It was right before Windows. Right before Windows. I think Windows had been out but you would It was an add-on. It was an add-on. You wouldn't boot it up. So I think we bought the disk for Windows, and you had to kind of bootload it from the disk, and it will open Windows, and you can click around.

54:24It wasn't that interesting because there wasn't a lot on it. So a lot of time I just spent it in DOS, and writing batch files and opening games and messing around with that. But it wasn't until Visual Basic that I started. So after Windows 95 that I started making real software. and the first idea I had was I used to be a huge gamer so I used to go to these LAN gaming cafes and play Counter-Strike and I would go there and the whole thing is full of computers but they don't use any software to run their business. It was just like people running around just like writing down your machine number how much time you spend on it and how much did you pay and kind of tapping your shoulders like hey you need to pay a little more for that and I asked them like why don't you just build a piece of software that allows me to log in and have a time or whatever.

55:12And I was like, yeah, we don't know how to do that. And I was like, okay, I think I know how to do that. So I spent, I was like 12 or something like that. I spent like two years building that and then went out and tried to sell it and was able to sell it and was making so much money. I remember McDonald's opened in Jordan around the time when I was 13, 14. I took my entire class at McDonald's. It was very expensive, but I was bawling. I had all this money and I was showing off. And so that was the first business that I created. And then when it came to... And at the time, I started learning about AI, you know, reading sci-fi and all of that stuff.

55:54And when it came time to go to college, I didn't want to go to computer science because I felt like coding is on its way to get automated. I remember using these wizards. Do you remember wizards? I remember wizards. Wizards, basically. It's like extremely crude, early bots that generate code. That generate code, yeah. Yeah, and I remember you could type in a few things, like, here's my project, here's what it does, whatever, and then click, click, click, and it just scaffold a lot of code. I was like, oh, I think that's the future. Coding is such a... It's almost solved. Yeah, it's solved. Why should I go into coding?

56:26I was like, okay, if AI can do the code, what should I do? Well, someone needs to build and maintain the computers. And so I went to the computer engineering and did that for a while. But then rediscovered my love for programming, reading program essays on Lisp and things like that, and started messing around with Scheme and programming languages like that. But then I found it incredibly difficult to just like learn different programming languages. I didn't have a laptop at the time. And so every time I go to like wanting to learn Python or Java, I would go to the computer lab, download gigabytes of software, try to set it up, type a little bit of code, try to run it, you know, run into missing DLL issue.

57:10And I was like, man, this is so primitive. Like at the time, it was 2008, something like that. You know, we had Google Docs, we had Gmail. You could like open the browser and partly thanks to you and be able to kind of use software on the Internet. And I thought the web is the ultimate software platform. everything should go on the web. Okay, who's building an online development environment? And no one. And it felt like I found a$100 bill on the floor of Grand Central Station. Surely someone should be building this. But no, no one was building this. And so I was like, okay, I'll try to build it.

57:49And I got something done in a couple hours, which was a text box. You type in some JavaScript, and there's a button that says eval. you click eval and evaluates it shows you in an alert box. So one plus one, two. I was like, oh, I have a programming environment. I showed it to my friends, people started using it, I added a few additional things like saving the program. I was like, okay, all right, there's a real idea here. People love it. And then again, it took me two or three years to actually be able to build anything because the browser can only run JavaScript. And it took a breakthrough at the time Mozilla had a resource project called mScripten that allowed you to compile different programming languages like C, C++ into JavaScript.

58:36And for the browser to be able to run something like Python, I needed to compile C Python into JavaScript. So I was the first to do it in the world. So built, contributed to that project and built a lot of the scaffolding around it. And we, my friends and I compiled and I was like, okay, we did it for Python, let's do it for Ruby, let's do it for Loa. And that's how the emergence of the idea for Replit came, is that when you need a REPL, you should get it, you should REPL it. And so a REPL is the most primitive programming environment possible. So I added all these programming languages, and again, all this time, my friends were using it and excited about it.

59:15And I was on GitHub at the time, and just my standard thing is when I make a piece of software, I just open source it. And so I was open sourcing all the things. I was years building just like this underlying infrastructure to be able to just run code in the browser. And then it went viral. It went viral in Hacker News. And it coincided with the MOOC era. So massively online courses. Udacity was coming online. Coursera. And most famously, Code Academy. So Code Academy was the first kind of website that allowed you to code in the browser interactively and learn how to code. And they built a lot of it on my software that I was open sourcing all the way from Jordan.

59:51And so I remember seeing them on Hacker News and they were going super viral. I was like, hey, I recognize this. What are you using? And so I left a Hacker News comment. I was like, oh, you're using my open source package. And so they reached out to me. They're like, hey, we'd like to hire you. I was like, I'm not interested. I want to start a startup. I want to start this thing called Replit. And they're like, well, no, you should come work with us. We can do the same stuff. And I kept saying no. I was like, okay, I'll contract with you. They were paying me$12 an hour. I was really excited about it.

1:00:20Back from Amman. Um, but they came out to their, to their credit. They came out of Jordan to recruit me and spent a few, a few days there. And then, uh, you know, I kept saying no. And in the end, they gave me an offer. I can't refuse. Um, and they got me an O-Win visa, came to the United States. That's when you moved. So when, when, when was the first, cause you were born, what year? 1987. 87. What was the first year that you could remember where you had the idea that you might not live your life in Jordan and you might, you might actually move to the U.S.? Uh, when I watched Pirates of Silicon Valley.

1:00:49Is that right? Okay. Got it. All right. maybe 98 or 99. I don't know when it came out. That might be a good place to go. Yeah. Is it worth telling the hacker story? Because there's a version of the world where you didn't actually, like if that changed differently, maybe you wouldn't have gone to America. Right, right. Yeah, so in school, I was programming the whole time. So I just want to start businesses. I just like, I'm exploding with ideas all the time. And like the reason Replit exists is because I have ideas all the time. I just want to go type them on the computer and like build them. So I wasn't going to school.

1:01:19It was like incredibly boring for me. And part of the reason why Replet has a mobile app today is because I always wanted to program under the desk. It's like just two things. And so at school, they kept failing me for attendance. So I would get A's, but I just didn't show up. And so they would fail me. And so I felt it was incredibly unfair. And all my friends were graduating. Now this year, this was like 2011. I've been like for six years in college. It should be like a three or four year. And I was like incredibly depressed. I really wanted to be in Silicon Valley. And so I was like, oh, what if I change my grades?

1:01:57There we go. In the university database. And so I went into my parents' basement and implemented the polyphasic sleep. Are you familiar with that? I am. Leonardo da Vinci's polyphasic sleep. I didn't hear from Leonardo da Vinci. I heard it from Seinfeld. Because there's an episode where John Kramer goes on polyphasic sleep. 20 minutes every four hours. Yes, 20 minutes every four hours. And yes, and this somehow is going to work well. Yeah, and hacking, if you've ever done anything. As the meme goes, this has never worked for anybody else, but it might work for me. Yes. And a lot of what hacking is, is that you're coming up with ideas for like finding certain security holes and like writing a script and then running that script and that script will take like 20, 30 minutes to run and so you'll take that you know 20 30 minutes to sleep and go on so i spent two weeks just going mad like trying to hack into the university database and uh finally i found um a way i found a sequel injection somewhere on the site uh and i found a way to like be able to edit the the records but i didn't want to risk it so i went to my neighbor who's going to the same school uh i think till this day no one caught him but i went to him and i said um hey uh i have this way to change grades.

1:03:17Like, would you want to be my guinea pig? And I was honest about it. I was like, I'm not going to do it. Are you open to do it? He's like, yeah, yeah, yeah. They call it human trials. This is how medicine works. So we went and changed his grades and he went and pulled his transcript and the update wasn't there and went back to the basement. Well, it turned out that I had access to the slave database. I didn't have access the ambassador database. So find a way through the network, privileged escalation. It was an Oracle database that had a vulnerability and then found the real database. And then I just, you know, did it for myself, changed the grades and went and pulled my transcripts.

1:04:02And sure enough, it actually changed, went and bought the gown, went to the graduation parties, did all that and we're graduating and then one day I'm at home it's like maybe 6 or 7 p.m. I get a you know the telephone at home rings I'm gonna I'm gonna ring sound yes well hello and he's like hey this is the university registration system and I knew the guy that run it he's like look you know we're having this problem the system's been down all day and it keeps coming back to your record there's an anomaly in your record where you have a passing grade, but you're also banned from that final exam of subject.

1:04:47I was like, oh, shit. Well, it turns out the database is not normalized. So typically, when they ban you from an exam, the grade resets to 35 out of 100. But apparently, there's a Boolean flag. And by the way, all the column names in the database are single letters. That was the hardest thing. It's security by obscurity. And it turns out there's a flag that I didn't check. So when you go over attendance, when you don't attend and they want to fail you, they ban you from the final exam. So I changed the grades and that created an issue and brought down the system. So they were calling me. And I thought at the time, I was like, you know, I could potentially lie and it'll be a huge issue.

1:05:30Or I just like, I'll just fess up. I'll just fess up. So I said, hey, listen, look, yeah, I might know something about it. Hey, let me come tomorrow and kind of talk to you about what happened. So I go in and I open the door. It's the deans of all the schools. It's like computer science, computer science. They were all working on it for like days because it's like, it's a very computer heavy, you know, university. And it was like a problem. And they're all kind of really intrigued about what happened. And so I pull up a whiteboard and started explaining what I did. and everyone was engaged. I gave them a lecture, basically.

1:06:06Your oral exam for your PhD. So you were like, this is great. They were really excited. And I think it was endearing to them. I was like, oh, wow, this is a very interesting problem. And then I was like, okay, great, thank you. And I was like, hey, wait, wait, we don't know what to do with you. Do we send you to jail? And I was like, hey, we have to escalate to the university president. and he was a great man and I think he gave me a second chance in life and I went to him and I, you know, I explained the situation. I said, like, I'm really frustrated. I need to graduate. I need to get on with my life.

1:06:44I've been here for six years and I just can't sit in school. The stuff I already know, I'm a really good programmer. And he gave me a Spider-Man line at the time. It's like, with great power comes great responsibility and you have a great power. And it really affected me and I think he was right at the moment. And so he said, well, we're going to let you go, but you're going to have to help the system administrators secure the system for the summer. I was like, happy to do it. And I show up and all the programmers there hate me. They hate my guts. And they would lock me out. Like I would see them, they would be outside.

1:07:19I would knock on the door and no one would listen. It was like, they don't want to let me in. I tried to help them a little bit. They weren't collaborative. And so I was like, all right, whatever. And so it came time for me to actually graduate. It was the final project. And one of the computer scientists came to me and he said, look, I need to call a favor. I was a big part of the reason we kind of let you go and we didn't kind of prosecute you. So I want you to work with me on the final project. And it's going to be around security and hacking. I was like, no, I'm done with that shit. I just want to build programming environments and things like that.

1:07:55And he's like, no, you have to do it. I was like, okay. So I thought I'd do something more productive. So I wrote a security scanner that I was very proud of, that kind of crawls a different side, that tries to do SQL injection and all sorts of things. And actually, my security scanner found another vulnerability in the system. And so I went to the defense, and he's like, you need to run this security scanner live and show that there's a vulnerability. And I didn't understand what was going on at the time, but I just, okay. So I give the presentation about how the system works, and I was like, oh, let's run it.

1:08:25And it showed that there's security vulnerability. Okay, let's try to get a shell. So the system automatically runs all the security stuff and gets you a shell. And then the other dean that turned out, he was giving the mandate to secure the system. And now I started to realize I'm a pawn in some kind of rivalry here. And his face turned red and it's like, no, it's impossible. We secure the system. You're lying. I was like, you're accusing me of lying. All right, what should we know? Should we know your salary or your password? What do you want me to look up? And I was like, yeah, I look up my password.

1:09:04So I look up his password. And it was like gibberish. It was encrypted. And I was like, oh, that's not my password. See, you're lying. I was like, well, there's a decrypt function that the programmers put in there. So I do decrypt and it shows his password. And it was something embarrassing. I was like, I forgot what it was. And so he gets up, really angry, shakes my hand and leaves to change his password. I was able to hack into the university another time. Luckily, I was able to graduate, give them software, they secured the system. But yeah, later on, I would realize that, yeah, he wanted to embarrass the other guy, which is why I was in the middle.

1:09:40Economic politics. Well, I think the moral of the story is if you can successfully hack into your school system and change your grade, you deserve the grade and you deserve to graduate. I think so. And just for any parents out there, or children out there, you can cite me as the most. You can say that I'm proud of me as the moral authority in this. One maybe lesson I think that is very relevant for the AI age, I think that the traditional sort of more conformist path is paying less and less dividends. And I think kids coming up today should use all the tools available to be able to discover and chart their own paths.

1:10:19Because I feel like just listening to the traditional advice and doing the same things that people have always done is just not as, it's not working out as much as we'd like. Yeah, that's right. Thanks for coming to the podcast. Thank you, man. Fantastic. Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode.

1:11:01As a reminder, the content here is for informational purposes only. It should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures.