How Replit’s AI Tools are Changing Software Development with Co-founder and CEO Amjad Masad

0:05This week on NoPriors, we explore Replit and its new AI tool, Ghostwriter. Replit now has more than 22 million users and recently started a partnership with Google. With Replit, anyone from beginners to experts can quickly build and release fully functional live apps in seconds, now with the help of AI. Amjad Massad, CEO, co-founder, and head engineer at Replit, joins a lot of me this week on No Priors. Welcome. Thank you. Amjad, just to start, you were working as an engineer at Facebook. What made you decide to start Replit? My journey with Replit actually starts way back in college. I was really interested in program languages, and I found it really difficult to just keep setting up the development environment.

0:48I didn't have a laptop at the time. And so I was like, why can't I open a browser window and start coding? Turns out nobody has done that. And I was kind of naive and thought I could probably try to do it. I got a prototype up pretty quickly, but my friends loved it in college. Everyone started using it. It was just like a simple web page with a text box and a button that says run. which is still kind of in a lot of ways what Repload is. But then that started me in a multi-year journey to actually build the first kind of in-browser sandbox. And at the time, we had a bit of a breakthrough. We were the first to compile Python and Ruby and other languages to JavaScript.

1:27I open sourced that project. That got me a job at Code Academy. I was a founding engineer. And after that, I went to Facebook. and I was one of the founding engineers in React Native. We made React and Just, Babel, and a bunch of tools that JavaScript developers use today. Our goal was to make mobile development as fun and easy and fast as web development, which I think we succeeded at. Towards the end of my time at Facebook, I just wanted to see what to do next and kind of looked around and whatever happened to this online coding space. And there was a bunch of online editors, but none of them really felt like web native.

2:14Like you can just share URL and someone can hop in and code with you. Kind of like how Figma or Google Logs work. And we decided to start in 2016. You have made a huge bet on AI as a company. Can you just talk about Ghostwriter and how it came about and the investment in this area? Yeah, throughout my career working on code that handles code, whether it's at Codecademy for teaching programming, whether it's my own project, whether it's React and building the runtime around React Native, I always felt like our tools that were handling code, whether it's compiling it, parsing it, minifying it, all that stuff, we're very kind of rigid and very laborious.

3:01In a lot of ways, you're building sort of a classical intelligence system, very algorithmic, and a lot of heuristics and all of that. And I always thought that you can probably apply machine learning to it. And I started reading around whether anyone had done it. There was this seminal paper in 2012 called On the Naturalness of Software. And basically, a bunch of researchers try to apply NLP to code. And what they found is that actually code can be modeled like any language. That's why they call it naturalness, because, hey, code is kind of repetitive. Like language, you can do things like basic Ingram model can actually start to generate code that is compilable.

3:41That had a huge impact on me. And every year or two while starting Replit, I would go look at the state of the art on ML on code. And nothing ever really worked all that well up until GPT-2. and you can take GPT-2 and fine-tune it and try to make it right code, and it was kind of okay. But obviously GPT-3 was like, okay, it's here. And that's when we started building what became Ghostwriter. Initially, it was a bunch of tools kind of sprinkled all over the Repl.ide, explain code, generate code here. And then we wrote our kind of autocomplete product, which is kind of like Copilot. As you're coding, it kind of gives you suggestions.

4:22And then we added the chat product and that all became sort of Ghostwriter, our AI suite. It's amazing that this idea of type ahead code autocomplete with local context has become as widely adopted as it has. Like your tool and Copilot are sort of a handful of tools that people like very much accept have changed workflow. Maybe if we just project out a little bit, where do you think that the like next big leaps in productivity in software development come from? in terms of AI? What else is going to happen? It's an amazing tool, but I think it's still very primitive. And we haven't fully explored the full capabilities of even GPT, but also transformer models applied to code in general.

5:08There's a lot to do. I think there's a lot to build on the sort of the layer just around the models. So they're basically how to give models better context, how to give them tools, the ability for the model to actually be able to go out and read a file, install a package, evaluate code, the ability for these models to be more agentic and be able to kind of write entire features by themselves. And so that's all the work just around the model itself with the model in the center. And then there's a lot of work on the models themselves that haven't been fully explored. the way we train models today is we just give it a large corpus of code.

5:49But you can imagine different ways of training models. And we've played around with different techniques at Replit. There are ways to make the models, give models more intuition about how code execution happens as opposed to just static code. Because you're training them on static code, they understand the structure and syntax of code, but they don't really understand the semantics of code. So you can imagine a way to train models where you're not only feeding them code, but you're also feeding them the results of the evaluation of that code. You can imagine ways of training models, not just to be good at code, but also be good at the editor, the debugger, everything that comes around code itself.

6:32And so I feel like we're in the very early beginnings of it. I think for professional programmers, perhaps it's like a marginal improvement. And some people claim it's a huge margin based on our research of how people use our products. We think it's 30 % to 50 % more productivity on coding itself. Obviously, there's a lot of things other than coding, such as debugging, deployment, meetings, being able to translate specs into code. And everything that we do just around the act of coding itself, AI hasn't touched yet. And I think it will touch every part of the software development lifecycle. cycle, I think actually the people that are getting the most amount of value today tend to be the beginners, tend to be the people who don't have the skills typically to start a startup.

7:19And we're seeing a lot of those on Replit just today. We've profiled an entrepreneur that has a quarter million ARR startup in a few months and they haven't, earlier this year, they didn't know how to code. And I think this is where most of the impact is right now. But I think if you project forward, those people will also get a lot more tools and going to get more productive, going to hit superhuman productivity. But the professional programmers, this is where actually you start to have that 2x, 5x, 10x multiple productivity. What do you think is the timeline on that? Do you think that's a year away, three years away, five years away?

7:55I'm just sort of curious about when we both see really massive productivity gains and then also when we move towards this more agentic world that you mentioned where you have AI go out and do things on your behalf in terms of generating a set of features or grabbing a package from somewhere or doing other things for you. My feeling is this is within reach, even in the current capabilities. If you hold constant the current capabilities, this is within reaching the order of 6 to 18 months. We're in the demo era of GPT models. Right now, we're doing the hard work of building infrastructure on Replit.

8:32We are building this service that sits inside the container that embodies the AI in the development environment such that it can do things like go to the internet, like install a package, like read a file, write a file, start another project, get cloned, do all of that stuff. That's a lot of work. And that's not just one model. It's probably multiple models. And so within the current capabilities, there's a lot more to do on the infrastructure side. So I think we're within months of actually having really cool, basic agentic experiences where you can give a high-level task to a model or to an AI such as build a login page and for it to do a decent enough job to give you a starting point for that.

9:18Now, if you try to project the actual advancement in AI itself, increased context size, increased performance, better prices, potentially even net new advancement or new ways of training code models, it becomes really hard to predict. I think software could fundamentally change in the next three to five years where it looks totally different to someone sitting today and looking at it into the future. Maybe it looks something like programmers tend to be more like engineering managers. Like even ICs, you're just like working with employees. And those employees are mainly AI. And most of the time, you're reviewing code and giving instructions.

10:01And you're not actually spending 50 % of your time typing. It's hard to know. But I'm confident to say within the current capabilities, there's still a lot to do over the next six to 18 months. And then I guess one other thing that you have as a real advantage for your product is just you have such user liquidity, right? So you have, I think it's on the order of 22 million users or something in that range. And that obviously can create an enormous feedback loop for training your own models, for reinforcement learning, et cetera. There's a paper that recently came out from Google comparing a very specific type of reinforcement learning through human feedback versus AI feedback.

10:37And it roughly netted out, right? You could start to use AI for certain tasks to effectively train itself. How do you think about that in the context of code and where you're heading? I think it's easy to overstate data modes. You need to be measured about that. I think the real advantage of Replit is actually just the platform, just the end-to-end platform. Because I think you can try to build these code models, but without having the platform to be able to apply it on, to be able to get feedback from, it gets very difficult. But also, even if you just have a code editor or editor extension, that doesn't give you all the feedback that you would want because you would want feedback from deployment.

11:20You would want feedback from execution of the code. And not just user feedback. I think user feedback, you're right. For our LHF type things, I don't think that's the real advantage. The real advantage is the end-to-end journey from the first line of code to the deployment, to getting crashes in production, to making edits, to all of that feedback cycle, which typically in most cases is divided between GitHub and VS Code and AWS and all of that. And the real magic of Replet is putting all of that together. And this is, I think, where we're going to get the kind of richness of training data that allows you to train over a large action space.

12:03And that's really exciting. Can you talk about the decision to train models versus use existing APIs? Like, was it just a latency thing? How did that happen? So when Microsoft did it, like they took GPT-3, they distilled it down, and they hosted it on their own infrastructure. sure they did a bunch of caching, a lot of things like that. If you're someone who's just using the OpenAI API, you couldn't do any of that. And it was very expensive. It was very slow. And even now, there's no completion models. Now all the models are chat models. They're not going to be releasing any completion model going forward.

12:42And so if you want to build a completion-based product, it was actually fairly difficult to do it using commercial APIs. And we felt like we can make a model that's both cheap and fast and good enough for that autocomplete use case. And we found that the 3 billion parameter size is kind of the nice sweet spot where you get enough raw IQ from it, because like 1 billion felt kind of too dumb in a lot of ways. and it was going to be cheap enough and fast enough to host and be able to do a fast inference from. And it was a time after the Chinchilla paper first came out and LAMA had been announced and the idea of just training them longer had just been in the air and we're like, okay, what if we apply all these open source tokens on a smaller model?

13:34And then we applied some more tokens from Replitz data that gave us a 50 % improvement over the model that we open sourced. So we open sourced the base model because we think that's sort of the ethical thing to do because we trained on open source data and we want to give it back to the world. And it was state-of-the-art model was better than most open source code models at the time, even things 20x its size. But the need for it came from a product need. We needed to build that product. I think a lot of founders now see it as a point of pride or status to train a model. And the moment we put out the 3 billion parameter model, like three or four copycats came out after that.

14:15And I'm not a fan of what's happening in the open source world where everyone's competing over these silly benchmarks. I think it's important to start from a product and start from a user and customer need and then go from there. Right now, if we could do what we can do with Llama, we'll just use Llama. If we have to train a model, we'll train a model. And that's how we always approach building this company. is start from the customer and the use case and then solve any technical challenge that you get along the way. One of the interesting debates I've been hearing about Llama recently is how long will Facebook continue to sponsor them as open source models?

14:54And so far, it seems like Llama 2, for example, is the one that a lot of people are viewing as the most performant with Llama 3 perhaps being a big step forward in terms of catching up relative to GPT-4 and sort of other close source models. What do you think happens if Facebook stops open sourcing? Do you think somebody else steps into the void? Do you think everybody's kind of on their own? I'm a little bit curious how you think about the open source world relative to models, given how important Llama has become so rapidly. So when I came to the realization early 22 that we can't build the product that we want to build using commercial APIs, I was kind of depressed for a moment.

15:34It was like, oh, we didn't have a lot of funding or whatever. It's like, okay, Microsoft's going to be the only one who's going to be able to build these kind of products. And I got a meeting with Zuck, actually. And I pitched Zuck on this idea. And I remember around the time he did the Open Compute project. So what Open Compute was, Google and AWS are going to keep all their data center secrets because it's a company advantage. but Facebook did not sell compute, did not sell cloud. And there's a lot of other companies that did not sell cloud. Cloud was a compliment for them, was not a competitive advantage.

16:10And so they pooled their resources together and they worked on this open compute project to be able to optimize computers for data centers. And it was a huge success. And what I told Zuck at the time was like, hey, why didn't you do that for LLMs? Obviously, open source AI is going to be great for you because you can build better products. It's a complement. And so it's sort of the traditional, the classical commoditizer, complement business strategy. And at the time, he just kind of nodded his head. And I don't know if it was a novel idea. Maybe he was already thinking about it. But that's basically what he's talking about.

16:48I think in one of his interviews, he talked about open compute as an analogy for what they're doing with LLMs. So I think it makes a lot of sense for them to do that. What I'm actually surprised by is how little industry participation they've got. And I think the difference between LLMs and open compute is that the AI safety angle makes it a little toxic for a lot of companies to touch. like they wouldn't want to be associated with something that spews something that is toxic or they don't want to touch. So that's another wrinkle that I didn't appreciate. And it takes a certain amount of guts to release an open source language model, especially with political heat that Meta's getting from that.

17:34Now, what happens if they stop? Yeah, I think it would be bad. I think it would be bad for the ecosystem. Will there be another player that will emerge? The problem a lot is that, But as you know, there isn't a lot of guts in the industry. That's our problem. It's like, and Zuck has balls, right? Yeah, no, I think it's been super impressive what Meta has done with Llama. And I think it's really helped push forward the entire industry in an amazing way. And so really kudos to the company for doing it. I think beyond guts, which I agree with, there's also the cash side of it. I actually think the money exists in the ecosystem to pay for open source and to help train models.

18:08I think the talent base is much harder. And the magic of what I think Facebook did with Llamas, they had an amazing talent base and then the capital to train something and then the guts to release it. And if you look at every other open source software wave, almost every major one had a big corporate sponsor, right? Linux was sponsored by IBM to the tune of like a billion dollars a year for some period of time. WebKit was obviously sponsored by Google and Apple. Lots of the really big things that were important in terms of fundamental pieces were sponsored by a company. at one point or another.

18:42And I think in the absence of Meta providing not only capital and guts, but also talent, there may end up being a gap. But that's one of the reasons I was asking the question is they have that really amazing intersection of guts, money, and people. Right. And it's unclear to me what the replacement there is. Yeah, it's also unclear, like you said, which other companies would be able to participate in that. And I'm sure they tried to do outreach and they tried to work with other companies. I think on a long enough timescale, obviously, there's going to be open source models that are as good as commercial models.

19:18But the current landscape makes it hard to kind of imagine how it would play out. And also there's this whole question of like, if the architecture gets more and more complex, it actually makes it harder and harder to open source, right? Like there are tools and there are services. Tools especially are easy to open source because you can, it's a binary you can run. And basically that's what we have today. You know, you can just run it. So that's like a binary blob, right? But if there's a mixture of experts and there are stateful components for running this stuff, then it becomes exceedingly harder to create an open source alternative.

19:58I'm curious if we can talk about bounties for a second. Sure. Because you're somebody who clearly believes in agentic development. And then you stuck a marketplace in your product that allows developers to collect dollars for code that they've produced. Do you think we end up doing that with development agents? Or how does that play out? So the idea behind Boundaries is that at Revlet, we have all this talented people across the world that are learning how to code and that are actually becoming great programmers that no one knows about. And I felt like it was almost our duty to make it known that there's so much talent around the world that you can leverage.

20:44And a lot of these people don't have any resumes, don't have a lot of skills to be able to go get jobs themselves. So part of it was this community building activity. Can we get all these people money? And we have people in Africa, India, Thailand, Serbia, all sorts of countries now that are making money on bounties. The other thing was my view at the time or my thesis that I wanted to test was it's going to be a lot easier and it's going to be a lot cheaper to be able to get basic software tasks done. Because AI, like I said, will have in the short term the most impact on beginners. So beginners are actually getting a 2, 3, 10x multiplication on their productivity.

21:28you learn a little bit of code and now you can actually build a product that makes 200k arr and so the idea was that okay how do we leverage this uh this opportunity how do we monetize it and so okay human plus ai the human who knows a little bit of code plus ai could actually be a really great developer and so yeah turns out it was right you can go to replet and get a get a startup prototype for startup or an mvp for startup for as little as like 50 or 100 dollars and in some cases. My hope as well for Replit in general is that there is some kind of currency or way to transact built into the platform.

22:11And that was upstream of the idea of bounties. Bounties was kind of the first step into that direction. My view is that there's something broken about open source where the currency in open source is sort of stars, right? Like how many stars you get. And it's sort of a fake currency and creates all sorts of weird side effects. We talked about how open source tend to be the most impactful open source projects tend to be corporate sponsored, right? But that doesn't really have to be the case. I think the software economy is missing some native currency for exchange of value. And open source is good and great.

22:55We want more of it. But also we want it to be sustainable. and we wanted the people to be rewarded who are contributing to it. And then there are all sorts of things you can do with a platform that has concept of money and transactions built into that. So if your development environment and your deployment environment has a currency built into it, what kind of collaboration and ways of doing things does it unlock? One idea is that you sort of publish a module or publish a component or publish a service to the network. And you kind of price things per function. And I can sort of call these functions and be able to have a wallet in my application, be able to kind of spend money on the different functions and different databases, different services that I want to compose to create an application that someone can consume.

23:50And AI was an interesting additional component to this because I think agents in the future will also be able to use money to do transactions, do things in the world, right? So right now, our AIs, again, we're using them as these co-pilots, right? They're basically something that's helping you do some kind of action or task. They're predicting the next step. They're literally doing the autoregressive action. But what if we get to this place where we actually have these agents that can act semi-autonomously? What would you want them to do? Obviously, we want them to interact with a wider internet so that they're coding, they're evaluating code.

24:36But you also would want them to be able to spend money and make money. And in the future, I could see people creating bounty hunters that are fully automated. So can you create an AI that can go earn money for you while you sleep? And I think that that would be a really cool idea. And finally, I think on the data side of things, one thing that I thought was missing from the data landscape around code is complete project spec or product spec to actual end product. And so we have this data now where you have a literal, like complete end-to-end project spec and the actual code accompanied with it.

25:18Yeah, we host this college hacker group for my fund for this commit hackathon. And it is very cool seeing how sure some of the like 19 year olds are that AI is going to change development such that they don't really need to like learn to code in the same way before. right? Like, and if you ask them, like, well, do you need to know architecture? And they're like, nah, like, you know, it's more prompt guidance. Like, do you, like, who's going to debug? And they're like, oh, another agent, right? And these are people who, you know, 20-year-old MIT kids that are like designing and building their own GPUs.

25:51They're very interested in computing, but the orientation toward, I would rather orchestrate some of Amjad's like bounty hunter type developer bots is very strong. So I'm excited to see if that happens. I think there'll be some of your first users? I think embedded in their answer is this attitude of like, if it becomes a problem, I will learn it, right? If the AI can't do it, I will learn it. And I think that's the right attitude for hackers is that you learn just in time to be able to do that task. Yeah, absolutely. I'm not going to like move myself up the MIT computer science curriculum hierarchy.

26:30I'm going to go figure out what I want to build and then kind of work backwards. And look, I can jump 20 steps forward because of Ghostwriter now. It's a very cool setup. One of the things that you mentioned was, you know, different aspects of design, payment, different activities, etc. Do you think from a payment perspective, anything needs to change in terms of financial system? Is it crypto? Is it just centralized payments? And it's just the standard old rails? You know, I'm very pragmatic about like technology, like, if Stripe gets us there, Stripe gets us there. If it is Bitcoin, then it's Bitcoin.

27:06I think it's a combination of both. I don't view them as that separate. Actually, one interesting historical tidbit is that Stripe, Square, and Bitcoin started the same year. And Satoshi was behind all three, it turns out. Maybe. It's actually Jack. Yeah, it's Jack and Patrick working together. Right. But, you know, it was obviously in the air, this idea of like the internet is missing this core feature. Money is a coordinating mechanism, right? Like, there's so many cool things you can do with money as a primitive, not only for people to be able to kind of buy and transact and value. But all the things around the people in the Ethereum community that sort of invented with staking.