Vibe Coding: Everything You Need To Know — With Amjad Masad

30:21Is there now oligopoly of a few model companies that are able to create these state-of-the-art models and there's no downward pricing pressure, right? Are there investors starting to demand better business fundamentals? I don't know exactly what's happening. we should talk about the Chinese open source models in a second because I think that will introduce an interesting mix to this. But it certainly is the case that we're not seeing token prices go down. The main reason we went to effort-based pricing is, let me explain what effort-based pricing is. So when we released Replit Agent V1, Version one of Repl.Agent would work for like two minutes at a time.

31:12You would give it a message or go try to do something for two minutes. Either succeeds or fails. It gives you a checkpoint, commits the source code, and charges you 25 cents. And the reason it only worked for two minutes is because the capabilities of the models meant that it can only work for that long. now models got better and we we had we knew that models are going to get better and they're going to be able to work for 10 15 minutes and so with version two of replet agents started in beta in in february came out of beta in april the model would work for for 10 minutes and so we can't charge 25 cents for like a 10 minutes so what we started to do is came up with the heuristics, every nine tool calls will do a checkpoint.

32:06And so as it's working, you'll see it make a checkpoint, checkpoint, checkpoint. That's a hack, right? That often means that if you make a small change that costs us five cents or whatever, you still get 25 cents. But also, if you make a big change, you might be costing us a lot more than what we charge you. So it was really out of whack. Now, that was a hack and we need to move to a place where we're charging the user proportional to how much the model is working and the cost on us. And we think that's the best way to create a long-term sustainable business. And when those two things are aligned, also opens up new opportunities where when we do optimizations, we're always optimizing.

32:57We actually had like 20 % optimization on cost recently. We pass it straight to the user because now cost and price are tracking with each other. What happened with our community, the first thing that happened is there was a sticker shock. So you're used to seeing 25 cents every 10 tool calls. And suddenly you're seeing$1.5 or$2 after 15 minutes of work. So that's one. Two, it's true for some users who are really advanced, the cost has gone up for them because the projects are bigger, the contact size is bigger, their workloads are bigger. But early on in the project, it's actually cheaper. You mentioned that you worked for an hour, you didn't have to sign up for the core paid.

33:48We give free users$3. So you work for an hour on$3. Not bad. Yes. It's cheaper than a developer. It's cheaper than a developer for sure. And so that being said, we recognize that on advanced users, it is almost there's a tax as you go on. So we're trying to optimize the context window and make sure that advanced users are not getting more expensive experience. The other thing that happened is we introduced thinking mode, reasoning mode, and we introduced high power mode. And people are enabling those. And sometimes they forget them enabled. And now we actually start to hide it under advanced. Don't enable this unless you know what you're doing and you want more power.

34:32And there's like a 5x multiplier on it. So a lot of people are enabling those, getting these large checkpoints. and we're like, we put out content, we put out a video, we put out some documentation, a blog post. Here's when to use reasoning mode. And you should always have it on. So just describing all of that that's happening, there's a macro trend in the application space where a lot of companies were subsidizing the cost of, like a lot of companies were paying more money on Thropic and OpenAI than they were making. Was that where you were doing that? And on V1, no. On V2, yes, because the pricing model was out of whack with how we're charging.

35:19Actually, the median cost per checkpoint kind of went up only a little bit. So on the lower end, we're charging user less right now. But it used to be that on the lower end, we're charging users more. On the upper end, we're charging users less. So now it's more proportional, more fair for both. And so now we have solid business fundamentals that allows us to grow. And I've been talking about how Replit has been my mission, my passion for eight, nine years as a company, 15 years as a side project and a vision. and we're not trying to rapidly expand revenue while losing money in order to flip this company, to sell it.

36:05We've seen all these acquisitions or raised like the next big round. We're really trying to build a business for the long term. And Replit is made of all these different components. So we have costs, not just on AI, we have costs of traditional compute, CPUs, storage, databases, all of that stuff. So kind of to summarize, I've talked a lot about what was happening specifically in Replit. I don't know what's happening in Cursor. I think for sure that their situation is a little different because their dynamics is – I think they actually did raise prices. You should talk to them, but I think it's a little different dynamic than what happened in Replit.

36:49to summarize there is a concerning trend where token prices are not going down is that going to be the case for the future because that sucks because we want to be able to use more tokens to create more intelligence to be able to create better applications for users is that going to be the trend forever are we reaching a steady state in cloud for example will kind of reach that steady state. When you have a monopoly, there's no pricing pressure. But when you also have an oligopoly, they not intentionally without talking start colluding. Because it's like a market dynamic where it's like, if you don't lower a real price, I'm not going to lower my price.

37:31It's not in our incentive as a whole because we own 25 % each of the market. Okay. I do want to ask you about something that you didn't mention when you looked at the different factors for why prices might not be going down. There might be investor pressure. There might have been this equilibrium reached. Or is it possible that these models have just gotten so big and expensive to run that the fundamental economics of AI are just not working? So explain why. You can surmise the bigness of the models based on speed, token throughput. It's not perfect, but if you remember GPT 4.5, GPT 4.5 was an experimental model from OpenAI.

38:19It was the idea, let's train a trillion parameter dense model, meaning it is not sparse, meaning all the neurons are activated on every request. And it was so slow. It's really hard to run these things. The new models, even when they're big, they're sparse models. They're called MOE, mixture of experts. So in every request, there's a router layer that takes it to the expert part of the circuit in order to answer that question. So there are models with trillion parameters, but any given request is 32 billion active. And that's like a kind of small model. and what we're seeing based on speed and things like that is actually probably the models are getting more efficient.

39:06I mean, DeepSeek showed that the models are getting more efficient. And if DeepSeek open source was able to make it, you better believe that the labs are also getting more efficient. Okay. I do want to speak with you about DeepSeek and Kimi K2 and other Chinese models. Let's do that when we come back from the break right after this. Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points and miles. Now, imagine a credit card with rewards that can grow in value. With the Gemini credit card, you can earn Bitcoin or one of over 50 other cryptos instantly with no annual fee.

39:40Every swipe at the store or gas pump earns you instant rewards deposited straight to your account. Plus, sign up now for a$200 Bitcoin bonus to kickstart your rewards. Visit Gemini.com slash card today. Check out the link in the description for more information on rates. Again, if you're looking to invest in Bitcoin but don't know where to start, the Gemini credit card makes it easy. The Gemini credit card is issued by WebBank. In order to qualify for the$200 crypto intro bonus, you must spend$3 ,000 in your first 90 days. Some exclusions apply to instant rewards in which rewards are deposited when the transaction posts.

40:18This content is not investment advice and trading crypto involves risk. The Gemini credit card may not be used to make gambling-related purchases. Shape the future of enterprise AI with Agency. A-G-N-T-C-Y. Now an open source Linux Foundation project, Agency is leading the way in establishing trusted identity and access management for the Internet of Agents, a collaboration layer that ensures AI agents can securely discover, connect, and work across any framework. With Agency, your organization gains open, standardized tools and seamless integration, including robust identity management to be able to identify, authenticate, and interact across any platform.

41:03Empowering you to deploy multi-agent systems with confidence, join industry leaders like Cisco, Dell Technologies, Google Cloud, Oracle, Red Hat, and 75-plus supporting companies to set the standard for secure, scalable AI infrastructure. Is your enterprise ready for the future of agentic AI? Visit agency.org to explore use cases now. That's A-G-N-T-C-Y dot O-R-G. And we're back here on Big Technology Podcast with I'm Jeff Massad, the CEO of Replit, talking about all things AI code, vibe coding. And now let's talk about these Chinese models. So this episode will air a couple of weeks after the emergence of Kimi K2.

41:43But we're talking about Kimi K2, which is another Chinese model. And of course, this DeepSeq moment was a big moment where we found out that this seeming small hedge fund in China with some GPUs was able to engineer a more efficient model. That story will be debated about what actually happened for a long time. But let me ask you one influence of DeepSeq question, and then we'll get into the others in Kimi K2. So you mentioned before the break that Western models have taken after DeepSeq. So do you think they learned what DeepSeek did and sort of put those new innovations into play in their own models?

42:22Or was that coming anyway? From what we've seen from the Twittersphere is that it seemed like there were some surprises because researchers just talk a lot. It seems like there were some fundamental innovations from the DeepSeek models that weren't known in the West. But have they implemented those now? And that's probably why we're getting more efficient models. Like, yes, I'm sure. Like, the models are getting more powerful without getting slower. All right. So tell me about Kimi K2. When Anthropic came out with Sonnet Quad 3.5, that was a fundamental shift in the industry where the models got a lot better at coding.

43:05and suddenly instead of making small snippets of change, Sonnet could generate entire files and enable things like Cursor Composer where it was a start of Vibe coding where you can put in a prompt and generate entire files and all of that or generate large edits. Then Sonnet 3.5v2 was the first model. It was a computer-use model. was the first model where you could sense that there's agentic, true agentic behavior. I don't know what they did. They cracked RL, whatever happened there. You can give a model a VM and it can give it a - Virtual machine. Virtual machine. You can give it an objective and it can slew the round in the virtual machine, look at the files, run some commands, and then write a program, test it, and then solve the problem.

44:01that experience, there's a benchmark called SWE bench, software engineering bench. And you start seeing the score going up dramatically. I don't know. I think we were at like 10 % last year. And now we're at like 70 % and 80%, 80%. World-class coding. The interesting thing about SWE bench is it's not just coding because there are other benchmarks that just do like the code generation. right? Sweetbench, I think the harder thing about it is the agentic workflow, is writing the code, testing it, running commands, finding files, understanding files. And this stuff was like a huge jump that happened with Sonnet 3.5v2, then 3.7, then 4.0.

44:51And they've, you know, kudos to Anthropic. They've been able to make, create a lead that hasn't been bridged by the other labs. Gemini is getting there on the agentic stuff, but I would say OpenAI kind of lagged behind. O3 has some interesting agentic capabilities, especially around deep research, but it hasn't been as good as the other models on this agentic stuff. I mean, they did some interesting stuff with Codex. I don't know if those models are in the API, but everyone is using Claude for the agentic coding experience. The interesting thing about Kimi K2, I would say, is they caught up not to Klotzon at 4.0, perhaps Klotzon at 3.7.

45:37At least that's the vibes right now. Before the other labs. Wow. Yeah, I think that's really underreported. Again, this is vibes. Everyone's trying to figure it out. But it looks like it has a really good sweet bench. It is doing 65 on sweet bench. Sonnet is 72. 72. If you do sampling, which is for every step you ask the model to generate n number of solutions, you can get up to 72%. It can be competitive with Sonnet. And this is with export controls? Yes. And I think in the paper, they talk about the solution is scaling reinforcement learning. We also saw that with Grok4. Grok4 spent as much on reinforcement learning as they spent on pre-training, which is unheard of.

46:29But that's an important point because with that big spend on reinforcement learning, Grok is a competitive model, but they spent billions, hundreds of millions on RL, which is this goal-setting form of training. And it's not like it's a new category. So it shows there are some limits. and XAI is an amazing team. And they've been able to achieve so much in so little time. But it's also well-known in the industry that they're computer inefficient. They're so compute rich that they're throwing computer at the problem in many ways. Yeah. So what is the significance that Kimi K2 is now as good as some of these anthropic models?

47:12A small research lab. I think the rumor is like the 200 people. Again, there's export controls as well. was able to figure out how to catch up to near state-of-the-art agentic coding models before big Western labs that are highly capitalized, a lot more researchers, was able to. And does that mean then that they can undercut them on price? So let's see. Right. Are you going to integrate Kimi K2? We're looking at it. We're looking at it. So far, we're impressed. so far we're very impressed so I mean look these things sometimes they overfit to certain things and I would say it requires a month from the entire community to really have consensus over whether the model is really great and similarly with Gawk 4 I think a lot of people are playing with it but my sense is that it is good enough and again the economics are so good that that you can expend more tokens to get more intelligence.

48:24So it is not at the frontier, but it is near frontier. But given that it's cheap and fast enough, you can spend more tokens. That creates some more interesting potential for us to create new capabilities in our platform because it is cheap and fast. How much cheaper is it than the Anthropic models? Man, I am bad at this, but I would say, I don't know, one-fourth maybe? Oh. Yeah. That's on their official API. Perhaps more even. I forgot. Maybe you can look it up after the show. We're going to have to – this show is going to come – it will come a couple weeks after we record, but we'll have to release this segment early because that's astonishing.

49:10One more question about Anthropic. I can vibe code and clot. I do it all the time. And they also have this cloud code product where people are writing prompts, getting code. Are they your competitor long-term or how do you see them on that front? Because that's the question is eventually do the labs just subsume everything else that's built on top of it? I think the question is for them, right? Like you should ask – I know you're going to talk to Dario. You should ask him the question. Listeners, viewers, this will air a week after Dario, but I'm about to after this go in and speak with him. So you might see this question a week earlier.

49:47Yeah. So look, we're committed to our relationship with Anthropic. They're a great company to work with. We have a great partnership. And it's not like we didn't anticipate them wanting to build products in addition to the models. Every model company is building products right now. The thing that they're going to have to manage is their pricing. If they're going to compete by undercutting everyone on price, they're going to destroy the ecosystem. I think Replit right now has the advantage of this platform that we built over eight years, that it's going to take a lot of blood, sweat, and tears to build, and also the user experience that is focused on that sort of non-technical user And we really care about this idea of empowerment.

50:44Right now, Cloud Code is used by developers and loved by developers. And I think they're competing head-to-head with Cursor, Windsurf, and those kind of products. Whether they're going to move into our space, again, you should ask them about that. But I think a more interesting question, how do they want to nurture the ecosystem versus just go and – because they can compete on price. They can steamroll everyone. Right. I mean, Cloud Code is – the max package is$200 a month, and you see developers getting thousands of dollars of API value out of that. This is not good for the ecosystem. You must notice this.

51:26Yeah, yeah. Not good for the ecosystem. I don't think so. Why? Because, again, you're competing on price, not how good the product is. And there's a price at which maybe the quality doesn't matter as much as how many tokens I'm getting, although Cloud Code is a really good product. But then Cursor, no matter how good they make the product, they're still going to be more expensive and a disadvantage. and people are like, well, I really like cursor, but I can get 10x more value out of cloud code. And so the marginal gain in product quality will not matter as much. Right. And that will destroy the ecosystem.

52:13Fascinating. I mean, I think that this question is just one small question or one version of a big question we're going to be asking as these AI models get bigger and better and more intelligent. So I want to spend the rest of our time talking about some philosophical questions, if that's okay with you. There's this idea that the AI research houses want to use the code that they generate to sort of – or these coding applications to speed up the development of the next model and compress the time it takes to get better models. People call it an intelligence explosion or things of that nature. Do you see that as feasible and is that something we should want?

52:55So you should think about what are the limiting factors to the next version of a model? What are the bottlenecks? Where does that innovation need to happen? I can think of a few areas. One is research. So this is algorithmic research, like figuring out the next algorithm, next improvement. in training algorithm, in inference algorithm, whatever it is. And then systems engineering. These training runs are massive. That requires a lot of interesting distributed systems engineering. Will AI coding help with AI research? On the margins, perhaps? Perhaps they can spin up Python notebooks faster. I don't think it's that impactful.

53:52The models can't do AI research, can't come up with ideas and test them really quickly. Will it help with distributed systems? Perhaps it is not as impactful right now on writing Rust code or C or Go, whatever, as it is on JavaScript and Python and higher-level languages. and like I said it requires a little more precision and better system design and that the bottleneck to really good distributed systems is design and not like the amount of number of codes you can generate which is more true on the product side you need to generate tons of CSS and JavaScript and try a lot of things and delete a lot of things and iterate and do A-B tests and all of that stuff so like volume of code is important there.

54:44I would say on the backend distributed systems, I don't think volumes of code is. So I'm reasoning in real time now. And I guess my answer would be, I don't think it's going to have anything more than, you know, marginal improvement on speed to the next model. All right. I guess that makes me rest a little easier then. By the way, just on a, you know, You speak with a lot of people in the AI industry. Of all the economic activity in the AI industry today, how much of it do you think is code? Just a rough guess. Someone actually made that slide that's been going around. I think it was something like 1.1 billion of ARRs in the AI coding and VIB coding space.

55:30Okay. So it's actually kind of small compared to the total revenue. Yeah. So Anthropik has$4 billion. Right. ARR. Yeah. $4 billion ARR. Let's say they also have their own products, their own coding products. I don't know. Let's say$1.5 billion off of that is AI coding. It's substantial, but it is not the entire thing. But then you have$10 billion of ARR on open AI side, and that's more consumer. Now, on the rush to artificial general intelligence, which we've talked a little bit about, do you think Silicon Valley is the one that should sort of possess this or be the one that controls it? I mean, it's an interesting place.

56:13There's a lot of kooky ideas here. And it seems like if this is possible, it's going to be something that's controlled by or owned by one or more of the labs here. Is that good? And assuming it'll happen and assuming one company will reach their first and have some kind of advantage or monopoly over AGI, which I'm not entirely sure I agree with this assumption. But if you want me to make this assumption and then answer the question, I'd be happy to. But I just want to get clear that - Yeah, let's make those assumptions. Okay. I know there's a lot of things that need to happen in order to get there.

56:50Yeah. I might have some fundamental disagreement with this assumption. Wait, talk through the disagreement. I don't think AGI is any point in time, for one. And I think there's going to be, right now, the distance between any lab is just an order of a few months. And I think that really matters. Between O1 Preview and DeepSeek was like two or three months. between, I mean, the biggest one was this Kimi K2 one that we just talked about that was like maybe nine months or something like that, but it's still sub one year. And so whomever reaches AGI first, they're not going to go into intelligence explosion and just like suddenly superintelligence gets born.

57:36Other labs will catch up really quickly and then there's going to be a lot of models. I don't think it's going to look that different from the ecosystem that we have today. And if you assume that AGI will actually have an impact on model development through research and speed of development, then everyone will get the benefit of that as well. And so actually you might get even more competition once you have AGI. So I don't think it's going to be a monolith. Okay, but if it is? Okay, if it is, would I want Silicon Valley, and I guess it's like a moral philosophical question I wouldn't want any human being to we're all fallible that's why markets work that's why that's how a human society evolved over time it is you know Darwinian evolution and free market capitalism it's all based on competition and the idea that one system would be this model controlled by one human being.

58:49We've seen disasters and massive human suffering happen when there's this top-down sort of Leviathan type thing, whether it is in Soviet Russia with all the deaths that happened there or in China or whatever. And oftentimes, like as I understand it, in the Soviet era, they had this kooky idea about evolution. I think what was it called? Lushenko, Lushenkoism or something like that? I'm not familiar, but I'd love to hear the explanation. Yeah, so basically they had, they thought that evolution is this bourgeois idea. You know, communism has this idea. It was like anything that's high class bourgeois is wrong.

59:40And so they have this ideological view on how evolution works or should work that led them to do agriculture in the wrong way and led to famine and that sort of thing. And so oftentimes they do kill people and cause mass suffering, mass poverty, even if they don't intend, even if like outside of the gulags and all the other oppressive, explicitly oppressive system, those systems are inefficient because they have these wrong ideas and there's no competitive pressure to have better ideas. And so that's fundamentally broken static system that doesn't improve like competitive systems. And I think if we have a super intelligent monolith controlled by a single company or single human being, it's bad.

1:00:38It's fundamentally really bad. I agree. All right. Last question for you. We're seeing a lot more AI love bots come out. Is that a good thing or a bad thing that people are going to fall in love with AI more often? It's a bad thing. Like a priori bad thing. Like the reason humanity grew and flourished and all of that is because we have babies. And anything that takes away from that, especially given the fertility rate is so low right now, will potentially lead to really massive problems, especially since capitalism is based on large middle-class consumerism. Like the current instantiation to have the economy work requires that, requires taxpayers to fund Social Security and like elder care and all of that.

1:01:41The welfare state is based on this large young population. And when that starts to collapse, you're going to have massive instability in these systems. So even if humanity doesn't go to an extent like Elon would say, although Elon is the first person to create a really interesting mass market companion, I think, right now. Interesting is a fun word for it. It looks like it's really compelling. I see people right now talking about it on X so much. It's got some work. Yeah. But these type of things are going to definitely become real partners to people. people when this technology has been bad or hardly workable have gotten married to them right before llms so it's going to happen again and in greater numbers hey the question is i wrote this uh i used to do like more creative creative writing um i i wrote this uh this uh essay on the hyper real.

1:02:47So I think it is like French post-modernist theorists like Bolliard wrote about this concept of the hyper real. And the idea is like we have reality like you and I are interacting right now. And then you have media created realities. and the reason sometimes it is hyper real, it is more intense than reality itself and more enticing than reality itself. So, uh, you know, even in real things, you know, for example, um, when you, when you get a, when you eat like a, I don't know, a Twinkie or something like that, like fatty, salty, sweetie, kind of a snack, it is like, it is not like a piece of chicken or beef or whatever, it is this hyper real thing.

1:03:39It hyper engages your senses and it makes you addicted to it. And similarly, social media is hyper real in a sense that I can go there and get a lot of social interaction, tweet something, get hundreds of likes. And it's much easier than going out in the wild and finding a hundred people that could like me. And so we have these technologies that are, and the market around it that is bootstrapped to make us addicted because there's so much more enticing and low effort than the reality that we know and experience day to day. And I think that is a huge danger for the existence and evolution and longevity of human civilization.

1:04:38and I think it is I talked about how good free markets are how important, how competition is important this is one thing that capitalism is so adversarial to humans at and so I don't have a solution for it I think in the past the solution was religion for example in Islam you can't depict humans or animals in art That's why in Islam, the art became more geometric. And if you go visit the mosques or whatever, they have all this geometry or calligraphy that's really interesting. And I think part of the idea there is, I think, the hyper real. Like if the ultimate expression of something so enticing is a virtual being like we're seeing right now.

1:05:36And I'm not saying Islam had the foresight or whatever, but I think religions used to have this built-in mechanism to protect against these predatory consumer products. And I wouldn't know how to solve it in the future, but perhaps it is potentially societal, maybe governmental. I'm always kind of skeptical of that or religious kind of protection. We're going to need something. Yeah. So Lord help us. I'm Judd. Great to see you. Thanks so much for coming on the show. My pleasure. All right, everybody. Thank you so much for listening and watching. We'll be back on Friday to break down the week's news.

1:06:19Until then, we'll see you next time on Big Technology Podcast.

1:06:36You