First Time Founders with Ed Elson – The AI Company That Codes For You

24:55That's actually because even GitHub, Microsoft's product, is not vertically integrated. They are relying on external models to build their application. And we view that as, hey, the model and the product and the infrastructure is so critical to delivering a great experience. Why would we not have control over every piece of that? My understanding, you guys don't use the actual data of your individual users. So how is the model getting trained? Yeah. Yeah. So we actually do sort of two different things. And I'll let Jeff add on how this affects our enterprises and the customers that use the product.

25:28So first of all, we use permissively licensed code that is available on the public internet. And we also attribute it on generation time. So we actually take sort of copyright and licensing very seriously as a company. But on top of that, when we release product from our user data, you're right. We don't take the data from our users, like let's say when they're auto completing stuff and copy that code and put it into our training site. But we can see, hey, users are accepting these types of suggestions more and these types of suggestions less. So we have preferences on what users and humans like.

25:58And that actually informs us to actually build products that are better and more willing to use. And then that actually has a virtuous cycle in that now people are willing to try more complex things on our product because the easier things, they have high confidence that they work. And suddenly the frontier of what we are actually able to experience as we are able to give the user increase more and more, Largely because we have a product that is so well beloved. We now have over 600 ,000 users that use our product. Yeah, it's unbelievable. I think one thing we might have glossed over in the beginning was Codium made a very conscious decision to make the product free for individuals.

26:30And if you think about, you know, Varun just said, Copilot loses$10 to$20 a month per user. That's after they've been paying a subscription too, right? So having that infrastructure background, being able to make it efficient to deploy these models and then giving it out for free for individuals allowed us to build this very large user base, probably the largest user base for a coding assistant. That's free. I mean, just the data here, you started out last year with less than a thousand users. By the end of the year, you had half a million. Yeah, and we have over millions of downloads across all the plugins.

27:01And the reason why that's very important is because of what we just talked about. If we roll out multiple models, if we are changing the temperature or the thresholds here and there, we are getting so many signals as to what is the appropriate settings to tweak. And I think every hour we're getting like a million signals. So we can run all these experiments on these free users of all these models we train to really, really get the best results that you could possibly get. And then only when we've validated, like, okay, we've trained this model. This is better than all the other ones we've trained.

27:30These are the settings that make the best results. Then we could deploy that to our on-prem or enterprise users, right? Because we can't, after we've deployed it, we can't really get that much more information from it. It's completely, it can be hosted even in an error-gapped environment. So I think that's a big element of why we are successful, of being able to deploy these models. Because if somebody's trying to start from scratch right now, how are they going to know that their model is good? How are they going to tweak the model, right, without a very large user base? So that's part of the secret sauce, is having that free user base.

28:00Yeah, and it's still free, which is what I find pretty fascinating. And you said, I was just reading your mission statement, you are, committed to having a free tier forever. The natural next question is, you know, how is it going to get properly monetized and how are you going to maintain that free tier into perpetuity? I think people underestimate kind of the demand for both just the on-prem instance, but also some of the Teams features we add on the SaaS product. So we have a free individual tier, but if you create a team and add onboard users to it, that is a paid product. And there are things like analytics and actually bigger models and seat management that are going to be better than the free product.

28:42But we are committed to making the free product the best coding assistant out there, no matter what. So whatever other coding assistants come to market, we will make sure our free product is still the best one. We want to make sure, you know, disincentivize others from entering, but we want people that always have the option. We don't want them to get forced to buy a copilot, for example. And then maybe one thing to add to Jeff, like we monetize enterprises, right? So we We have some of the largest Fortune 100s, have thousands, even over 10 ,000 developers on our product. And those companies, obviously, they want security guarantees.

29:11They want personalization for many, many repositories that exist. And they also want support across all source code management tools. Less than 10 % of Fortune 500 companies are on GitHub Cloud. And that is the competitor that we have. That is GitHub Copilot. And they have committed to making their product differentially better if you are on GitHub Cloud. Right? So we want to take the approach of almost being Switzerland. We don't care what programming language you write. We don't care what IDEs you use. We don't care where you store your source code. And ultimately, we also just don't really care what seniority the developer is.

29:46We will provide the maximum amount of leverage there. Whereas a lot of the larger players in the space are focused on being tied to another brand. And the reason why we don't think that that makes sense is we think AI is such an up-leveler. We think it deserves to be in a category of its own. You recently raised$65 million in a round led by Kleiner Perkins. It valued you at half a billion dollars. Congratulations. What is that money going to be used for? I think the way we would like to think about it is we have ways of spending cash, not only to train models, but to also make it so that we can build a better user experience for the end user.

30:25But one of the cool things for us is our product has such high ROI that we think that there will be a real payback period on that. Enterprises and companies will see enough value there that they will be able to eat the cost that we will need to spend up front. But also on top of that, we want to spend a lot on making sure that we can become better partners for our customers. We're onboarding some of the world's largest companies and we're onboarding tens of thousands of developers. That's going to take a little bit of effort to make sure that we do that properly. One company that we're working with right now that we have a multi-year engagement with, they account for 0.15 % of all developers in the world.

30:56Just that one company, right? So, you know, I'm a little bit of a different type of founder in that I do not like the idea of spending money unnecessarily. But if it comes down to we are doing it because we make our customers more successful and our users more happy, we'll do it any day. How have you guys restrained yourselves in terms of spending? because, I mean, yeah, the story, the narrative in AI has been that it is an arms race. And the thing that I hear about in the venture industry, or at least what AI founders are being told, is just go out and raise a shit ton of money as much as is even possible.

31:32One, because you just want to develop a war chest. And two, you want to get the headlines. You want to be the AI company that's working on cogeneration, the AI company for finance, whatever it is. So I guess sort of two questions for me here. One, is that accurate to your experience? And two, how have you been so, you know, responsible in terms of spending while you see all of these headlines of other companies spending so much money on talent and trading their models? I think Varun mentioned earlier that we run very lean. And the reason we're able to do that is we hire people that are generalists or ex-founders and they're capable of doing many job roles at once.

32:12So our company size is probably like a little misleading. It's probably way more effective, not just infrastructure, but the headcount also. What is the headcount? So right now we'll be almost about 55 at the end of the month, I think. And the thing is like the people we hire are able to just slot into different roles almost at a moment's notice. It's like, oh, we don't even have a marketing team, for example. The product's growth has been organic, but we do want to do some marketing experiments to make sure that we are getting ourselves out there, just as an example. And then people, you know, randomly, someone will be like, I have an idea.

32:42Okay, go do it. And then we all of a sudden have a Google ad strategy. All of a sudden we have a bunch of blog posts. We're on podcasts like with you at Elson. But I think that my point is part of the function of not spending the money is just being very, I guess, practical of where the money goes and running lean. And I think when there is a moment that says like, hey, we need to train a much bigger model and we need to spend this much money, we're all for it actually. We actually are very conscious about what the ROI is of everything we do. How do you maintain that culture of leanness and what would be your recommendation to other companies that are maybe not as big as you, but trying to be?

33:20This is probably the hardest problem that we're trying to solve right now. Because we want to hire people that are very good, very technical, maybe they're generalists, like I said earlier, but it's very hard to hire those people. So actually, this is probably one of the things we focus on in the next month is what is our recruiting strategy? How do we hire the best people? Maybe part of that is getting Codium's brand name much more aware. Maybe it's like a big push of like user adoption. Maybe we're just going to have to be scrappy and be very creative of how we hire people. For example, I don't know if other companies are just like pinging every ex-founder on LinkedIn, but we are, right?

33:58So we are trying to scale with creative means. One other thing is like as a company, I think culture is, we have some cultural principles and we run lean as one of them, but you know, who would want to say you don't run lean? So I think, I think how do you actually live that out? We are a five days a week in-person company. So we don't do remote work. We don't really do hybrid work either. So people see what it's like to work at the company. And, you know, until very recently, our CTO was ordering snacks. And that's not to say that's a great use of his time. it's more just that no one is high enough to not do some work to test a hypothesis and we don't hire specialists at the company until generalists outgrow that role and i'll give you another example of this you know when we ran that gpu virtualization company even though we were making money we never hired a sales rep and that's not because i don't believe in enterprise selling no we have a great vp of sales now at the company and i think we might have in terms of talent density one of the strongest enterprise sales teams in the world, actually.

34:57But why didn't we hire someone? I just didn't believe that if we added one new person, I would be setting them up for success. Because the reality is, if I could not get$1 of additional sales, I cannot expect someone else to get$10. This is one of those things where we have a mentality of, we try to do it ourselves, and then we try to eliminate ourselves from the role. We give away our Legos, we let someone else take that over that understands the role much more, but we don't do things prematurely. And I think there's a tendency across people that the idea of building a scalable organization is really valuable.

35:31And I do see that, that you do want to build a scalable organization, but sometimes people get too excited about this notion of org building or fundraising rather than the idea of having customers, having users, because ultimately people that join our company don't care about how much money we've raised as long as we will survive. And our customers genuinely don't care. Let's look at it this way, right? If you look at a company as big as JPMC that makes hundreds of billions of dollars, to them, does it make sense if we raised 100 or 200 million? It all looks like peanuts to them. It's all like 10 basis points of the amount of revenue that they make a year.

36:03So to them, what they really care about are companies of this size is, are we the best partner for them? Are we the best product to them? And as long as we're laser focused on that, we should do whatever it takes to build that up. We'll be right back.

36:49We'll see you next time. baking soda, and parabens. So you can give your underarms the same level of care you give the rest of your lovely self, body, and mind. With scents like cucumber and green tea, dragon fruit and coconut cream, and yes, coconut and pink jasmine, Dove Aluminum Free Deodorant offers freshness that won't leave you stranded. So the next time you're setting out to explore the world, don't forget to invite Dove Aluminum Free Deodorant along for the ride. Because you were made for adventure. And your deodorant should be too. The search is over. An aluminum-free deodorant that won't let you down.

37:23Find your one true Dove. Choose Dove aluminum-free deodorant for 72-hour freshness and odor protection. Buy online at Dove.com or at a retailer near you.

37:48With LinkedIn ads, you can target the right people by industry, job title, and more. Start converting your B2B audience today. Spend$250 on your first campaign and get a free$250 credit for the next one. Get started today at linkedin.com slash campaign. Terms and conditions apply. Tuesday on NBC, Jimmy Fallon and Bozema St. John host the highly anticipated new competition show. I hired 10 creatives from all walks of life. They will be battling it out to see who can impress the world's biggest brands. This is a huge opportunity. This is the battle for the next big idea. This is not Play Play. We're spending millions of dollars.

38:27I'm so excited to embark on this adventure with all of you. Make the best idea win! On Brand with Jimmy Fallon. Series premiere Tuesday on NBC.

38:41We're back with First Time Founders. Sort of a more personal question. I mean, you started this company in 2021, sort of just as the AI hype was bubbling up. And you now find yourself at the epicenter of the hottest industry, and you are one of the hottest companies in the hottest industry. Just a personal question for both of you. How does that feel? Like, what has it been like getting used to being the guy in AI? I told this to the company. I tell everyone just get ready to get get destroyed assume that something very bad is going to happen always and this is where us having gone through that pivot is very critical things are going very well for us as a company a lot of the reason why we haven't spent a lot of money is now we make money which is a unique property about a lot of companies apparently in this space where most companies talk about vision rather than actually building a product that people use but I just I tell everyone, get ready for something really bad to happen.

39:40And this is why it's very important that we hire people that are truly in it for the long run. And I tell people this when they join the company, I think we could be a company that's worth over$100 billion. I think we can. And that's largely because of the total addressable market of what we're building and the amount of impact that this can have, given how important technology is, could be massive. But also a series of bad decisions that we make could completely kill the company. And that will happen very fast. I think this is where us building a very truth-seeking company, and that actually is very hard because people want to believe that what they're doing is correct, and they want to embrace psychological safety.

40:20And what I tell people is, hey, if you feel something is wrong, lean into what you think is wrong and tell everyone. Tell everyone because we should not have pockets of people that want to report up the chain and tell their manager or tell me we have a very flat company or tell me things are going fine. I would much rather hear everything is on fire and have paranoid people at the company than people who are just happily going to work. And this is why I think startups are so much harder than big companies. It's actually not that you're taking a massive risk on the monetary side. You still can make a six-figure salary, right?

40:55These are not people that are living hand to mouth. But the really hard part is the lack of psychological safety. We make a bunch of people at the company make a series of bad decisions and the entire thing can go to zero. Whereas if you're at Google, you have very little accountability. If your team doesn't perform well, Google makes so much money that you are a rounding error. You will be shuffled to some other part of the company. You never really need to deal with the impact of your decisions and consequences of your decisions. And that's why it's always a little bit of a funny statement when someone at a big company is like, I work at a startup at a big company.

41:29No, no, you don't. Imagine, imagine the idea of you potentially losing your job every quarter or every month. Exactly. One thing that the way I think about it is radical transparency. And we have a lot of conversations within our company of, you know, let's be super transparent about everything. And even in my personal life, I'm like, I'm just going to be like totally upfront. That's the cleanest, most kind of hygienic way to operate. Do you ever feel that there could be too much truth? Do you feel that there's a possibility that, you know, if you're encouraging everyone to tell the truth, tell the truth, be transparent, tell me everything, that they'll kind of go overboard?

42:12I think this is the hardest part about a startup. You know, the two things that I say is a startup is really hard because you need to be both irrationally optimistic. Because if you're not optimistic, the answer is always Microsoft is going to beat you, right? It's the biggest company of all time. They have the most capital. They have the most people. They have the most distribution. Why does any company win? But clearly that's wrong, right? There are companies that have beat Microsoft at different areas. We always say this, oh, they have the most capital they're going to win. It's very easy from my perspective to say that.

42:40Exactly, right? And then HBS, I try to think about what Harvard Business School would say 10 years from now for us as a company. And I tell people this, it's like, you know, we fail. And what they write is in a world in which technology was changing, Microsoft had all the distribution in the world. They had this property called GitHub. It was inevitable that they would win. And because of that, they won. They won the entire space. And all of these startups, it was a fool's errand. Why did they even try? And then the world in which we win, they were going to write in a world in which the technology was getting disrupted so materially, there were a set of companies, and one of which was Codium, that had such a technological advantage that there was no way a slow-moving gorilla like Microsoft could even compete.

43:24So what I think is very hard. Yeah, exactly. They will write whatever the future looks like, and the history will be written by the victor, and no one will know exactly what the pages of the book look like. And I think the hardest part about a startup is you do need to be irrationally optimistic to believe that you can win. Because by default, if you don't, you will definitely lose. But then also uncompromisingly realistic, which is that sometimes actually there's no point continuing in a particular direction. This is where I think it comes down to what you just said, which is that if you are too truth-seeking and everyone is paranoid all the time, it could lead to paralysis.

44:03And I think there's a fine line there. There's a fine line where one time we lost a deal and we We actually, every day for dinner, talked about it for two weeks in a row for multiple hours. We talked about the implications as a company, and that was very useful. But if we extended that out to an entire year, we would not go anywhere. So you're totally right. There's a level here. But what I do feel most companies do is probably err on the side of not being truth-sinking enough. They become too complacent with what they're doing. And I think the thing to really think about, and this is not true at a big company, which is why you really need to think about it as a startup is you do not win an award for doing the wrong thing for longer.

44:42So the sooner you can rip the bandaid off, the better your company will be. You will be way happier that you did it. It is going to be incredibly painful for like a week, but just do it. I love that. You guys have been generous with your time. So we'll begin to wrap up here. Jeff, I'll ask you this question. What do you think has been the greatest challenge that this company has faced in the past couple years? I think the pace at which things switch is really challenging. We relied on some technology features to be the selling point. And then our competitors would come out and it's like the same thing all of a sudden.

45:19And every time that there's a news release for one of our competitors, we immediately go into like a code red and go into a conference room, reverse engineer what they're doing. And this is constant. This is like every month this is happening. The question is like, will this last forever? Like, will we just always be panicking? And the question, their answer is probably yes, right? I think for people listening to this podcast, you two are the closest thing to AI experts as we've had. You're kind of on the front lines of this. What would be your advice to anyone who is working in the AI industry or who wants to work in the AI industry?

45:58And I think that doesn't just have to be founders, but engineers, product managers, business operators, etc. What's the most important thing that they should understand about AI right now? There's an interesting property about AI right now in that it is actually imperfect. You know, when you use the products, it sometimes says the wrong thing. And despite that, it is actually very useful in some domains. That's not like anything in the past. When you used the internet and you ordered something off Amazon in 2002, it's not like they would ship you something incorrectly. Or maybe they would do that, but that's not, there was no expectation going in.

46:34You would buy one book and you would get a different one. And somehow that is still fine right now, which is a cool part of the technology, which is that it actually gets perfect. It's going to usher in a brand new set of applications as well. But I think the key thing to really think about is not to think about, go back from a demo and try to build that today. So think about what products can you build today that are actually imperfect, but still can generate a lot of value. And that is a lot harder than you would think. Because in a lot of domains, if you are imperfect, like let's say you're reviewing a legal document, and it actually completely reviews the document, but 10 % of the time it's wrong.

47:13You can't give that to an end customer. So actually thinking about the trade-offs of how good does the quality need to be to ship your product, right? How fast does the experience need to be? In a world in which the quality is not perfect, it better be fast. There's no world in which I'm going to wait 24 hours for something and the quality is imperfect, right? And then if the quality is imperfect and it is fast, how quickly can I correct it? And these are all important factors of a product. If you don't hit the sweet spot here, you will have a product that's a cool demo, but no one will ever use it.

47:46I think this is the most important thing to really think about. But adding AI to just any field that exists doesn't just suddenly make the product usable. It needs to be a product that is useful in its own right. I think if you give away a product for free and everybody keeps using it and a lot of people keep trying to get it, then you know there's something of value and there's product market fit. And I think if you look at a lot of these AI tools and AI demos, maybe they're not realizing that. Maybe they're just building something that is a cool demo or it's like a really, it pushes the limits of the technology, but it's not actually building something that people want, right?

48:21Or will find value from. I think that's maybe our biggest message to other founders or other people that are building products in the area. It's just think like, if I gave this away for free, is everybody going to want to use it and keep using it? I think that's maybe something people miss. Do you think maybe that's the model that, I mean, I was going to say software founders, but maybe all founders that you should just start with giving the product out for free and see where things take you from that. It worked really well for ChatGPT, right? Yeah. Yeah, exactly. And it worked well for you. Yeah.

48:49That's actually an interesting principle that Jeff just said. Obviously, in some scenarios, the reason why ChatGPT could do that and OpenAI could do that is they own the infrastructure. And if another company did that, they would have gone bankrupt. So you do need advantages in particular places to be able to do that. But at the very least, if you did burn money and you gave it away for free, if no one runs to use the product, you're probably in a world of trouble. That's a great place to end. Varun Mohan is the founder and CEO of Codium. Jeff Wang is the company's head of business. Guys, thank you for joining us.

49:22That was awesome. Yeah, thanks for having us. Thanks for having us.

49:29Our producer is Claire Miller. Our associate producer is Alison Weiss. And our engineer is Benjamin Spencer. Jason Stavis and Catherine Dillon are our executive producers. Thank you for listening to First Time Founders from the Vox Media Podcast Network. Tune in tomorrow for Prof G Markets.