The One-Person, Billion-Dollar Startup? — With Thomas Dohmke

0:00The CEO of GitHub takes us to the cutting edge of AI, where AI is actually writing code for engineers. and he tells us how this might change the economy and importantly, how its success might translate to the rest of the AI field. All that and more is coming up right after this.

0:20You're used to hearing my voice on the world bringing you interviews from around the globe. And you hear me reporting environment and climate news. I'm Carolyn Buehler. And I'm Marco Werman. We're now with you hosting The World Together. More global journalism with a fresh new sound. Listen to the world on your local public radio station and wherever you find your podcasts.

0:47Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. We're joined today by the CEO of GitHub, Thomas Donka, who is here to talk to us about the field that is probably being influenced and impacted by generative AI the most. And we're probably going to learn a lot about what this era in the coding world is going to pretend for the rest of the technology industry and industry at large as generative AI gets smarter and spreads across more disciplines. We're also going to talk about how it's going to change our economy by making it easier to build tech products.

1:22I'm so excited to have him here. Thomas, welcome. Hey, thank you so much for having me. Thanks for being here. All right. First of all, just a bit of news that we got this week. Elon Musk is boasting about the fact that they now have the most powerful, XAI now has the most powerful AI training cluster in the world. He says they've got 100 ,000 liquid-cooled H100s, which are NVIDIA chips, on a single RDMA fabric. And basically, he says that XAI is going to be on par with the rest of the AI field or better by the end of this year. A lot of people that I speak with in the AI industry sort of have this perspective of, we know Elon's doing something.

2:02We don't really know what he's doing. First of all, I'm curious what you think about this AI cluster. And second, do you think the rest of the tech world is underestimating Musk when it comes to his ability to build AI? Oh, sounds super cool, right? So many super cool chips. I don't think the industry is underestimating him at all. I think he's looking at what Sam Altman has done in OpenAI and what other model companies are doing. They're training larger and larger models. I don't think we have reached the scale limits of creating bigger models with more parameters. And I think that's where Elon is looking at well, using all the data that he has through Tesla and other sources and then looking at building even better models.

2:47And are you of the belief that sort of the more compute and the more data that we throw at these models, the better they're going to get? You talked about the scaling law. You believe that's going to apply to a certain extent or where do you feel this nets out? I think so. I think we are not at that scale limit yet. I think we can build smarter models with more knowledge, with more reasoning capabilities, multi-step or chain of thought, I think is what OpenAI calls it. And there's a lot of problems that you can't solve with Copilot or JGBT today, even in just the coding world where things are fairly limited in output and there's lots of more creative work to be done.

3:30So I think larger models will help us to build cooler stuff. And so you mentioned OpenAI. You're the CEO of GitHub. GitHub is a partner, is a subsidiary of Microsoft. Microsoft is a partner of OpenAI. so you're using the OpenAI technology and what you've done with it is quite fascinating. Effectively, what you've done is built this Copilot application which is, the way I look at it, is an autocomplete for engineers the same way that when we're in Gmail and Google autocompletes our sentences, you're autocompleting what engineers are writing and from your internal studies, you found that using GitHub Copilot increases the productivity of an engineer by 55 % or what I've heard is that from you, the way that you say it is that basically engineers can complete their projects 55 % faster than other engineers that aren't using this AI assisted co-pilot for code.

4:26Is that right? Yeah. So maybe let me time travel you back about four years. It was June, 2020, and we got access to GPT-3, the previous major model release that OpenAI did at a time when almost nobody outside the tech world was talking about AI or I don't think the term generated AI was a big thing back then. And we played with that model. It was 2020. So we were all on a call. Everybody was in lockdown and we saw that the model was able to generate code. That's the thing that engineers write. And we were surprised how good it actually was. In fact, We fed about 200 or so coding exercises, things that engineers do when they apply for a job into the model.

5:12And even four years ago, it was able to solve 92 % of those. And that was that aha moment. The light bulb went on, and we built the first co-pilot. And all it really does is it helps developers in their editor, the app where they're writing the code, to think ahead and to predict what's the next word, the next line. maybe multiple lines if it's a more complex algorithm. And if you think about, you know, other than the AI capabilities themselves, what really is crucial here is that you have an assistant on your side that prevents you from losing the flow. You no longer have to context switch between the editor and your browser.

5:53And, you know, engineering browsers look the same as your browsers as well. Like lots of tabs open, you know, the thing that I wanted to buy in the vacation I'm planning and the world news. And there's always, you know, something else going on on hacker news and whatnot. And so with this context switching, you know, between your editor and your browser, where you look things up, where you look, you know, for algorithms or documentation and things like that, that is made easier with Copilot. We can just stay where you are. You get somebody, you know, the AI thinking ahead of you. And if you don't like it, you can always, you know, just keep typing or edit what you got out of Copilot.

6:29Right. So you're basically in the editor, you're typing, you're coding, and the AI is suggesting code, similar like the way that if you're like writing a sentence, like you might get it on Gmail. So one of the things I've spoken with CEOs who have who are like, we have an AI strategy. We need to implement the AI strategy. And they always talk about how code is the first place they're trying to do it. But then I also ask them, like, are you seeing the impact? and up until recently, I was sort of being met by a shrug, sort of saying, yeah, our engineers are using it, but I don't see our engineering team increasing that much.

7:06Or some are just like rejecting the suggestions, some just prefer to do it the old way. So I'm curious what you're seeing on your end, like on the aggregate, is there uptake in the co-pilot? And are you really seeing the change in organizations here? Absolutely. And you already mentioned the 55%, And that came out of a case study where we asked 50 developers with and without Copilot to solve a coding problem. And the group with Copilot was 55 % faster and about 10 % more successful than the group without Copilot. Now, obviously, when you go into companies, they don't have 100 engineers do exactly the same thing.

7:42And you can compare both groups against each other. Most engineering managers and leaders have better ways of investing all their manpower. And so we look at real customers, we look at how often do developers accept the code that they're getting from Copilot, and how often do they press the tab key and getting the job faster done than without it. And the number we see there is 35 % on average. Some are a little bit lower, some are a little bit higher, depending on what programming language they use. But that shows you that in a third of the time, when a developer sees a suggestion, they also accept a suggestion.

8:20I think that already gives an idea of how much code is written with Copilot. We have Copilot deployed now in more than 50 ,000 organizations with more than 1.8 million paid seats. And companies like Infosys and Mercedes-Benz, Ernst & Young, PwC, but also the cloud natives, Mercado Libre, Etsy, HelloFresh, Autodesk. If you look at those companies, they're all writing millions of lines of code that is coming from the AI and no longer from the developer. The developer is still there and the developer is controlling the co-pilot. He or she is the pilot that is flying the plane, but the co-pilot is providing lots of code that the developers can just accept and as such get the job done faster.

9:09And so are these organizations building more? Like, have they told you, because that's the one thing that I say, like, all right, so like you have like, let's say one third of this code accepted by engineers. But I also wonder, like, is that leading practically to more stuff being built or in the other side of it? Is it leading companies to like not have to hire as many engineers because their group that they currently have can get more done? What can you tell us about that? so let me give you an example Accenture for example they saw an 84 % increase in successful builds with the help of Copilot and they saw 88 % retention of the suggested code and most of the developers told us loud and clear in surveys that Copilot allows them to stay in the flow allows them to spend less effort on repetitive tasks and Ernst & Young had 150 developers commit more than 1.2 million lines of code with the help of Copilot.

10:09So we see these statistics all over the place where companies are measuring over a period of time an increase in productivity, an increase in developer happiness, and ultimately an increase in output. The build that gets deployed to a server is what Copilot is helping developers to deploy more. And are they seeing like a specific like ROI there? Like, are they like, all right, so it's one thing if you put more code in the project, but we all know that that doesn't necessarily mean like it's going to be a better product or that it's sort of getting a better return. So what can you tell us about that?

10:47So we, you know, at GitHub and at Microsoft, we obviously also use Copilot ourselves. And we see, for one, from developers, from developer surveys, we do, you know, developer experience surveys every month or so. So we see that developers are clearly more satisfied for the use of AI. They feel happy in their job. And, you know, a lot of friends in the industry, they're like, I would pay, you know,$20 just for making my developers happy. And the rest doesn't matter. It will come by itself. But if you look at the overall, you know, metrics, engineering metrics, we see more throughput. We see basically more ideas, you know, more work items in a planning system taken and shipped, you know, to the customer.

11:29And as such, the throughput of software developers is increasing through Co-Pilot. But the challenge of these questions always is, how do you compare teams against each other, given that the thing that developers are building is a really creative task? Most feature requests, most bug reports, most security vulnerabilities, all these things are not easily comparable to the next task and the next task. That's why you see software projects often being behind on schedule because it's really hard to estimate how long work takes. And then it's also harder to estimate how much did you actually save. If you didn't know how long it would take in the first place, it's also harder to figure out how much you save.

12:13But I'd say from our own data at GitHub and from customer evidence, we clearly see in that range that I mentioned, 35 % of productivity gains. Yeah, but I also would say you could probably more easily measure it than you're letting on because you could say within an organization, how many of our projects were behind schedule before Copilot and how many of them are behind schedule with Copilot and then the delta is your increased improvement. Well, I think that's true in the sense that, you know, if you would do the same thing all over again, you would be able to measure this. But as the complexity of your software system grows and as your developers, you know, constantly have to tackle new challenges, it is somewhat challenging to compare, you know, the feature you shipped in two months versus the ones that you shipped two months ago.

13:03Right. Right. Or the other way I would do it is maybe say, you're on the same project building on the same timeline. What time do you get out of work? If you were getting out of work at five and now you're getting out at three, then I would say, thumbs up, this thing is working. But if you're getting out at five still, then it might be a question there. Is that a stupid test? I think, you know, if you look at, you know, I've been working in software for over 20 years. I think the best indication you have is just ask the developers if they feel that a tool that was put into their workflow is actually helping them.

13:40And let's face it, most of us hate when somebody comes and says IT. It's like, oh, here's this new tool and you have to use it. It's prescribed by the company. Don't touch my system. I know what I'm doing is kind of a standard response. And, you know, what I've seen with Copilot ever since we first shipped the preview three years ago is that developers are excited about it. Even those that were skeptical at first, you know, after a few days of using it, have seen the magic. You know, we often see on social media that people are saying, I didn't believe it until I used it. And then all of a sudden it understood what I'm trying to do.

14:19and part of that is it's not only predicting the next word, it's actually taking everything in the file above the cursor, below the cursor, adjacent tabs. It takes what you have already in account and as such it knows how you're writing code and what you're trying to achieve. And so developers after using this are happy about having this AI available to them and it doesn't even feel like AI, right? It's just a tool that is in your editor while you're doing your work. Okay, I'm buying your argument that this is something that is helping engineers within companies. And then another thing, like, so let's say, let's continue saying that this is something that is helping and increasing efficiency by the numbers that you're putting forth.

15:05You also have this other idea that it's not just within companies, but also that it can empower individuals to build basically anything their mind can think of. because the coding becomes that much more efficient to do. You did a TED Talk recently and you gave this example of wanting to build an app that just sort of logs every flight that you've taken. And no one would ever build that type of app because like the amount of time it would take and the ROI is questionable. But if you want to build it for yourself, you said you were able to do it in the time that it would take you to finish a glass of wine.

15:40Now, even if you're an extremely slow drinker, that's a pretty fast amount of time to build that type of application. So talk a little bit about what you think this means on an individual level. We've talked, we've heard even in the tech world about like, is there going to be one person that builds a billion dollar company? Right. And so I'm curious what you think about this and if this enables that. the first thing that you know comes to mind is when you think about individuals trying to code is that they often squeeze that into their day you know they they have you know another thing that they're working on whether it's you know me as CEO of github being being on podcasts or having meetings with my team or planning the next quarter or it's developers that do hobby projects in between all their work projects and their personal lives.

16:31And so you're constantly in this off and on switch where you have to context a switch from what you were working on before to the new thing. And with the help of Copilot and specifically the chat functionality, you can easily get back into the flow, into the thing you were working on. You can ask questions. You can even ask questions about what did you build yesterday? Like describe that code again to me from yesterday. and what I said at TED is you can do that in the language that you grew up with. For me, it was German. Obviously, many people speak English, but also many people on this planet do not speak English.

17:07When they want to build something, what they don't want to do first is to learn a language that they are not speaking every single day. And so it democratizes that access even more. So if you think about a six - to seven-year-old that wants to build something cool and is fascinated by computers as many kids are, Maybe they have already seen some computer games, and now they want to go and build a little game. I showed at a different conference, a snake game, you know, a game that was on the Nokia phones back in the day where you control a snake and it eats an apple, and the tail gets longer and longer, and then you have to prevent not eating your own tail.

17:43You can do that by just asking a couple of questions into the check functionality, and it gives you all the code, and you can copy and paste that, you know, into the editor, and then you can keep asking questions iterating with the AI, and you don't have to figure it all out by yourself. You don't have to read a book first and you don't have to find the right web page where this is all described. And even then, if you find that web page with the code examples, you still have to make that work while you have Copilot helping you every step of the way. So I think there's a future coming soon where you can create any software you can imagine with just a few prompts written in human language.

18:23And as such, you know, you're building, you know, your own software ecosystem in the same way that, you know, we're designing our living rooms or building Lego sets and whatnot, right? We're exploring our creativity in so many ways as humans. Yeah, I think the line you said is the floodgates of nerditude have swung wide open. I like that. Yeah, because, you know, like there's so many other things, right? Like I mentioned Lego already. Obviously, if you want to play a music instrument, all you do is buy the music instrument. And these days, probably watch a couple of YouTube videos or find a web page and you can start playing.

18:59Nobody is stopping you. It might not sound great. But as long as you do that in your home or on your own computer, you have all the creativity that you have as a human available to you. And now you have also the tool that helps you through that process. Right. Right. So the example of the$1 billion value company created by one person, do you see that as something that's feasible? I think so. I think that, you know, I've seen a bunch of examples where small companies like Instagram comes to mind, you know, that started really small. And by the time they got acquired, they were still very small.

19:37WhatsApp is a similar example. And so I think, you know, that can exist. the question is a single person company how they're also managing uh support and accounting and all the other things that that are outside of creativity and maybe a copilot also helps them with that and you know answering support questions but i think there's you know it's more fun if you have a smaller team uh of people available that's less lucrative yes so let me let me read you this example that i saw on reddit of a coder that was talking a little bit about how they've worked with generative AI to build. So they say, it's mind-blowing how quick I can move now.

20:18They're using Sonnet 3.5, which is an anthropic model. I'm pretty sure I could implement copies of the technical parts of the most popular apps in the app store 10 times as fast as I could before large language models. I still need to make architectural and infrastructure decisions, but stuff like programming, the functionality is literally 10 times faster right now. And this is the process that they use. The first thing they do is they think hard about the feature and probably discuss it with Claude. The second thing they do is write a basic spec for the feature. It's just a few sentences and bullet points and also iterate with Claude on the spec.

20:53And then they are sure to provide Claude with all the relevant context and ask for the implementation, the code. So basically what they're doing here is brainstorming an app with Claude, specking out an app with Claude, and then having Claude code it. I mean, that is remarkable. Is this something that we're going to see be more common? I think we're going to see it at a smaller scale. For small projects, you can probably get there even without a lot of computer science, computer engineering knowledge. For larger projects, I think the step missing is the architect, the software engineering expert that knows which database to pick, which cloud provider, how to make the app, scale from 10 users to 10 million users.

21:41And the model can kind of help you with that by giving you options, right? We have all seen that when you ask ChatJPT a question, an open-ended question, it gives you options and explains to you kind of like how to get there. But to navigate then this tree of information, you still have to have subject matter expertise. I don't think that goes away, but I think if you have a well-defined task that you can describe to a certain level, to a model or to a whole system like CoPilot, you will have that agent, if you want to call it like that, do the job for you to 90 % of what you expect. The flip side of that is if you think about when you work with other people, whether it's on software or other projects, it's like how long can you have a person go by themselves when you give them a task until they're going so far off track of what you actually wanted to achieve versus what you described to them, right?

22:41Like more often than not, we need the feedback loop as humans. We can't work in isolation for too long until we're either completely off track or we come back with a work result that isn't really what the person or manager or our customer expected us to do. and think this is where we have the boundaries of these models. If the human can do that because ultimately the customer can't describe it or the manager can't describe it to the level of degree that you can actually fulfill all the requirements, then the model can't do that by themselves as well. That's why we believe the human needs to be in the center.

23:20The human needs to be involved at the step of the way to make sure that we're not getting into the wrong direction. But this is exactly what the person is describing, that they're not only asking Claude to write the code, but they're dialoguing with the AI bot about the different spec and the decisions and how to set up the components and things like this. And then it builds only at the last step. You know, I called this, I don't know what it was, I called it the second brain. It's kind of like we have an extra, you know, outside of our brain memory chip that gives us all the information that we can store ourselves.

23:57And even if, you know, we have a lot of things that we learn, you know, in university and high school and even before that, that we can't really store and we often forget these things. And so the AI is helpful to retrieve this information again. You just need to know how to ask the right question and then work with you. That's why we ultimately called it co-pilot. And it helps you to have fun with the things that you want to work on. And it takes over the boilerplate, as we call it, encoding the stuff that surrounds all the creative part of the process. Right. Now with Gmail, Gmail allows me to write emails within the Gmail application and will suggest some text for me as I write.

24:45Sometimes I accept it, sometimes I don't. but also like i could just go to claude and ask claude to write the email so i'm thinking from your in your circumstances like is there a is there a reason why people should be using like the co-pilot within github as opposed to like having this conversation with let's say an anthropic bot and just having that write the code and then dumping it into the code editor by the way you can also use the ai to summarize the email so um it's better you the one side writes the email with AI and the other side summarizes the email with AI at which point you can ask the question why not just send the prompt to the other person and and save the time on all the uh you know friendliness and and the salutation and whatnot that we put into emails because it's or just become proficient at writing concisely but I think as a journalist that you know I know that that part of the world is not gonna is more difficult to proselytize than others Sorry, go ahead.

25:45Yeah, and I think to some degree that will happen. And to some degree we have so much information around us now that the summary is good enough if you only want to read the headline and the summary and not dive into a 10 ,000-word article because it ultimately means you have more time for other things. We're all dealing with limited attention, limited lifetime ultimately. and so if I can shortcut some of these things that means I have more time for other things. Coming back to your question now why not use the generic chatbot the power of Copilot is that it lives in the work environment of the developer so yeah you can copy and paste everything that you see in front of you into a generic chatbot and have it give you an answer but it's much more powerful to have the chatbot sit within your environment where it knows what files are open It knows what you wrote before that.

26:42It can look at adjacent tabs. It can even look in the developer world at the output, the debug output is what we call that, and the console and error messages and those kind of things. And so it has much more context available. It helps it to answer the question within the specific context of the project you're working on. And, you know, a very simple example is that it knows, you know, whether you like your variable names with camel case or capitalized or whether you write in German or English by just looking at the context of your file. One question for you. So basically, you can write the prompt of what you want and the code will be developed on the back end.

27:28Is there going to come a point where we're not going to need code at all to build? because if you can just prompt, then why do we really need a B in the code? You might say we're already at that point to some degree where when you go and ask ChatGPT a question, you get an answer, and when you ask a question to plot the chart, for example, or do a mathematical calculation, it actually generates a Python script that then plots that data into a chart and it shows you the chart and still shows you that step where you see in between the Python script, but they could as well hide that and you just see the chart output, right?