AI Tooling For Your Dev Team: To Adopt or Not to Adopt? | CodiumAI's Itamar Friedman

0:00There's a very good chance that if you have a serious pain, there is an AI tool for that. Focus on two, three pains the most, if not even one. Go do the research. You probably find a few tools. Think about what's your biggest pain. Research, check the comments, check issues, check usage, and see those that are companies in your domains and using it. Choose two or three. Experiment because it could be different for you. I don't think all of this should take more than one week. And I think one of the great things about AI tools that many of them are really easy to use, like ChatGPT or so. In today's fast-paced business landscape, the success of your business relies on the success of your engineering team.

0:40And the impact of engineering leaders has been clear. We must deliver exceptional software while driving business outcomes. In short, we have a dual mandate. Enter LinearB, the software delivery management platform designed specifically for engineering leaders. With LinearB, you gain the visibility and automation you need to streamline your processes and unlock your team's full potential. Imagine a world where your engineering organization effortlessly manages business outcomes. Linear B empowers you to accelerate software delivery while ensuring your projects align with your strategic objectives and reporting out to key stakeholders.

1:16Gone are the days of disjointed workflows and missed opportunities. With Linear B, you'll harness the power of advanced analytics and real-time insights, allowing you to make data-driven decisions that drive business success. From tracking progress to identifying bottlenecks to optimizing resource allocation, Linear B has you covered. It's time to take control of your software delivery pipeline and unlock new levels of efficiency. Experience the power of a platform designed by and for engineering leaders like you. Discover the future of software delivery management. Visit LinearB.io today and start transforming your engineering organization.

1:54Hey everyone and welcome to Dev Interrupted. My name is Ishaibiri. I'm the CTO at Linear B and I'm filling in for data lines today. Some of you may have seen me from previous episodes, mostly of our labs. That's because I work a lot with our data team, and I'm exposed to a lot of the exciting stuff that we're doing with understanding the behaviors of developers, dev teams at large, using the huge data that we have. I'm happy to dive into AI today, the buzzword that everyone is talking about, and I have a special guest. So welcome, Itamar Friedman, the co-founder and CEO at Codium. Hey, Itamar.

2:30Hey, Shari. Happy to be here. Thank you for inviting me. Great to have you. Thanks for joining. So I always like to start with my guests. Give me a brief overview of your background, your journey so far. So you're now co-founder and CEO of Codium, but how have you started and how did you get here? Great, I'll do it shortly. So I was a two-time CTO of VC-packed startups. So actually 20 years of R &D or CTO management and hacking. My last company I sold was acquired by Alibaba Group. Then I joined Alibaba Cloud. Alibaba's journey was fascinating. I was there for four years. We did quite a few projects, but two products that I'm really proud of is one that we build an auto ML solution as part of Alibaba Cloud.

3:17It means that we automated machine learning ML model creation. And the second one was actually a B2C app that within one and a half year of development from Israel, we reached to 10 million monthly active users. And it's growing nicely and already monetizing, etc. Before that, I did a bachelor and a master degree in a technique on mastering and machine learning. Optimization was one of my favorite topics. And I worked on here and there are a few startups, maybe in companies, maybe worth mentioning that in the early days I worked in verification, like in Mellanox, and actually it was hard verification.

3:57And then I worked on system verification in a company called Silk. So that's also relevant to what I'm doing today. So roughly speaking, that's my background. Great. So you moved from more like the formal stuff of ML and, you know, verification all the way to the new fluffy area of the, I don't know, generative AI. Can you tell us a little bit about Kodium? What does Kodium do? I'm mentioning it because my ever first employee was today my co-founder, so we know each other for 25 years or more. But it's to say that I did web, mobile, hardware, system, networking, storage. But my major is machine learning, actually even robotics.

4:54But almost in all of these, I had algorithms involved and machine learning. So I've seen algorithms from the previous century, those of the 2000, 2010. And my master degree, I actually deal with a bit of deep learning. So I see taking part in this amazing progress and evolution, I actually see it like a step-by-step progress. Yes, the progress is going faster. We do have even a growing gradient, but I do see it as a progress. So for example, in 2016, there was already the gun, the generative adversarial network with generating a network that is in charge of generating content and then one in charge of like verifying the content, etc.

5:43And then there was 2018 attention is all you need, etc. So I see it as a direct continuation of what we're seeing today. I wasn't surprised from, you know, the LLMs. One of the reasons I left Alibaba in the end of 2021, the beginning of 2022, is because we developed with large language models. We even created a few foundation models, etc. It's a certain scale. And we saw how powerful these models became. And if you can frame a problem, if you can frame a problem as a language, not necessarily the natural language, it could be like a DNA sequence. It could be a user usage of your website. If you can frame it as a language, a large language model could hit you really far with analyzing the content and also generating relevant content to that.

6:32And then I thought to myself that, hey, I as a R &D manager for 20 years, maybe it's time to tackle one of my biggest problems, which is code logic testing. Similar to what I did maybe in the hardware area where we did formal verification, we had tools to do because it was really expensive to tape out. It means like to send hardware to production. and then having bugs there. But in the software, we have less fitting. And there are reasons. It's not just how much it costs to ship software. There's reason. And I realized that some of the reasons behind why we don't have good products and tools to check code logic can be mitigated or even solved with AI.

7:20And that's why when I left Alibaba and some ideation about the exact vector and, you know, resting. And in July 22nd, which is almost one year, we're celebrating soon one year, I opened a company together with my partner, Daddy Credo. And what we're doing in Coding AI is building an AI coding assistant focused on helping developers reach zero bugs. Okay, the second part is the most important. There's a lot of code generation tools, but we are more like the adversarial, actually. We are helping to reach zero bugs, helping to verify that the code is generated as expected, that what you write, whether with an AI or not, is fits to your intent, etc.

8:05So that's what we do, and that's how we reach that, through our background in machine learning and large language models, through our own pains as developers and managers. Amazing. So you're deep in that space where AI and developers meet each other. And, you know, as we prepped for this show, you said like maybe one of the greatest questions that's now hovering over our industries is, are we going to even have developers in 10 years? So what's your thing on that? Is AI going to eat our lunch? AI is going to eat our lunch, but probably we have much more lunch to eat in general. Okay, so first of all, it's very hard to predict, especially the future.

8:48But I'm still going to try to do that. I think it's worthwhile separating between 5, 10, 15 years. Largely, I have a general thought, general thinking process. I'm thinking this way. When I have a riddle, I like to think on stream. So I have a question for us. Think about a future. Imagine a future with no developers at all. Can you think about it for a second? You ask any piece of software, anything that you want, whatever, and get it from AI. Even software that was not written before or requires really high coordination between, I don't know, different systems between people, etc. So when I put it in a framework of, I don't know, like 10 years, I don't see that.

9:38And, you know, I think that maybe the AI needs to rule, to some extent, needs to rule the entire, let's say, ecosystem needs to rule the agenda. And then that may happen. And that's why I said, like, maybe 15 years, it's another issue. Now, the other way around, do you see a case, like, in five years that their AI is not taking a major part? It also doesn't make sense. You see how much it's doing right now, and you see all the effort that is being put into it, and diverse effort. And I see in five years, I think we're going to have an intelligent software development stack that enables every developer, actually even creators, to be able to code and deploy and think about software as if they are enhanced, but also as if they have like 10 more skilled developers obeying to them to some extent.

10:38So that's how we see like the five year and the 10 year is like where AI is doing a lot. And as software developers, we can reach really harder like tasks, like, you know, reaching Mars and whatever. And 15 years is like really hard to predict. It really depends where AI is take us, generally speaking. So that's how I see it. So let's park this large question. I see where you're going with looking at the edges of this problem and understanding probably that it's never going to be an all or nothing. But now I'm a development manager, I'm a VP of engineering somewhere, or I'm a developer starting or in my journey.

11:18What are people asking? What is on top of mind now for people that are starting to see this AI in development happening? Great question. So I do want to say, I think you should have two angels on your shoulders right now. Sorry, two advocates, I meant. A devil advocate and an angel advocate. And first I'm going to relate to the angel advocate. I'll elaborate a bit on what you just said. I think listening to your angel advocate, you should be really open and being active about listening and adopting AI. At the same time, with the devil advocate, you need to ask yourself a few questions. What is the transition?

12:04What is the best tool I want to try to use right now? Which one of them is actually mature? Or if you want to be a bit more positive, which actually really fits my use case? So you consider them both and maybe later we can put more focus on a devil advocate. But first, I'll focus on the angel. So I think like really high level speaking, usually as developers, we have the setup phase, right? Like it's here or set up environment. It's here our communication with the product team on what are we building, you know, doing the research about how we want to accomplish things. Then there's the implementation phase.

12:45You write your code, you debug, you commit, you have the process with the development team. And then there's the deployment stage. I'm including them not only like the CI, CD in the cloud, but actually even the feature flagging. And now my point here is that I think that for each one of these elements, and there are sub elements it's part of these task or sub process and there's uh there you will find a tool that is somewhere or even a few between starting to half mature half bake to to mature and what you can do is try to start with your paints try to start look like i'm like breaking down my difficulties as a manager or as a developer by the way and let's focus as a manager for a second and where are my difficulties look on this matrix try to think the pains that you think that you can imagine that if you had like an intelligent like a creature don't think about the product could could it help and then try start looking for products and most like a really good chance that you'll find something and let's look at coding AI if I may say about like what we're developing so let's say that you're you're a manager that you're really struggling with reducing amount of bugs in production.

14:10And the way that you're doing it, I don't know, is like asking your developers to adding more tests and increase code coverage. But at the same time, you know that code coverage could be a proxymetric to actually good testing or even sometime vanity if you don't treat it correctly and it's hard to treat it correctly. Then, okay, you have a pain. And then you look for these, you know, words and you'll find ConeUBI because this is exactly those are the pains we want to help with, like helping developers. If you look on the comments of our product, you see the people saying, I hated testing, now it's fun.

14:45And oh my God, I found two bugs in production, you know, things like that. And like in the code that is already in production. So that's my main recommendation. Think about AI as another intelligent being and how can that apply to my problem? Is that like, what if I had another developer on my team that I would put just on these tasks? Because developers tend to be intelligent. If I'm looking for a general answer through intelligence, if I hired another developer and asked them to do this, is that a good initial proxy to think about what I can get from AI? Just more firepower with brains, probably cheaper than a developer.

15:29But is that ideally on par with what an AI could give me? like one word yes two words yes no and four is yes no maybe so because I'll explain like I think AI like the core capabilities are relatively different in my opinion and then it depends on how the product was packaged a product that is AI empowered could behave like a developer but it could do other things it's like claiming like and just for the analogy Basically, you can say that Google basically does one million people, one million developers work to go over the entire Internet and index it or something like that. But it's impractical, right?

16:16Like it's not visible to some extent. So it's yes, because some product are like, for example, those that are trying to like to be chatty or so like imitate like talking to a developer. but some of them are really bringing like UX UI that is really different. So I agree that it's one way to think about it. But another way is that there are capabilities like indexing and looking things at a whole. And then one more perspective is that, you know, you also care a lot about developer happiness. Maybe you don't care so much on AI happiness. Okay, right now. By the way, I do care. I'm just like considering that in the future.

17:01So I think that, for example, one of the reasons, it's only one reason that Coding AI, we decided to tackle code integrity, like code logic testing, is because I think right now, like developers either do not do too much testing because mostly they hate it, or because they like the creation and not like the verification, or they do testing and they do practice verification, code logic testing, etc. but they hate it. So either you don't do it and hate it or you do it and hate it. And here you go. There is a tool that can help with developer happiness and spread love with this best good practice.

17:41Okay, so to summarize this point, yes, it does make sense to sync with this framework of I'm actually buying more developers and much cheaper, but I think it's more like I'm adding more key, and additionally adding more capabilities that maybe developers couldn't do, especially things in large scale. just in time, you know, you never go to sleep. And as well as doing this harder or the stuff that developers maybe wouldn't want to do. You still want the developers as the drivers, if you ask me, and not the AI. AI right now is more like empowering. Using that metaphor to imagine where I could use AI, right?

18:21If I'm stuck with resources or my developers don't want to do this. and we know we all hate verification, then I say, okay, what if I had a magic developer that loved it and was good at it? But you're saying it goes beyond. Some skills or capabilities are very different. For example, having access to troves of data and being able to index or work very fast on a large scale problem. Okay, so number one, you're saying, look at your current pains as a practitioner or a manager and know that there's a new class of solutions bubbling up around probably most of these pains. So actively look for something.

19:07Sometimes what I'm seeing is the co-pilots and other solutions. Developers are not necessarily finding them because they have a pain. I know how to eventually write my web app or my database service, but I'm still using Copilot. So sometimes it's not a direct, like I'm not looking for something to ease a pain, but I find something cool and actually saves me time. And so how do you go about this? That's a pain. Well, where I'm not even aware or acutely aware of the pain. Cool. So I think like, I know I'm like a bit of generalizing here, but But develop like R &D manager or director of engineering or whatever.

19:57Like, what's your purpose? What are you trying to aim? Like maximum value, right? For an organization and minimum time. So you're basically like, right? Like maximum value, getting out maximum value from my R &D team. Develop the product and the business. Yeah. In minimum time. So basically time, like spending time is a pain, for example. and I think pre-copilot, then you would spend a lot of time in searching in Google and Stack Overflow and things like that. It would take you a lot of time to search and also combine searches. I think copilot is basically disrupting that part. How do you, you know, instead of going to search on Google or Stack Overflow, etc., And unfortunately, for different stakeholders, then you just search inside your ID, but also like it kind of combines a few searches.

20:52That's what it's true. So it actually saves you a lot of time. The second thing is that for junior developers, if I may say it's subjective, then it actually even increases your level. Like it's not only like help you with the search and combination, but actually choose for you the best one. And to some extent, if you're a junior, probably raise your level. So that's a pain, like two pains. Reducing time from if you want to make your software. But I agree that like maybe you wouldn't think about it directly. And this is why I also think with all the respect that Copilot is a really productive tool.

21:31But I think if you think it's a game changer, I think in five years you will think differently. that you will see like game changers tool, like really addressing, like, again, like I know what I'm saying is like more like a vision. I'm not saying that right now, Coding AI helps you like guarantee that you have zero bugs. Like it's like Tesla zero mission. Not necessarily they reach, but they're progressing really nicely. So think about if you can know that you have zero bugs in production and how much fast you can move when knowing that you're being covered this way. I think that's a game changer.

22:04So again, I have a lot of respect to Copilot. like cogeneration. I think it's like really boost productivity. And maybe you didn't think about that as a pain originally because it's not that acute pain. But if you think about the two pains I told you, then I think actually you do find it like they are quite big, maybe not acute. And that's why it's fun. And also magical a bit, right? Like as a developer, oh, I can't believe that AI is doing it for me. It was original. Like it was one of the first AI products. That's I think why we also love it so much. Yeah. So if I'm a dev manager, there's looking for my specific pains and actively looking for solutions what about like my developers and you know people on my team are probably looking for those solutions too starting to use them how do i become more proactive and you know just not just finding something to help me with the pain but also helping my team discover and start to use these how do i we're going to talk about pitfalls in a bit but even before yeah even in the rosy area.

23:05It's one level that says, oh, I have a pain, let's go for a solution. How do I enable my organization to go and jump on this and harness this revolution beyond just looking for direct solutions? Everyone try out things. Do I have a dual program? How do I approach this? I have 100 developers in my org. What now? I have a bit of a few, perhaps generic suggestions, sense but i actually want to like kind of twist them towards the ai tools because i guess that most managers would know the or the trivial ones like for example you know define how like in your core do you want to be a first adapter like early adopters and want to try and you want to be in the front and you want to think like i'm mature do you care or easy ease of integration or do you care about how much of the pain it solves you.

24:02So there's a trivial thing I just mentioned, a few points, and I think they are worth sitting and thinking about. I don't think more than 30 minutes or so, like a discussion between a few people. But twisting, thinking about a specific AI angle into it, is that AI is like a statistical creature. By the way, there is a lot of debate. Machine learning is like a nice framework around statistics. or two different things. I don't know if you know all these memes. I have some opinions, but it's not the topic. So allow me to say, there's a statistical creature. And it doesn't necessarily give you... It depends on the product, but consistent answer.

24:44There's different guardrails that developed by the product owner. And it could, because it's like that, it could work differently between two different companies or two different code bases or two different teams, for example, or so. It doesn't mean that there's got to be a big variance, but it could be. So what I'm saying here is that I think like, I actually think that one thing you need to look at how easy for me is to check it a bit. And you would like to check a few and see how well they work for you. For example, I heard one company trying for one quarter 13 tools and then choosing two. and they were really happy about it because it's like a lot of investment, but they said that they wouldn't necessarily pick the two that they thought about and that was really helping them in a meaningful way that they couldn't think about it originally.

25:38So maybe that's an extreme, I'm not sure. I'm experiencing this. Like enterprises are using Coding AI. We offer easy trial and that's like the atmosphere right now, I believe. I think Linear B, et cetera, on that role right now. I hope it's easy to integrate and then just try it out. So I would just, what I'm saying here is that with AI, do consider how easy it is to try it out because it's not like a spec is written. It's going to be exactly like that. It's a lot about how it performs for you. So you're saying be active about experimenting. like don't you say I'm going to find one tool experiment with tools there's got to be variants maybe know that everything is changing so fast so what you see now may not be what you have in six months can I even afford not to begin experimenting let's say I'm not a early adopter I have other priorities it's not a huge pain I'm solving but is it a risk or a material risk not to begin an experiment, not to have people on my team begin to look and use these tools.

26:49Am I going to be left behind or at a severe disadvantage if I wait for a year? And then I'll see what bubbles up. I'll see what makes it into mainstream. And then I'll adopt. So we can take again the notion of having two extreme cases. Let's say that there will be, there are already, I think, starting to build, but let's say there will be AI tools that helps your team. We're talking an extreme case, just like 10x more productive. Okay? And this means that your competitors are going to develop 10x faster if they onboard, fully onboard, and you did not. Okay, that's an extreme case. On the other way around, if these AI tools are hard to try, hard to experiment with and basically bullshit, then you waste your time and you defocus and you're probably having a lot of pressure.

27:52So it's probably like somewhere in the middle. And as time progress, it actually goes toward the first extreme. That's my opinion. So that's why like so far we talked about like thinking about your pains, that they're there anyway and do start experimenting with them, they'd be a very good chance that some products could help you, even very meaningfully. But then as you get to use these products that you need a slightly different mindset, then you could keep progressing with more and more tools as time passes. So that's how I see it. I wouldn't recommend skipping because there's so many, I think not too much time would take you to look on your pain, experiment a few.

28:38And that's my point here. Actually, I built up to, and I didn't say that. I think there are those tools that are really easy to try. So I don't think the efforts are so big. And you shouldn't miss the opportunity. Again, there's the devil's advocate part, right? So you're mentioning the downside, or devil's advocate. Why not start? Yeah, so I think, like, let's take one of the most dominant tools out there right now. like GitHub Copilot. As an example, okay? So I heard from quite a few people, by the way, I personally use the tool and I love it. I just, I know I'm the CEO, but just I couldn't for two days, I was actually programming.

29:25I don't do it too much. I couldn't stop myself this time. We're going to release some new thing soon and I wanted to contribute. Right now, our tool is in the ID extension. and we're about to connect also to the ICD pipeline, et cetera. So I love it. But I heard from quite a few people, they're passing through the traditional cycle of, oh my God, this is amazing. Oh my God, this is actually ruining my development to, okay, this is useful, but I need to use it carefully. And why do we have this cycle? It's because at the beginning, like you see the magic of AI and you start like even adopting the way you develop towards being AI empowered, which is good.

30:08But then you fall into traps of, you know, the AI suggested something that wasn't really fitting for your use case, and you're already so used to click tab and continue, you know, and then you realize that you can, like, you can say AI will get better and better, but it's more than that. It's not like just the core capabilities of AI, it's even the UX UI. Like the UX UI right now of code completion not necessarily has the capability to really, like 100 % give you code that always works. We'll talk about it in a second. And then you learn to use it better. Then you go back to the driver and then you enjoy the tool.

30:45So I think by this example, you can see that one issue that I raise right now is the way you use AI. And you need to work with your developers and yourself not to blindly trust it. So this is obvious, but also not blindly. Let's start using, after you experimented, and that's what the team that I told you that tried 13 teams and then moved to two, they talked about each tool, like how it's the best way to use that tool. And like the way you need to start generating best practices around it. And then, so basically I'm repeating back, like it's a matter, I'm repeating again what I said before, like it's a matter of statistical product.