Iteration Velocity is the Key to Engineering Success | Vercel’s Malte Ubl

0:00Iteration velocity solves all known software problems. Eric Schmidt at Google always said, revenue solves all known problems. And I think that's actually bullshit. Revenue doesn't solve Google's antitrust problems. But if you're a professional software engineer, you have to realize at some point that you don't know the future. You're going to make a wrong turn here and there. But if you are able to react quickly, then it's not as bad and you can adjust, right? Developer productivity can make or break whether companies deliver value to their customers. But are you tracking the right metrics that truly make a difference?

0:37Understanding the right productivity metrics can be the difference between hitting your goals and falling behind. To highlight how you can adjust your approach to both measure what matters and identify the right corrective strategies, Linear B just released the Engineering Leader's Guide to Accelerating Developer Productivity. Download the guide at the link in the show notes and take the steps you need to improve your organization's developer productivity today. Hey, what's up, everyone? Welcome back to Dev Interrupted. I'm your host, Dan Lyons, co-founder and COO of Linear B. And today I'm joined by Malta Ubel, CTO of Vercel and former engineering director for Google Search.

1:18Now, Malta has extensive experience leading engineering teams at scale. And of course, today we'll dive into the changes happening with the rise of platform engineering, Vercel's approach to AI and how engineering leadership is evolving. Malta, welcome to the show. Thanks for having me. This is super cool. Awesome having you on the show today. And to our listeners, if you enjoy this episode, just take a moment to rate and review DI on your podcast app. It helps us continue bringing you great conversations with leaders like Malta. All right, let's kick this off into our first segment. Let's talk about platform engineering and some AI workflows.

2:05Malta, platform engineering has emerged as a centralized movement for supportive functions for software developers. And I know it's a topic you're passionate about. How do you see platform engineering changing the way modern software engineering functions? That's a great question. It actually reminds me. So I went to college. It's kind of embarrassing how long ago that was, but like over 20 years. And I technically have a software engineering and a business degree. And I hated every minute of the business part, but it probably was actually more useful to me in the long run than the software engineering part.

2:40Where'd you go to school? And I had this like... Where'd you go to school? It was in Germany, northern Germany, a school called Nord Academy. We literally had a class on IT organization. And the one thing I took away was that you have this oscillation between a centralized and a decentralized form of doing broadly IT. And this holds true for almost every kind of non-core function of engineering that I've seen across these years. So there is also this word of shadow IT. and like in every time you find you're like, we're way too inefficient. Like we have all these people here that kind of do, let's say platform stuff.

3:19Let's put them into a team so that they can professionalize what they do, right? I'm highly familiar. And then you're like, and then they have like, oh my God, their roadmap is like two years planned out. They're not going to do what I want. So I'm going to secretly like hire this one person on my team and I'm going to have them do platform work, right? And so you have this oscillation And eventually you're back to like, okay, there's so many of them I need to centralize. And so clearly like platform engineering as a discipline itself is like a centralization movement. Like because you're now saying, okay, we're going to professionalize this work that's been doing for a while, but just didn't have a name, probably didn't have a character.

3:57And like, I actually have been like, I've been part of this myself in my career at Google. When I started there, I was in a team that was part, actually funnily enough, in the Google Plus organization. and we were like well-funded. We could do like built infrastructure. And so we built Google's microservices orchestration system. I built the framework that now every single like consumer front end is using on the client side. And, but we were just this like small team. And then now, nowadays, and I mean, this happened maybe like five, six, seven years ago, Google created the core product area, which is basically Google's platform team.

4:31And it's not like a thousand engineers, right? But it was made from that and kind of this current team emerged in it. And so that's kind of my career path. And what we're trying to do at Vercel in a way is that if you don't have a platform team, you can kind of buy your software and it does a big part of that job for you. Yeah. And kind of we are that for you or you are a platform team. And because we're particularly focused on the front end space, it's very commonly true that people have like a very experienced back end platform engineering team. you know they may be focused on running their one cloud instance in in virginia right or maybe it's like dual homed but you're probably not doing global operations right and so it makes sense to have like something that from a different discipline kind of come on the side and so we are very happy kind of supporting platform engineering teams on the front end side of their of their operations business yeah do you have an opinion on you know i guess size of company when like decentralized is better or centralized.

5:35I've worked with like a ton of companies and I don't know if it's like a debate, but I see you said it was oscillating, but do you have like a recommendation or maybe for your customer base, what you see works best? Yeah, I think there is a size where the platform team is not even an option, like you have to do it. But the oscillation actually starts after, right? Because as you kind of get too inflexible because you have the centralized team. But I would definitely say that as your engineering headcount gets to about 100, the inertia of that organization is going to be so much going towards kind of messing things up slowly, but there's so many of them that things are not going to, they're not staying on the right track unless that's someone's job.

6:22And so that's when you have to kind of make that investment. Yeah, I think I agree with you. Like, honestly, the companies that we're working with, definitely over 100 developers, sometimes 1 ,000 developers. The best platform engineering teams that I've seen and the best, I guess, operating principles at scale, usually I see a centralized team. That's why I was asking you. That's in my experience. Now, you might have some of what you call oscillations. You have this centralized team, and they have a mission, and they have goals, and they're servicing the other teams. And maybe in some of the other teams, you might have one engineer that's like interfacing with that team more.

7:01But yeah, it was good to get your opinion because what I've seen like from the most productive teams, usually I would recommend centralization. So, I mean, the idea of DevOps was the opposite one, right? Of decentralizing it. And it was a good idea because now you have these like engineers, they're on the ground and they feel the pain and they kind of make their own thing better. It's just that that doesn't scale. And so you need both, right? Like that's how, where in the extreme you are between them is kind of the oscillation. But the professional thing actually is to recognize that both of these extremes have trade-offs and that you want to be somewhere in the middle, that you want to have the DevOps boots on the ground of like, I feel the pain, I'm going to make it better.

7:41And the centralized team were like, it's my job. I'm getting paid for creating great available experience, highly productive teams, etc. So it's my main focus and that just creates better outcome than it's being someone's 10 % project. Yeah. At best. And usually when I see these platform engineering teams, and this will be interesting to see what, you know, Vercel's up to, but typically it's a lot around, you know, I would call it like pipeline orchestration, like the flow of code, efficient system, CICD, getting things out to production. and then maybe building some of those more, I think you said like backend services.

8:23But I think you mentioned Vercel a little bit more on the front end that could be used by platform teams or if you don't have a platform engineering team yet, could kind of be that. Happy to have you dive in to a little bit of what you all got cooking there for platform engineering teams. Yeah, I think that, I mean, at our core, what we do is do a very opinionated CICD workflow that deploys your application, right? And we built the entire developer experience around it. And the other thing that we also do, and I think that is a core part of platform engineering, is we, first of all, built frameworks.

9:03Like we are the makers of Next.js, which is on the front end side, the most popular React framework. and then we make these things kind of one wholesome package versus having these, like, everything kind of be detached, right? And so because we don't treat the apps that people deploy to us as black boxes, right, we're not running, like, an EC2 for you and we don't really care. Even Kubernetes, like, you could put anything on here, right? Like, what do we know, right? We, while we support, like, I think 35 framework or some crazy account like that, we deeply understand the patterns, how people build applications inside of them.

9:44And that allows us to kind of automate the operations based on those primitives in a way that if you kind of treat applications as a black box, you just can't. That's really cool. And I think that's a, we call that approach framework defined infrastructure. And it's been really, really successful. You know, it's actually not completely clear that this approach absolutely scales to every angle of your infrastructure, but certainly in the front-end space where we are the experts, it has proven to be very, very, very successful. Yeah. You said that it's got kind of like a very opinionated, right?

10:17In the workflows or when you say opinionated, like what are the values of that opinion or like what's the philosophy behind it, if you can say in a few words? I have this phrase that I say, which is that iteration velocity solves all known software problems. It's a quote from, well, Eric Schmidt at Google always said, revenue solves all known problems. And I think that's actually bullshit. Revenue doesn't solve Google's antitrust problems. But like, if you're a professional software engineer, you have to realize at some point that you're not going to, you don't know the future. You're going to make a wrong turn here and there.

10:50But if you are able to react quickly, then it's not as bad and you can adjust, right? And so everything we do is basically about literally physically reducing iteration cycles. So that starts with the frameworks we make support hot code module replacement, right? You type in VS Code and your screen updates in absolute real time. We actually try to make this like literally 60 frames per second, which just might be a bit over-optimized. And then it's obviously build times. That's a very obvious one. We invest a lot of feature flags, right? Which is, I think, an important topic. We made a first-class platform feature.

11:27We are big believers in preview deployments. So every time you make a branch, you get an associated application copy from scratch. And then even when you deploy and you're in production, we have a feature called instant rollback, where we guarantee you to be back to the version of your choice in under 300 milliseconds globally. Yeah, very interesting. And if you compare that to like a traditional kind of more cubitical setup where you can be anything, right? It can be definitely minutes, but we have absolutely instant rollback. And you can also do it to any version that you had in production before.

12:04Like technically it could even be years old. Yeah, totally. I mean, it kind of had me at like improving iteration velocity and all of that, like decreasing wait times, decreasing build times. you know again all the best companies that i'm working with are measuring those things working to improve those things so that's kind of like right in line where we come from with the linear b product now ai obviously is super hot everyone's talking about it trying to talk about it want to talk about it how does like ai fit into these workflows or like what are you seeing in terms of either your company or other companies like utilizing AI within workflows?

12:43Yeah, I guess it's the question of the hour. I would say like probably like a year ago, maybe a little bit more. Obviously like that then literally every company would start saying like, hey, there's clearly some revolution going on here. What's our space in it? Like what's our role? Like do we even need to exist? And I'm actually really happy that at Vercel, we have found two answers. And I really love the answers because the answers are so, they feel so native to us. Like they feel really like, okay, this is actually, took us a while to figure out, but now it's like, obviously this is what we should do.

13:15Right. And so the two answers are that we make an AI SDK. So in the software development kit for building AI applications. And that really fits with us because we're at heart framework makers, right? We make software frameworks. And so we give you an SDK to build AI-based applications. right and the other thing that we have built is a software tool called v0 which is the initial idea was it builds the v0 of your app so you type in the prompt and out comes the react code that looks great that implements your app right and we've now kind of started iterating on that and turned it into that plus an assistant that like teaches you how to to use the frameworks we're making and so forth.

14:03And so we have this like thing that builds your app and as someone like that's, that's like our job anyway. And so being able to do that is, is, is I think really key and, and has like resonated wildly with the community, which obviously makes us super happy. I think both of these are interesting, but maybe let's take it one by one. You were talking about the AI SDK and you, you started with like building an AI application, like what is an AI application? and then how would I use like the AI SDK and then we'll get to the other thing. All right, it's such a big topic, right? But like the big picture thing is that you mentioned I was at Google for the longest time.

14:42So everyone around me was building AI apps and that was whatever, five, six, seven years ago. But everyone isn't quite true. Those were all like ultra experts. And that's the revolution of AI engineering is that now everyone who can call an API can be an AI engineer. And so the workflow is completely different. It used to be that you had to like go collect data and then you trained a highly specific model and then collect data. So like really specialized. Probably you can like do something quite reasonable like at your first shot, right? I mean, and you can literally prototype it by trying out ChatGBT and see what it would do for like your problem space.

15:19And so there are some other issues like because those are really expensive to operate and we can get into that. but like you start with something really, really good using the Frontier model and using that system is just calling an API. But it's not really just calling an API because there's some other stuff. And so we built this SDK and it basically has two jobs. It harmonizes the APIs between all the different model providers. And one of the learnings we had that it's actually less work than you would anticipate to switch from OpenAI to Entropic or to Google. Like that's actually something you could do.

15:52But obviously if your software treats them as the same. You can think about it like JDBC in Java land 20 years ago harmonized how you talk to a database. AICK harmonizes how you talk to different AI models. And that's really kind of again native to what we do on the front end is that we taught AIs about UI. And so you can think about it in your app, it can be internal, it can be external. You have domain UI. My favorite example is a seat map for an airplane. And you could totally imagine you have this chat assistant where you say, hey, I would like to change my seat in this booking code. And then like ChatGBT by default would like answer with like whatever, like all the codes for the open seats and you're like, what the fuck?

16:42But if it could answer with a seat map and you just go pick one and then the AI learns about the choice you made, that would be great, right? And so that's what this SDK does. It kind of, it teaches the AI about your UI component and your, really your business components, right? Like that, that belong to your domain and then it makes it interactive, which kind of is really, really helpful for building kind of sophisticated AI applications. That's really cool. I like that. What's been the like response to it so far? I know it's been, it's been really good. Like it's, it's growing like, like crazy.

17:16There's other things like, for example, it supports schema full JSON. So one of the things, first thing that you talk to an AI and kind of you really wanted to respond with JSON because you want to process the output. You don't want text. And that's kind of nasty. And so the SDK helps that. And that's been super, super successful and popular. And I think what we got right, and that obviously wasn't, totally wasn't clear that we would get it right. But what we got right was we didn't over-abstract. Early in the career of kind of modern AI, there were a few kind of frameworks, libraries out there that were kind of very complicated.

17:52And one insight I had like many years ago as like a framework designer is that you can make a thick abstraction if you have built this type of application many times. And I think the constant were like five. Like you really fell on your face for five times and you're like, now I really know how to do this. Right? Like now I can give you the perfect abstraction to, I don't know, to build a web application. Like that's been around for 20 years. We absolutely know how to do it. Nobody has any idea how to make an AI application. So we give you, give people like the absolute minimum abstraction that took away kind of figuring out how to get JSON out of something.

18:30But we don't have like some like crazy callback system, you know, whatever, like what you could imagine because who knows what people are going to do, right? Like everyone's kind of still figuring it out. And so this is really kind of like very simple piece of software that kind of even like evens out the the known kind of tricky things but it doesn't it's not an application framework of the sense that that you uh you know might imagine yeah that's really interesting i mean i'm sure everyone listening like definitely check that out i have a a note here you have a note saying like latency is always is a concern with large language models and other AI systems.

19:10And maybe you can talk about that a little bit, but how does Vercel approach this challenge in user-facing apps? Yeah, totally. Like I think one interesting change that wasn't at all introduced by AI engineering, but suddenly became kind of necessary is that you build applications in a fully streaming fashion. kind of everyone knows in quotes that's kind of stream processing of data is important but for like a normal api call you probably not use it because it's got x complexity and you know if the whole thing takes 100 milliseconds you're like okay i'm just gonna wait for the response right i'm not gonna do like partial json parsing or like you know use the the fully dot by die streaming version of grpc whatever you like like you like you don't go there because is only extra complexity.

20:04But if your API, as in the AI case, streams for 30 seconds, you have to do full-on streaming processing, right? And what we found is that just through the entire serving stack, because of these biases to say, like, whatever, we'll just do it without streaming, nothing was actually ready for it. So we deal out in the front end, React, the most popular framework out there, until a couple of years ago, it just straight up didn't support streaming. You could only respond with a single response. Now, everyone doing PHP would know, okay, no, no, we've been doing this for 25 years. What are you talking about?

20:42That's right, right? But basically, the biases were in the stack. Serverless infrastructure like AWS Lambda, until recently, just straight up did not support response streaming. And so we really kind of had to work through this entire stack to make everything, first of all, work on the infrastructure side, but also get both the AI SDK as the processing system, but the frameworks on top, like the next JSL we make, really ready for this kind of streaming processing and make sure that it works end to end. Because again, if your API takes 30 seconds, you want to show something to the user as soon as you can, and you don't want to wait the whole 30 seconds.

21:19Yeah, definitely. Thanks for sharing all that information. Definitely really interesting stuff, especially if I'm going back to platform engineering teams. Do you have any other advice of where platform engineering teams should be using AI to help developers or where not to? Can it overcomplicate things? Do you have any tips for our audience generally there for platform teams? Yeah, I think as a platform team, I interact on both of these angles of, A, I probably have my internal customers who want to build AI applications. because that's what everyone's doing. And that's my job now as a platform team, like enable these teams to do that.

22:05And the second part is, obviously AI is an incredible tool that makes engineers more productive, if possible. And so I talked about our own v0 tool, which is specific for the front end use case. The most obvious one is GitHub Copilot. And so I think as a platform team, it is my job as the leader of the team to be advocating for either introducing something like this or evaluating how it makes sense in my process. Yeah, it makes sense to me. I mean, most of the platform teams are sometimes actually maybe a little different, but like developer experience teams, most of them have a mission to say, like, how am I going to introduce AI to improve productivity for every developer in my organization?

23:00you know should you know definitely be dabbling there most of the the uh developer experience teams that i work with yeah they're starting with copilot um but just wanted to see if there's anything else that that you saw that that was interesting uh to maybe start dabbling with no i think it's really like the the the side that i think is is key for platform teams like like how do i enable my like my customer teams to be effective at building ai applications right Right. And I mean, AI applications itself is like, everyone goes to like chatbots and like those are actually kind of valuable things to build.

23:41Yeah. But what we see a lot is kind of what I now call invisible AI, where you use AI to do something that was before used to be, it was possible, but kind of maybe too much work. Yeah. And so you wouldn't prioritize it. And suddenly it becomes really easy. Like we actually have a blog post out over puny code domain formatting for Farsi domains, Iranian language. and there just wasn't a library on the internet that actually knew how to do that. But there was a spec and OpenAI GPT-4 old model can write down these domains. It knows how to do it just because the spec exists, right? There's just no software on the internet today that can actually do it.

24:29And so, voila, right? Like you have this problem solved that otherwise, you know, in hours basically, right? But it's the tiniest use case, right? Like, it's like I need to do a task or I need to get something. Yeah, that's interesting. But it's only an easy task if you have a contract with the AI provider, if you figure out all kinds of legal stuff, right? The first time you do it, this is actually not so easy, right? Like, the second, third, fourth time, you now have this, like, super powerful weapon in your toolbox that can be super effective. But you have to kind of break through that wall the first where I'm doing it the first time.

25:09If I'm switching topics here, because you've had a really interesting career. Let's talk a minute about engineering leadership. So you've led both large-scale engineering teams at Google and now Vercel. And how have you seen the relationship between individual contributors and managers in engineering teams change over time? Yeah, this is actually one of my back piece. Like of all the things that Google has done, the one thing that it got right is that it has like this very trivial job letter thing where there's an IC letter and a manager job letter. Yeah. And they both have 11 levels. Yeah. Right.

25:47So you can be a senior vice president or a senior fellow and you're on the same level. Right. And so no one's forced to go through the management side. Yeah, usually it's the management track to like grow your career. Yeah. Before that. Exactly. right and and like i'm you know they're that there's like the peter principle right like everyone gets promoted to the level of their incompetence which is like that's just a straight up true in these like traditional organizations where you at some point if you want to advance you have to go into management right and so it's like one of my absolute most core values to half that ic track so that the people who are on the management side self-selected to be there the other thing that i think is really important is that you know i correct people here all the time And when they sent like, so-and-so got promoted to management.

26:33And like, dude, that's not a promotion because it's a lateral transfer where if that turns out to be not the good idea because this person either hates it, it's just not so good at it. Like they were probably like, we gave them this chance because they were like really good at their jobs. So what are the outcomes if it was a promotion? They feel like they were like, they're not in this role and they're kind of tied up to it because going to IC would be a demotion. No one does that, right? But it was very clear it's lateral. Then you can say, yeah, you know, we tried this out for six months. It just isn't for me.