Hypergrowth startups: Uber and CloudKitchens with Charles-Axel Dein

21:47But the fact that we did not have the means we were basically flying blind and someone exposed us. I mean, it was kind of a good thing, but this stuck with me. I'm like, since then, I remember like, okay, how can we make sure that we will know and we will not have to have someone tell us like, okay, this is broken?

22:04Charles-Axel Dein:Yeah, frankly, to this day, I don't think even on my current team, we still have to work on those things, right? There are three metrics at Cloud Kitchen that we always look at for any incident. Time to detection, time to mitigation, and time to resolution. And the most important part for me is time to mitigation. It means how much time does it take us to bring the system back to a state where the customer doesn't see the impact, right? Resolution goes after. And if you waste like one week of time because you did not detect the issue, there you go. This is what you need to fix. And at scale, especially at uberscale, being able to detect is much easier because you have such a high volume that if something breaks, you must have some kind of monitoring that will detect it.

22:51One thing with hyperscaling is things grow so fast. You keep hiring people, but you're hiring people a lot slower than what your growth is in terms of customers, features, all those things. Automation becomes really, really important. You've always been a huge fan of automation. I remember at Uber, you kind of built your scripts to automate this or that. What have you learned about automation? How it works, how it doesn't work? Maybe some things are not as obvious about it.

23:18Charles-Axel Dein:The most recent thing I learned about this is this article I read about the ironies of automation, which is a pretty old article, but it's fascinating. And it talks about two things that I've definitely seen and I've made a lot of mistakes in this area. One, sometimes when we automate, we replace user error with automation designer error, right? Like we are lacking context. We don't understand how the software or like the process is used. So when we automate, we actually put in the wrong automation. That's one. Two, because you cannot automate everything, you usually automate the simplest things and you leave the user in charge of the most complex stuff.

23:56Charles-Axel Dein:So that's the irony here. It's like automation sometimes fails. So what can you do to prevent that? The simplest thing is observe and get a lot of deep understanding of what the process is. You have to do it manually, right? Like we say, startups should do things that don't scale first. So do the things manually first. This will give you the business context, product context, operational context to then automate correctly. And then the second thing is transparency, right? So many automation, particularly in the developer experience world, automate so much, you don't see anything. And then when something breaks, you are left to debug.

24:34Charles-Axel Dein:I'm sure it happened to you at Uber with some of the developer experience products. Yeah. And when you said to do things manually at Uber, we had this really interesting thing called sanity tests. Sanity tests were, we were in charge of our team-owned payments. So everything in the Uber app, there were like 12 different ways to pay from credit cards, Apple Pay, Android Pay. and there's a lot of like regional things like in India there was Paytm and a bunch of others and we would just, every week we would need to make sure that the payment would work so we would go there, we would order an Uber and like see that the payment went through and we would actually do this manually back on what sometimes we mocked up but every time we would go through it and when we did it long enough we're like, this is and we needed to do it manually for a few reasons one of them, it was just hard to automate because we had to talk with the real payment providers and a lot of them did not offer APIs to do this.

25:30There was also fraud detection. But we kept doing manually and after a while it became really, really painful. And then we figured out how can we automate this to some extent? And then we actually started to make progress. But we started with just this thing and we did it for, I think, many weeks. And in the end, we got so tired and frustrated that we were looking for creative ways to get out of it.

25:51Charles-Axel Dein:Yeah, I don't know if you remember, but we had to ping an engineer in India to share with us the code that was sent via SMS. It was like so inefficient. I mean, the end-to-end test topic is a fascinating topic, I think. There is something to be said also about automating them too much because the more, first like payment and login flows typically are designed against automation. So I think that's one of the reasons we couldn't really automate them for a long time is because they are specifically designed to counter that. And then the other thing is when you automate too much, those like end-to-end tests, then you get a lot of flakiness because you're not only testing your flow, but in the case of this SMS-based payment flow in India, you're also testing the quality of the SMS reception, any latencies there.

26:42Charles-Axel Dein:You're testing many different flows from the banks that you don't control. So if those tests are constantly failing, then you get flaky tests and fakey tests gets ignored. One interesting challenge during the hyperscaling time was hiring. And again, most companies will not be in hyperscale more, but every now and then you need to hire really, really fast. This might be because you get a big funding round or for other reasons. What did you learn about what works when you need to hire efficiently and you need to hire a lot of people, oftentimes experienced people? You look at the whole process, right?

27:16Charles-Axel Dein:Like you look at every single step at the process as a funnel, as a pipeline, and you see where you do you have a leaky bucket. So from a metrics standpoint, it's very important to have that. The other thing is you build a partnership with a recruiter. So many people don't do that. You have to build this partnership so that you have a feedback loop where your recruiter knows what you're looking for and make sure that those candidates are sourced, are front screened from a recruiter standpoint, and then get through the process. And then the last thing I will say is like every single step of the process, you have a continuous improvement mindset.

27:51Charles-Axel Dein:One thing you remember we did at Uber, we do it at Cloud Kitchens as well, is pairing interviewers. You have two interviewers, and this way they can give each other feedback, they can train each other. There is only so much you can do when it comes to interviewing training. And so to pair interviewers, you can share with feedback between the two. Yeah, I remember the two things that we did, and I don't think many places still do it. It's like first, we had a weekly catch up with the recruiter. So, NGO managers and recruiters would do it. It's a bit like a team retrospective. And, you know, they would talk about things like what are their sourcing strategies like?

28:25Oh, what kind of cohorts are they targeting? LinkedIn has this new feature that I can. And they would share tips that were like, oh, pretty cool. Or like talking about what kind of companies they're thinking of targeting. And we'd be like, oh, what about this one? I just heard that there, I don't know, there's some negative news coming from there. Maybe people will be interested of joining from there. And I've never seen that before. and it was really good. And when we started hiring, we would just sit down with a recruiter, like spend an hour talking through, here's what I'm thinking, here's what this person would do, here's what I think a good hire would be.

28:59But the recruiters bring a lot of expertise. So they would ask questions like, okay, like for example, and a lot of clarifying questions. Like I get so annoyed as an engineer when a recruiter emails you and said, like I'm looking for five years of C Sharp experience because if you've done, you know, Java, Python, whatever, you can pick up C Sharp. But we talked this to you with the recruiter. So they asked, like, what is your tech stack? And we said, OK, Python, a little bit of Node.js. Back then, we were juicing Go. So they were like, OK, so we need people with expertise in these languages. We're like, no, no, no.

29:29As long as they did some language, that's OK. In fact, we don't care about the language that much. And they're like, oh, OK. So and because of this, we help the recruiter find better people or not reject people who could have actually worked out.

29:44Charles-Axel Dein:100 % like this relationship between the recruiter and the engineering manager is critical. And actually something that came from Tuan Fan, which you should have also on your podcast. He told me that every morning he would come and go and check out with the recruiters and say, hey, what are some of the interesting profiles you saw this week? And it's a great way to have this feedback loop. The recruiter bring a lot of value and have like their specific skill set. and you know who you're looking for, you know who your team is missing. So this is like, if you don't have this tight feedback loop, it's going to take a lot of time to hire.

30:21Yeah, and then we had this interesting thing with the pairing, as you mentioned with interviewers, we always had a primary interviewer and a secondary. And the primary one would lead with the questions. The secondary would sometimes step in and you can discuss in advance. And we had this concept of if someone signed off as an interviewer or not. And when they were not yet signed off, they might lead the interview, but then the secondary was a lot more experienced, they would give them feedback afterwards. So after the interview, they would come and say like, okay, here's some feedback on, you could have let the candidate talk, talk, talk a bit more.

30:55It was great that you jumped in and gave tips and so on. And we tried to, again, this was something I've not really seen before. Most engineers I think are just not really good at interviewing and they rarely get feedback. And we were trying to get this feedback going, set expectations. And then we also did some interviewer training, but as you say, it's hard to do interviewer training.

31:16Charles-Axel Dein:Yeah, so we have the same process right now. So you shadow first the primary interviewer and then you reverse shadow, meaning you drive the interview and then the other experienced interviewer will give you feedback. Yeah, you're absolutely right. Like just because you're a great engineer doesn't mean you're immediately a great interviewer. You can become a great interviewer with training and with on the job feedback. So that's why this dynamic is critical. So how do you know if you're in hyper growth? And how might you know that you're getting out of it? Again, you've gone through this at Uber.

31:50At Cloud Kitchens, you might have had some phases for it. What are kind of signs that you're like, okay, or this is just really intense growth and I need to operate differently in this mode?

32:01Charles-Axel Dein:Yeah, I would say it's a bit what we talked about in the beginning, right? So a lot of hiring and then things breaking left and right without giving you the time to fix the fundamental root cause. And I would say this is probably the best sign that you're getting out of this hyperscale is when you feel like, okay, now I have time to actually look at this thing holistically and fix and build a system that I'm pretty happy with. Not necessarily a second system from scratch. It's not always a good idea or rarely a good idea. But when you get into that territory, that works. Speaking of thriving in hypergrowth, one of the biggest challenges is maintaining release velocity while avoiding outages as you scale.

32:41From my personal experience at Uber, this is hard to do. Most engineering teams think they have to pick one, shift fast and accept higher risk, or slow down releases to stay safe. But here's the thing. Hypergrowth teams at companies like Notion, Brex, and Atlassian don't have to sacrifice speed for safety. They use the same toolkit that powers Meta and Uber's experimentation systems, and now it's available to every engineering team. They all use Statsig. Statsig is our presenting partner for the season, and they've built a complete data platform that lets you ship features and measure exactly what each change does to your key metrics.

33:15This is the same infrastructure that growth engineers use to run hundreds of A-B tests per year. Every feature release shows you exactly how it moves conversion, retention, revenue, whatever metrics you care about. Built-in analytics, feature flags, session replays, the full toolkit. No switching between different tools or trying to stitch together your own measurement system. The gradual rollout approach is crucial here. Start with 1 % of users, see how it affects your metrics, then progressively roll out to more users. If something goes wrong, instant rollback. If it's working, you can confidently scale it up.

33:48The validation here is incredible. OpenAI was such a heavy user of Statsik that they decided to acquire the company. Talk about product market fit. Static has a generous free tier to get started and pro pricing for teams starts at$150 per month.

34:01Charles-Axel Dein:To learn more and get a 30-day enterprise trial, go to statcic.com slash pragmatic. What can you do when you're feeling really overloaded? I remember, and you were my manager, but there was a time where you looked pretty burnt out. We had performance season and a manager left the company and suddenly you, their reports also report to you. You had something 30 reports and you had to do 30 performance check-in conversations. I remember that when you did the last one, the few of us more experienced engineers, we knew like we saw your calendar. It was full. And when you came out, we were there. We're kind of clapping that you're done because it took like two weeks.

34:44And I remember at the time, like it was so you had 30 reports. You were expected to get still get stuff done. I think you were still doing some hands on work. and like you could tell that usually you're pretty chipper usually, but back then you were not chipper. What did you do to get out of this situation or to push through it?

35:05Charles-Axel Dein:Yeah, I would say there's two things. There's one, personal productivity. It's a topic I've always been super passionate about. So I've always invested a lot. The first book I read on this topic was Getting Things Done by Alan, which is pretty good. I think it's also a section in your book, by the way. it's a critical skill right like the earlier you invest in your product personal productivity the more dividend it's going to pay and i really believe in compound interest right like you invested this and then the second thing it's a principle that works wonder everywhere it's divide and conquer so yeah you're right i remember that time you have 30 reports you cannot do everything so now you're going to have to find people who can take over from you some of those topics so Or I remember asking you to take lead a project, for instance.

35:52Charles-Axel Dein:And it's a great way because you give responsibilities to people. You help them stretch outside of their comfort zone and you're saving time. So it's a win-win thing. I remember that you did it. And what I liked about it, you were very explicit about the ask. You were like, I need you to, for example, lead this project. And I don't have as much time to spend on it. I'm expecting to check in and win you once a week. if you need something more before that, let me know. And then I think you also went more specific. I'd like you to make a plan on how you're going to do this. Let's check in with it.

36:28And it was, yeah, so it was a way for you to spend less time, give more responsibility. And actually people liked it because people were growing professionally and they also understood the reason of why this added responsibility is going to them. And if you're smart, which I think most people were, they understood like, this is a good thing because with more responsibility means that I now have more opportunities to prove myself. I can now grow in a direction either to be a lead or if they want it to be later a manager or into the staff direction and so on.

36:58Charles-Axel Dein:Yeah, this divide and conquer is very similar to what we do with software, right? Like we create an abstraction, we give it responsibilities, we design an API that is clean, and then evidently some details might leak, some implementation details might leak. So with a project, you're giving the responsibility of a project to somebody, but some issues might require your involvement and that's all fine. But then at the end of the day, that's still what you expect from the person who take over, right? Like you're expecting them to drive and to somewhat hide some of the implementation detail from you.

37:31Charles-Axel Dein:And yeah, so it provides growth opportunities, that's for sure. One of my managers told me like one of the most powerful words or expression you can tell somebody is I need your help, right? Like this is very powerful, very simple, right? I need your help with this thing. And And the employee feels also empowered to make decisions. Yeah. And by the way, I think you can do this not just as a manager, but also when you're a more experienced person on the team and you have someone who's less experienced. And it does work magic. I think it does create a sense of responsibility. I think we also tend to sometimes forget that as soon as you hire someone, even if they're a brand new grad, like these are fully capable people.

38:09Like sometimes I think back on how in the industry, sometimes we have the tendency to kind of over baby the junior engineers like, oh, this person has no experience. Well, if you think back of like the people who built Facebook into the trillion dollar companies today, there were 19 and 20 like dropped out of college. So all I'm saying is like people have, especially the people who go through a pretty like high standard hiring process, they actually have a lot of capability to do. And I sometimes need to remind myself of that when I was a manager or a tech lead.

38:42Charles-Axel Dein:Yeah, plus the best way to learn is to make mistakes. So, you know, if you want to invest in your people, you have to let them make their own mistake. Yuki is still right. One of your most popular GitHub repositories, which keeps going viral whenever I mention it, it's called How to Be a Professional Engineer. It has a few tens of thousands of stars. And it's this collection of like really, really good reading resources. How did that start? and how are you updating it? I understand that you started to do it for yourself initially. And in fact, you already did it when we were at Uber and you forwarded it to people.

39:16You were like, oh, check this out if you'd like to become a better engineer.

39:20Charles-Axel Dein:Yeah, I think the way it started is I wanted to automate the process of giving people feedback and sending them to certain good articles. Or back to automation. Yeah. So I started compiling this list of topics and I put what I considered like the classic article on this topic. So really like articles that have really changed people's mind that are really influential and kind of driven like a lot of engineering practices. And then I started in doing this. So one thing, speaking of personal productivity, right, that you can also invest in as an engineer is I have a good process for keeping yourself up to date with what's going on in the industry.

39:59Charles-Axel Dein:And so I try to read, I mean, total one hour per day fiction work as well. We can talk about that later. But I try to read a good bunch of engineering articles every day. And if I find a new classic article, I add it to the repository. And I've been doing that for the past 15 years, I think, something like that. How do you find where to read? What are your sources? Yeah, so the main one is Aker News. It's a good one, isn't it? It's the best. Yeah, it's still the best. Aker News, I receive an RSS feed with a 10 top article of the day. Plus, I check it because sometimes you have like some diamonds that don't necessarily make it to that top 10 list.

40:39Charles-Axel Dein:There are some really good newsletters. The Bytes one for Fontaine engineer, it's hilarious on top of this. Like every time I have a chuckle because it's like just so well written. I also follow like newsletter about the programming language that I use. So there's Python, Java for CK. yeah a lot of the software tech lead is really good as well you were an engineer yourself you manage a lot a lot a lot of pretty good engineers can we talk about a story of of a standout software engineer and what made this person stand out yeah there are so many so many good stories i don't i'm not sure i want to single out somebody in particular but some of the traits of like an archetypally typically good software engineer, I would say there's a couple of things.

41:27Charles-Axel Dein:There's one, shipping, right? So I've written three or four career letters and a lot of career letters in the industry, they, after senior, they over-focus on meta work. Reviewing RFCs, attending meetings, influencing that, strategizing this. It's really difficult to understand what's going on. So the first quality, and we try to put that in the Cloud Kitchen's carrier ladder is the focus on building, shipping value, being creative, being an expert in your programming language, in system architecture. So that's one. Shipping is really key, right? So even at the above senior, like the staff engineer level, like you still place focus on, you still need to ship things.

42:11You need to get into production.

42:12Charles-Axel Dein:Absolutely. And we expect, for instance, staff engineer to really find creative ways to speed up execution or achieve a 10x improvement in quality. And that's what we expect. Not only reviewing RFCs or attending meetings, right? So that's one, shipping and lifting, right? Lifting people around you. This is critical, right? Like we are knowledge workers essentially, right? So you have to train people around you. You have to give a hand. You have to help. You have to have a good attitude. I think the best engineers that I've worked with have this amazing attitude of ownership, of taking a problem and not stopping at team boundaries, for instance.

42:50Charles-Axel Dein:A couple of stories here, but like a software engineer was like identifying a problem and they're like a mobile engineer. And then they go into the API gate where they see the problem is not here. And then they keep going. They go to the backend and they actually make the fix in the backend even though they're a mobile engineer, right? And by the way, the amazing thing is in this age of AI, you can really bootstrap that with AI, right? I feel there's starting to be no real excuses because before AI, you could say it is hard to onboard. It's hard to get to know it. It takes time and effort. And I mean, it still takes time and effort to become an expert.

43:27But as you say, like to unblock yourself, like at an age and time when a product manager can open a pull request that might be accepted. I think there's no I feel engineers are necessarily becoming full stack with an expertise in their home stack.

43:40Charles-Axel Dein:Martin Foller posted a brilliant article, I think, about the expert generalist. It's funny because he mentioned that this T-shaped person that we often talked about, like this person who's an expert in one thing and one thing only. And then like broad on. It comes from the Valve software Unbook. But I've never seen that in real life. And it made me realize that actually, right, what you're looking for is more like a rake. somebody who can go deep in a lot of different areas. And with AI, you're absolutely right. Like for instance, one of the best use case I have seen so far for me is like, there's a problem with this feature, find me where the code is.

44:21Charles-Axel Dein:And then I read the code and I understand, or maybe I need to understand how much time it's going to take to fix this or that or to implement this or that. Find me the code where it is. And we use a monorepo at Cloud Kitchen and that helps, right? Everything is in one spot. So you can ask those kinds of questions and get this kind of answer. And so what other characteristics? So shipping, ownership? Yes, lift. So ship and lift. Ship and lift, yes, sorry. And I will add, there are so many things, right? But I will add one thing, humor. Taking yourself lightly is really important. I mean, it's evidently not limited to solid engineer, but I think humility and being able to self-deprecate.

45:04Charles-Axel Dein:It's pretty... I liked how you put it, Because I feel if you have self-deprecation humor a little bit, it's hard to have a big ego. Because I feel that's the thing that gets in the way of some engineers who are really standout and really smart, but they become insufferable if they keep putting their ego ahead of them. And it goes against everything. They're not going to lift others if they don't have that learning mindset. Yeah, and also sometimes you have those debates. So you have those debates on RFCs, on code, etc. And you see some tension and some conflicts. You more diffuse these things, right?

45:43Charles-Axel Dein:So it's a really powerful move. Yeah, there are so many other things like structure and method. I think a good solid engineer has like a method in place for fixing problems. It starts with observability, metrics, right? A good story here would be we are in an incident. You're debugging. Everybody thinks like the problem is in system A. And then you have like a really creative engineer who says, actually, it's here. And we are all looking in the wrong direction. And here's a super quick way to mitigate this problem. This is like also solid engineering. It goes back to the shipping thing. You can only do that if you keep close to the code.

46:22Now, in this repository, you have a lot of really good articles. And I love how it starts that I would like this repository to not be overloaded. So I will limit to the most important and impactful documents. And again, about 10 years ago when you sent it to me, it was a lot smaller, but now it is becoming a little bit overbearing. Just there's so much reading right there. Again, I really appreciate that every one of them is worth reading. What is your take on the importance of reading versus doing? Because if you read all that article, I'm sure it'll help you to some extent, but let's be real, if reading can make you a great engineer, then anyone could just read the whole thing.

47:05How do you think about balancing this thing? Reading, doing?

47:09Charles-Axel Dein:Yeah, absolutely. So evidently this repository is not designed for being read, like head to cover, right? Like cover to cover. You are confronted with a topic. And what happens is usually during the day, you're in doing mode, right? So you need to get to results. I think people will start looking at you sideways. if you start taking a book and reading a book on the job, like people usually don't really see that super well. So what you do is during the day, right, like you try to make it work and you're going to try many, many things. You might choose AI to unblock yourself, right? And then what you can do outside of work time is read a book about the fundamentals of the topic you're confronted with.

47:50Charles-Axel Dein:Because when you're trying to make things work, sometimes what you're missing is like the fundamental of this topic. And if you add them, it would unblock you and considerably speed up your process. So that's why reading and doing, they go hand in hand, right? And you have to do both. Yeah, so you're saying a good way is to read related to the problem that your head is in anyway, right? Yes, absolutely. And yeah, reading outside of work, I mean, I know people might be concerned about saying something like this, but I would say, hey, we usually spend a lot of time on our screens. So rather than spending mindlessly 30 minutes on a social network, why not reading for 30 minutes?

48:33Charles-Axel Dein:And by the way, you don't have to always read technical books, right? There are some really good non-technical books that are also useful. One of the best ones I've read is Complication by Sergeant Atul Gavande. It's an amazing book about how do you learn from mistakes and how to have a scientific process when it comes to learning from those errors. And there are so many other books like that. Another good example is like all of the case incident post-mortem, aviation incident post-mortem. You learn a lot of things about the human component of like an outage that are very applicable to software incidents.

49:10Yeah, especially because in software incident reviews, we try to keep them blameless. So we keep the human part absolutely out of it, which I guess it's good in some ways, right? There's no finger pointing, but it can miss the very human nature of like, I mean, we're people, we make mistakes. It happens and it's kind of to be expected as well.

49:31Charles-Axel Dein:Yeah. Blameless is very important and the intent is right, but sometimes it's not well executed. And when you completely remove the human component, I think you're missing 80 % of the incident potential mitigation. Yeah, like there was this incident where I remember it was a small outage at the middle of the night at 2 a.m. The engineer woke up and turned it into a big outage because, I mean, you know, like they were just tired. It was frigging 2 a.m. They thought they were doing the right thing. But because it was a blameless postmortem, we kept going about how the system could have prevented this.

50:03And I'm like, I'm sorry if you're like, what could have prevented this is like not having to make a decision or that end you're not deciding to try to mitigate it on the spot. And I mean, there are some things, but like there was that context of like, look, like when you're waking up in the middle of the night, like either it could have been OK to ask for help or it could have been fine just to leave it and kind of come back into the morning. But again, because like this was one downside that I saw and it doesn't always have happened like this. But in this case, that very important thing was missing.

50:36As a tired person, you're going to make mistakes.

50:39Charles-Axel Dein:Yeah. And being too blameless or at least not involving the human component removes the opportunity for finding training needs. Right. And as an industry compared to all of the other industry, we do so little training. So, for instance, I was talking about like the future flag. if somebody's never been trained on why it's a good idea to use FeatureFlag, how to use them, what are the internal tools that you can use, what are some case studies, sure, they will not use it. And it's a huge missed opportunity. So if you don't cover those aspects in a post-mortem, you're missing those training opportunities.

51:13Charles-Axel Dein:And yeah, we need to also be completely okay saying, yeah, mistakes happen and it's a way of life. And as my mom used to say, only those who don't do anything don't break anything. Yeah. speaking about training i i had this really interesting experience with you i i'll see if you remember so back when i started at uber we were in hyper hyper growth mode and we had a lot of projects going on and engineers needed to lead projects we we just didn't have the product managers didn't have the capacity to lead them and so you decided like okay well since everyone's leading projects we might as well get people some training on what does good project management look like because we're all software engineers we're not the experts on this you looked and you found an expert on this you brought that person in for a two-day training and at lunch break all hell broke loose engineers were like what the hell is this like this guy like is talking about some old school thing that doesn't work the rule was so big we canceled the the training and i still think of this of like what went wrong or like could have we even done training or were we just doing things that was just not trainable or would have we needed to do our internal training?

52:24Do you remember this?

52:25Charles-Axel Dein:I absolutely remember this. It was a very humbling experience because I did choose the training company. The main learning for me is you have to build your own training. Only you know what your team needs. And the mistake I made was to have somebody else come in and do a training. Because you spend a bunch of time selecting who to bring in. Yeah, no, no. I mean, a big mistake for sure. Since then, very simple. I just build my own trainings. I know exactly what the team needs. And the other thing is I keep on improving it. The true thing is like project management is not rocket science, right? Like we could go into details, but it's essentially managing the trade-off between time, cost and quality.

53:09Charles-Axel Dein:And once you've said that, now you need to apply it to your team, give case studies, have people engage, and only you can build those trainings, right? And then that's what we actually did afterwards. I mean, I don't think it was intentional, but I started doing this. I started to put a document together for my team saying, okay, here's how we usually do it. Here's examples. Here's like pointers to internal sites. Here's how we try to keep it as lightweight as possible. Because I think that's the thing with a lot of these things. I wonder if this is why training is possible. Well, you, as an engineer, you want to make the process as simple as possible.

53:41You need to have some, you know, like, I don't know, checkpoints. Sometimes they're optional. But in the case of project management, for example, I think the only thing we had is, okay, you know, kind of do some planning upfront, decide how you want to do it. It's up to you. Do you want to do a big meeting? Do you want to do whiteboarding? Doesn't matter. Put together a plan, make that a document, send that out for comments. That was like kind of a fixed thing. And then I think the only thing we mandated is have a kickoff meeting because we realized that when we don't have a kickoff meeting, the projects don't go that well.

54:09And then I think the third thing we said is like every week, send a weekly summary to your team, everyone on your team, and a few stakeholders. And there was a bunch of recommendations in between. You could do this, you could do that. But these were the three things. And we did it because over time we figured that this works for us. It might not work. Another team might have added additional things. Again, I know some teams, they mandate, let's say, retrospectives and all of those things. But in the end, every team kind of came up with their own process. Some teams kind of forked it or copied this document.

54:40They added some things. They removed some things.

54:43Charles-Axel Dein:Yeah. You have a ton of good content on your blog about project management. Maybe one thing that most people don't realize is this weekly update thing, which is something I learned at my first job, an internship in South Africa. And my manager was sending this weekly update to almost the whole company. He does so many things, right? So one, it forces you to be explicit about your goals, your success, your lowlights, what you can improve. So that's great. And two, it's a good perception tool. It's incredible how just sending a weekly update drives a perception of movement. And it's critical. You have to manage the perception of your work.

55:24Charles-Axel Dein:It works for a team. It also works for an individual contributor. You've been tasked to do this or that project. Very simple move. Send a weekly Slack message. This is what I'm planning to do this week. This is what I did last week. That's it. Pretty much like a stand-up and maybe some key metrics, right? In writing, it works wonders. Yeah, and I really liked how you suggested that we do. For every week, we did the status of, you know, like, is it on track? Like, when are we planning to have the next thing shipped or whatever that was? Highlights, you know, some good things. And then we also had lowlights.

55:58We just tried to keep ourselves honest. And what I actually found is by adding lowlights and by being honest, and of course, with lowlights, when there's like something at risk, like mitigation, what are we doing about it? It just kind of forced conversations early. We didn't really have surprises. When a project was late, we knew it well ahead. And when we didn't, we then talked with the team or the engineers saying, did you actually know this? And usually it turned out that they kind of knew. They just didn't want to bother with it. But by having that trust, again, it wasn't about like, you know, you're going to get in trouble for being late.

56:29It's like, no, we just like to know. So we can help. We can talk about it. We can talk about the trade-offs because it's all trade-offs, right? Like with project management, you can always decide when a project is looking that it's late. You could either cut the scope. You could try to pull in more people. You could just extend the timeline. Yeah, I mean, I think that's the three things you can really do.

56:48Charles-Axel Dein:Yeah, absolutely. So the lowlights is an important section. I don't always include it, frankly, but one thing I always make sure to include is ETAs. A deadline is the ultimate source of inspiration, right? Because if you look at the cost, which is essentially the engineering time, it's always a dynamic thing. How many engineers are working on the project is not always like super easy to measure. The quality, which is the scope and what the project is aiming to ship, is also very dynamic. Like you have macro optimization at the feature level that people might not have. The date, super objective, right?

57:24Charles-Axel Dein:We said we would ship on that day. We're going to be two days late. And then the discussion about, is it okay? Should we cut scope, increase the number of engineers, which does not always work, right? Mythical man months. But then those dates and making sure they are in your weekly update, ultimate source of inspiration when it comes to project management trade-offs. And I feel that getting better at project management, managing your own project that you're working on, whether it's a one-person thing or a few people, it's a superpower as a software engineer. Because I think as software engineers, we think that, okay, our job is to write code.

58:00But actually, our job is to solve problems and to ship solutions to problems that the business has, may that be new features or products. And you become more experienced. and frankly, you know, higher paid when you can take on more complex things. And there's only so far you can go without project management. So I remember that at Uber, there was some pushback. Some people said like, I don't really want to do project management. And I was like, well, okay. So I guess two things that happened. Like one is someone else can do it and you'll just, you know, you're not going to do it. Or like two, we could bring in a dedicated project manager.

58:33We could literally hire a person and they will be the project manager. Non-technical. They're going to keep pestering you for updates. you know, you're probably going to hate them.

58:42Charles-Axel Dein:Yeah. But a lot of places still work like this. And some of it has to do with some people are just really resistant. So I don't want to do it. Great. Well, you might actually be worse off. Yeah. I've never worked at a place where you have like those non-technical project manager. Like usually you have this TPM role, but it's most effective, I think, when you have like a project that impacts 20 or 30 teams. Very useful. But then when it comes to like, once again, it's not rocket science. It's like a weekly update. It's making sure the project is on track, making sure it's still going to be delivering the value that people are expecting.

59:18Charles-Axel Dein:And I would even go as far as saying that project management is one thing, but I would also recommend getting into product management, right? Because the irony of the software engineer's work is there's a lot of blame, a lot of things that can be blamed on the software engineer, right? Like a feature is buggy, hasn't been tested enough, there's an incident, a wrong architecture decision was made. And it's very objective how you measure. There is one thing that is relatively difficult to measure and that most companies, I think, don't do a great job at, which is making sure a product is doing the right thing and was the right strategy in the first place and is shipping the right value.

59:54Charles-Axel Dein:So as a software engineer, if you don't do that, you risk wasting not days, but weeks, maybe months of time working on a feature that is actually not going to move the needle for the customer. Yeah, or even years. I think we see products that even like larger companies bring out, which are just like big flops. And they put so much time and effort into this. I mean, we could even see it at Uber, projects that were killed. I still remember to this day, we had a team who were doing jump bikes and they were integrating it into the Uber app and they were changing some parts of our stack. And they came in and they didn't add tests.

1:00:35Like they added a feature or something and they didn't add tests. And we kind of got into standoff with them because we're like, I mean, you know, like our quality bar, it needs to be tested. In fact, I think they deleted tests even worse. and at first I was like like this is how can they call themselves software engineers like we're at the same company if if they they miss these quality things and for us it was really important because we knew that uh we want to have stability we have the real world impact and so on and after a while when I talk with them I start to understand that they're coming from a different angle they were not sure if their team would exist in two months and actually it didn't in like six months it was kind of sold off and and uh or moved to a different entity but they were just in different stage they were in in survival they were like let's just get it to work let's test it if it's there and this is what made me realize that there's just different levels of software engineering you know when when you're working on a mature product that's making a bunch of money which was us you have different goals than when you're this was a startup and mvp stage They weren't sure if it would even work.

1:01:39Charles-Axel Dein:Yeah, the time cost quality management is very different depending on where your product is. So if you don't understand that, you're not going to be making the right decision, in particular when it comes to technical debt, right? Taking on technical debt is a perfectly valid thing to do when you don't know if your team is going to exist next month. It might be much trickier if you're building the architecture for the next few years. and I would say it's especially the case for internal projects especially because internal projects you have an internal platform usually don't have product manager so in a way you are also the product manager How would you suggest software engineers to to learn to kind of make a balance on tech depth on how much to focus or how not to focus or what is kind of your mental model of deciding like how important is tech depth or not because again I've seen most engineers I think that care about the craft can kind of frankly go overboard on this.

1:02:36You're trying to fix it, make it perfect. But honestly, it doesn't matter all that much. But on the other side as well, right? We have this kind of hacker mentality who just gets things done. And yeah, it's a mess. It's spaghetti code. It's hard to maintain after.

1:02:49Charles-Axel Dein:Yeah, I would say two things. One, it's tech depth if you know it's tech depth. If you don't know it's tech depth, it's just recklessness, right? It's moving fast and not knowing what you're doing. It's, I think we call it vibe coding those days. Right? That's not tech debt. It's just recklessness. So you have to know, you have to care a lot about your craft to know where you're taking a shortcut. And two, I would say condone the technical depth, right? Like hide it behind a clean interface, use good patterns to make sure that then you can revisit that technical depth with minimal changes and keep track of it, evidently.

1:03:22One thing you've done a lot is hiring. You've hired so many people over time. Right now, the job market is kind of bouncing back a little bit, but still pretty stuff out there. as software engineers, as engineering managers to get hired. What is your advice on a software engineer, let's say experienced software engineers to get hired? How would you suggest that they prepare?

1:03:45Charles-Axel Dein:Yeah, so first you're absolutely right. The press and the media is full of articles about lowering a difficult job market for engineers. You've shared an article yesterday, but it's not what I'm saying. We are hiring, for instance. So I would say don't listen to those news. I mean, there are so many advices on the internet about preparation and things like that. So it's difficult to say original things about it. But I would say definitely to have a structure and a method. You come in, most of the interviews you will have are technical interviews, hands on. If you have a good method, if you have a good structure, the interviewer even if you don't succeed necessarily at solving the problem at times will see that and will reward you for that i think so for instance for algorithm it's to start with a brute force solution and then to keep improving and optimizing right for a system architecture interview it's to start with a product and then go with like the requirements technical and non-functional and functional and non-functional and then start with an architecture etc focusing things on the trade-offs and things like that.

1:04:53Charles-Axel Dein:So I think the structure method, I would say, definitely super useful. Then there is a topic of AI. Do we want to talk about AI in interviewing? Because I think that's top of mind. Yeah, but before we get to the interview, when you're hiring on your team, I guess it starts with either applications coming inside or you also have so-called outbound sourcers who you talk with them and you tell them, here's this hyperprofile I'm looking for. what could help you being discovered uh i guess this will be your linkedin profile and and also for for resumes is there anything that again there's been so much advice i even have a book about resumes but but is there some resumes that like you know you think are better than others the main thing people will look for is what are the experiences right your experiences which obviously is not a valid advice for for people who are just out of college for instance but I would still recommend that they promote like the projects that they've been doing.

1:05:51Charles-Axel Dein:And for instance... Well, and also I think this is important because when you're choosing your next job, if you have options, you could think about what will help my resume stronger, all things being equal or similarly equal. I had this early on in my career when I had the opportunity to join Skype. I actually like I... It was the first job that I didn't take any pay rise. It was the exact same thing, but I would have taken a pay cut to do so because I knew that that was the first company that I actually recognized. And later, once I got there, suddenly messages started to come in because I was now at a company.

1:06:26At the time, it was well known. You know, nowadays it's discontinued, but still.

1:06:30Charles-Axel Dein:Yeah, you picked the right first experiences. You're absolutely right. Like you came up on our radar because of like the experiences that you had. So I would say go toward companies that are known for the quality of their engineering because that's going to be the main feature. As far as like resume and LinkedIn, I mean, don't overthink it. Most people don't spend too much time on it. Don't use AI. I would say don't use AI to generate your resume. I've spent the last few days reviewing 30 resumes, 25 out of those 30 were AI generated. Some even add in the skills, attended stand-ups. And there were a lot.

1:07:08Charles-Axel Dein:So now I can tell like an AI generated resume, I mean, it has a lot of like those relative number improvements. So there's going to be reduced deploy time by 30 % and no absolute number, which reduced by 30%. I mean, if you reduced it from two days, it's, yeah, sure. It's not very impressive, 30%, right? So don't use AI. Keep it simple. Keep it short. Keep it creative as well, right? Like sourcers and hiring managers and recruiters, they review hundreds of resumes per day. So if you want to stand up, be original, be creative, don't use AI. Yeah, and I think maybe as you're looking, it could be worth looking at other people's LinkedIn profiles and just making notes of things that you think are kind of cool, original, and you can use some of that as inspiration.

1:07:58Charles-Axel Dein:And this goes back to choosing the right company to work for, which is what are the technical problems that are going to really move your career forward? So that's why also it's a good idea to prepare questions for your interviewer. And one of those questions should be, what are the technical challenges that I'm going to be exposed to? Because then it's going to give you a good direction for your career and you're going to be able to check whether this company makes sense or not. And then once you're on the interview and you're talking with, I mean, the technical interviews, I think that's pretty straightforward.

1:08:30You just need to be good at it. But when you're talking with the hiring manager, someone like yourself, you know, like who are people who do pretty well on the interviews? I'm assuming don't ramble, for example.

1:08:40Charles-Axel Dein:Yeah, I would say don't ramble and also short answers, right? The dynamic of the interview, it's the same here, is like somebody's asking questions and driving the conversation. It is what it is. So if you use all the time, the interviewer is not going to be able to ask that question. For the hiring manager, I mean, it depends if it's a cell call or if it's a behavioral interview. But by cell call, I mean, there's calls where you're not sure if this person is interested, but you would like to convince them to apply typically for more high profile people, oftentimes industry known, maybe people who are at a company that is known to have really good benefits.

1:09:19For example, someone working at Meta right now, it's kind of hard to sell them because the stock is just going upwards. So their compensation might have gone up, right? So that's a sell call.

1:09:27Charles-Axel Dein:Exactly. And then you can prepare by asking smart questions, right? So you research a company and you have a good list of questions to ask. I think the best candidate prepares the interview, even those conversations with the hiring manager. And what this says about the candidate as an interviewer, it says that if this candidate prepared the interview really well, that means that they're probably going to also prepare their work really well when they are at the company. It's kind of surprising that we even have to talk about these things. But I talk with a recruiter at a publicly traded tech company.

1:10:00There's a few thousand people there. They're a well-known company. I just don't want to say the name because I don't want to put them on the spot. But this recruiter said that they were recruiting for director and VP level candidates. And the people who are coming in did not know the company's products. Like this company makes a bunch of different products. And she said that like 90 % of people just didn't do the basic research of like, and we were not talking about software engineers. So I think I'm going to just plus one that when you get to an interview, first of all, it will be a red flag when you have been ignorant.

1:10:32And you actually, interesting enough, you will stand out by putting in a little bit more than the basic preparation, which, by the way, could just be interesting to learn about a different business, what they do, think about what their business model could be. This goes back to you can actually use this later when you're in an elite position. Maybe one day you'll become a founder. All useful information.

1:10:53Charles-Axel Dein:Yeah. And you have to manage your career, right? So if you don't know anything about the company you're interviewing for, you cannot know whether this company will actually empower your career or not. So this preparation is also in your interest. Well, yeah, and also preparation is in your interest to see how the company is doing, for example, financially and business wise. It would be nice to know if there's a likelihood of the company doing cuts in the next six to 12 months, which might be hard to tell from the outside. but if you do zero preparation on the podcast, I had Jambi Klarna, who's at OpenAI, a software engineer.

1:11:27She interviewed at 46 different companies and she researched all of them. And then she made this list of like, based on, she actually researched their products. She tried to use it. She looked for Reddit reviews for all of these to decide finally where she would go because she wanted to go to a company where it has a good trajectory. You know, her stock could be worth something, et cetera. And I realized, well, yeah, I think more people should do that.

1:11:51Charles-Axel Dein:Yeah, if you're that careful about your career choices, I think that says something also about like your qualities as a software engineer. We were talking about like a previous colleague of us, right? Like who has the same structured mindset when it comes to picking their next startup, right? And that applies to everything. You have the same structure and method for picking your next career opportunity and also to pick your next technology to use in this or that architecture. So with AI, how is AI, from your perspective, creeping into the hiring process? It's everywhere, right? You already told me that you can already see the interviews, sorry, the resumes that have been written or prompted by AI.

1:12:35But what else are you seeing?

1:12:38Charles-Axel Dein:Yeah, we see it a lot in interviews as well. So for instance, we explicitly ask not to use AI. It's actually quite interesting. I think we'll go back probably to on-site interviews in the industry a lot more than we used to. It seems unavoidable. And by the way, this is not to say that AI is not a good tool when you're on the job. It's just a really bad tool when you're interviewing somebody and you're trying to get signal because you don't know if you're only testing their prompting abilities, which is a very important skill, but it's not the only skill. So that's one. So I would say use AI as a coach, as a trainer, but certainly not as a cheating partner.

1:13:15Charles-Axel Dein:And as a trainer, it's a really powerful move, right? Like you can, for instance, you can prepare those questions about the company and ask whether those questions make sense. And you can also prepare an architecture interview. There is so much you can do to prepare and train yourself, but I would not use it during an interview. So you've worked at a couple of fast-paced startups, Uber. You later worked at other startups. You're now at Cloud Kitchens. What have you seen people do to be successful at these startups? I've seen when I hired people at Uber, a lot of them hit the ground running, but some of them struggled because it was just a big pace change.

1:13:50And these are all fast-paced startups. And, you know, hyper growth is probably over, but fast-paced startups are here to stay. In fact, what I'm seeing with startups of today, they're even faster. They ship faster. They get to product market fit faster. what have you seen engineers do who do well in this environment?

1:14:09Charles-Axel Dein:I would say first like personal productivity helps a lot. So investing in this early in your career so that then you can hit the ground running really rapidly and you stay structured and methodical when you're on board on a new company. And I'm sorry, but by saying investing in personal productivity, so figuring out how, what works for you, like what kind of methods, may that be to-do lists or Pomodoro or focus time or whatever, Exactly. Yeah. So how do you keep track of your commitments? How do you keep track of your reading? What worked for you or what have you kind of what phases have you gone through in this sense?

1:14:42Charles-Axel Dein:I mostly use a method called getting things done, which is not rocket science, fairly simple. And then I've used the same tool, which is called SYNS for the past 15 years. I think a plain to-do file. And then I also use a GitHub repository for my personal notes. That's called PKM, personal knowledge management. That helps a lot as well. And I try to memorize stuff as well with flashcards and key. Those are things that I've used consistently, and it pays dividends still to this day. For flashcards, what kind of things do you memorize? So, for instance, I memorize a whole Python standard library.

1:15:22Charles-Axel Dein:Not everything, but a lot of it. And it did help a lot in the beginning. And then you memorize like architecture patterns, data science methodologies. Love it. Standard library of your current language or like some issues you have, bash commands that you use consistently. Yeah, so I would say personal productivity does help reading fast and being able to synthesize and summarize like document. I guess the best way to do it is to practice, right? So like read. Read, exactly. Definitely prioritizing meeting with people and understanding their context. Because there is only so much you can find in the internal documents and in the code, right?

1:16:07Charles-Axel Dein:So understanding the history of the technical decisions helps a lot when it comes to maintaining that platform later on. So I'm not sure if you recommended this or not, but at Uber, one thing I started to do after a while is meet when I became an engineering manager, meet other engineering managers on the team. Or when I went to, for example, San Francisco, we were based in Amsterdam, spend that time to meet with them in person. And then starting with like I edited a short summary of short presentation of our team, just a few slides to say, here's what we do. It was meant to be really short, but it was just a conversation starter.

1:16:39And then I asked like, hey, what do you do? How did you get into Uber? What are your plans after? And then as you warmed up, what are your plans after Uber? Like, where do you actually want to go? And those things have become really useful. And I realized, like, as an engineer, I probably should have done more. And I saw some of the best engineers. They actually just met in person with another tech lead or another engineer on the team or grabbed lunch with them. And it makes all the difference.

1:17:02Charles-Axel Dein:Yeah, I think AB talked a lot about it. I think it makes sense. She might have been the instigator. It makes a lot of sense, right? Like you're going to work with people. You're not going to only interact with God. So getting to know them on a personal level is critical if you want to work well with them. And they'll just be nicer to you. And it's nicer. Yeah. And you'll be nicer to them. Yeah. So that's why coming back to this like humor and self-deprecation, you want to work with colleagues who are really good at what they do. That's for sure. But you also want to have fun, right? Working with them.

1:17:36Charles-Axel Dein:Like it shouldn't be like an annoying thing or frustrating thing. So the best way to do that is to get to learn because I'm convinced that everybody has like an interesting story and stuff you can learn from them. Yeah. And I guess one thing like in a fast girl company, like conflicts are a bit more common just because people try to get things done. There's a lot of email or Slack messages. And sometimes, you know, they can come across as aggressive or passive aggressive. And I still remember to this day where this was again back at Uber. We were emailing a lot back and forth with the San Francisco platform team.

1:18:10and there was this guy who was just an absolute asshole at times just like in in in the response and then once he was in Amsterdam and he was such a nice guy and but once we got to know him you know and someone asked like why do you write this like oh I just write really fast like it was really late at night turns out he was writing at 3 a.m and he wasn't caring too much and like every now and then I get back to the point of like well I guess software is oftentimes more about people or at some point people start to become really important, especially inside a company.

1:18:41Charles-Axel Dein:Yeah, that's why my number one advice when a code review is not going well or like an RFC review is getting into terms of companies, just meet, right? Like meet face to face. You're going to hash this through in less times and it's going to take you to have all those exchanges and frustration and everybody's second reading. The thing, I mean, let's be honest, also a lot of people in startups are like English as a second language. So we don't necessarily understand all of the nuances of the language, contrary to code, right? And also some people cannot express their own nuances or they have a hard time, especially in writing.

1:19:18Charles-Axel Dein:Yeah, absolutely. What are some non-obvious recommendations to thrive inside a high growth startup? Yeah, non-obvious. So I talked about memorization. Yeah, using flashcards to memorize stuff, very powerful. and second I'm not sure if it's super non-obvious but extreme ownership so that means you you feel like the team is yours the project is fully yours and really going out of your way to understand all of the dependencies and and overall context it fits in is really critical to be super successful I talked about like the best engineers don't stop at team boundaries they will keep going our industry our work is mostly about building abstractions on top of abstractions on top of other abstraction, in reality, there's always leaky abstraction details, implementation details, right?

1:20:07Charles-Axel Dein:So the more you understand about the stack you rest upon, I think the more effective you're going to be as an engineer. It also applies to your tools. It applies to your programming language. It applies to everything. Yeah. And then one thing you used to say a lot is under-promise and over-deliver, which I guess is easier said than done. Yeah. This is where also like the weekly update helps. is like you set your timeline and if you find like a creative way to reach a timeline faster, it's a pretty powerful move. When it comes at working at high scale, at fast-paced startups, a lot of people have imposter syndrome.

1:20:44Did you have this? And did you see people have it? And how did you work around it? How did people conquer this, who you work with?

1:20:55Charles-Axel Dein:Yeah, you always have this. I think to this day, it's a good thing to have it, actually. We often talk about the imposter syndrome as a bad thing, but I think actually it's the engine that drives your curiosity and is like what's going to move you toward essentially like improving yourself and having this continuous improvement mindset. What's really interesting also in the world of like startup mindset and culture and things like that is when you look at it, have been really interested in ancient philosophy and stoicism. And when you read those ancient philosopher readings, it's still very applicable to us today, right?

1:21:35Charles-Axel Dein:Which is don't over-focus on your emotion, but just get at it, right? Just try to work on those aspects so that you become a better engineer. I want to do just one quote, which is Dan Heller, which we both know. It's a really good way to phrase this thing, I think. Imposter syndrome is underrated. A lot of talk goes into overcoming imposter syndrome. I say embrace self-scepticism and doubt yourself every day. What else to add, right? Like it's a good drive. It's a good engine. Even Twan, the city of Uber, acknowledged it. And I think it takes a lot of humility to say that you feel like you have imposter syndrome, but it's a good thing.

1:22:13And I guess you typically have impulsive syndrome when you feel people are smarter around you, where you feel you might not be fully up to speed with wherever you are, which is typically at a high growth startup, like it's going to be a rocket ship, right? So I guess it's kind of natural to feel that, right? In fact, if you didn't feel that, like you might ask, are you at the right place? Is this really truly a high growth startup, which has ambitions beyond your own ambition?

1:22:36Charles-Axel Dein:Yeah, you want to be impressed by your interviewer, for instance. That's a good way to know whether you're going to be at the right company next. What are some non-obvious reading recommendations you might have for engineers who want to get better? Yeah, so I would say read the fundamentals, not necessarily only the most recent technical books, right? So for instance, one of the books that I, non-obvious book, I would say, is like the Linux programming interface, which goes into what is the API that is exposed to by the kernel. It's fascinating because not only does it explain the kernel and it's super useful because most of our stack...

1:23:13It's built on top of Linux.

1:23:15Charles-Axel Dein:Exactly. So that's one. And two, it also goes into the historical technical decision. There are some really interesting algorithms as well. Like why did the kernel choose a red-black tree, for instance, instead of hash maps for certain data structures? So it's really interesting, really fascinating. So fundamentals, fundamentals about your programming language, Adjustment fields. So I mentioned complications by Atul Gawande. Fascinating read. And I would say fiction as well. Read a lot of fiction. First, because, I mean, it's not all about like work and also because it trains you to be a better writer.

1:23:49Charles-Axel Dein:Reading good fiction is a great way to improve your English, improve your writing skills as well. And I guess your reading skills. You mentioned how it is helpful to be able to read fast and digest information fast. And I guess it's like going to the gym and, you know, working out, you're now working your kind of mind. Yeah, I've also been reading a lot more philosophy lately. I mean, evidently software engineering is a very scientific endeavor, but there's a lot to be said also about the fact that we handle concepts. There's a very conceptual work that is very similar to philosophy. And so, for instance, I listened to your interview with Dr.

1:24:27Charles-Axel Dein:Osterhout about the philosophy of software design, right? And he mentioned that at the core of software design, you have this decomposition of problem. Well, it's a core of human thought. It's a core of reason, right? Like reason is about distinguishing between different concepts. And I mean, it's not an irony that there is philosophy in the title of this episode because it is what it is, right? Like philosophy is exactly that. It's distinguishing between different problems. That's really what classical philosophy is about. Yeah, in fact, when it comes to, for example, designing a new system, You kind of need to go down a path.

1:25:07Like you can argue if it's a philosophy or not, if you're doing microservices or monoliths. But the argument can turn into kind of philosophical. There is no one right or wrong. And I guess at some point you need to go with one of them. It's fascinating how there are some parallels.

1:25:23Charles-Axel Dein:Yeah. When you're writing an RFC, you're making a case. So then there is a big component of logic. How sound are your arguments? So, for instance, at Cloud Kitchen, in the competencies, we have a big section about truth. And we have one competency, which I like pretty much, which is called QED, right? QED? A quad erat demonstrantum, what needed to be demonstrated. And it focuses on how well do you make your case? Do you eliminate bad arguments? Because when you read an RFC and you have like one good argument in the middle of like 15 bad ones, you feel like yeah I'm wasting my time. As a reader it helps you uncover that and as a writer it helps you realize that so that you focus on this good argument that you have and you make it really strong and it really helps your career I think.

1:26:11It almost feels like if you're a good debater and you're also good at laying out your arguments in a logical way it helps you be a good software engineer because we need this right? It's a logical field where the best people like obviously there's a human part but you need to build up your arguments in a way that's understandable, easy to read if it's in a document, which a lot of it is.

1:26:33Charles-Axel Dein:Yeah, there's two virtues that are really important. And I think under-focused in the startup world is courage, like the strength to go against difficulties and maybe opposition and humility. Because also when you're focused on logic and truth, you can also say, hey, I was wrong. I mean, sorry, you make a case, somebody comes back with a really strong rejoinder and you say, well, yeah, you know what? You're right. This is not the right architecture. And it's a great way to grow, right? Yeah, and the best engineers always share that. No matter how senior or experienced they are or how fancy they had, they had a roof in mind and they would change there and they would go like, okay, yeah, let's build this idea or let's go with your idea.

1:27:14Zero humility. A lot of the advice that comes out of, you know, like people who worked at Uber at a hypergoat time or today it'll be at OpenAI, we could argue a little bit. Is that survival bias or not?

1:27:27Charles-Axel Dein:It is. I think it is. It's definitely Survivor bias. You listen to us, for instance, and we're going to talk about all of the things we need to fix some of the chaos. But in a way, you could argue that the chaos is probably why the startup was successful in the first place, right? the speed up that you get when you're focused on shipping definitely has some trade-off in terms of quality and maintainability and reliability. But I would submit that in most cases, it's probably not what you're looking for. You're looking for first building a product, building a successful business, and the rest comes after.

1:28:09Charles-Axel Dein:And I think the industry has a tendency to over-focus on what comes after, the quality, the investment, in the internal platforms. Well, actually, yeah, it's fine in the beginning if it's not standardized, if it's not that structured, if what you get in exchange is speed. I think Uber had this thing, which we just got rid of just as I arrived. It was called the ping. Every five seconds or so, the app would send a ping to the server. It would ask me, send me the data, and the server would return a blob of information, the waiting times, the EDAs for the different categories, and a bunch of other things.

1:28:49And the app would display that data. So this was a pull mechanism. And initially, it started off with a few pieces of data. And they did it because the mobile developers were tired of waiting on the back-end developers. And they could just add things into this ping package. And it became a pretty large package. And it was just causing a lot of data. And in 2015, Uber was still working like this. It was now scaled to millions of people. And it was just really, really inefficient. but then eventually i mean it kind of worked and a refactor came and then uber changed it to be a pole where the so it was pushed so the server was now pushing and it was far more efficient faster shorter delay etc but in the end i was like it worked there's so many examples like that we were

1:29:31Charles-Axel Dein:talking about internal tools and and automation uh in the beginning right i think when it comes to automation a lot of automation project over focus on quality and constraints when actually people want speed and being in control. There is this fascinating article about malleable software, software that leaves a user in charge. Best example being spreadsheets. So many internal tools start with a good Google spreadsheet because there is so much you can do, right? You don't need to ping an engineer to add a column to your table. You can just modify it. And what's great with spreadsheets is it's the purest.

1:30:07Charles-Axel Dein:What you see is what you get, right? Like it's everything in front of you. It supports mass change. I would submit that a lot of startups have an incomplete buggy implementation of Google Spreadsheet somewhere in their tools because it's so effective. One interesting story from UberTime, the user's table and the trip's table had two interesting columns. One was called user and trip tags. So user tags, trip tag. Another one, which was a free form list of tags. And another one was user attributes and trip attributes, which was a key value stored as a JSON in this column. It was insane when you think about it, right?

1:30:49Charles-Axel Dein:And it was massively overused. A lot of teams did not want to add columns to the trips and users table. So what they did instead is just stick stuff in those columns. Free text pretty much blob. And then it started exploding, I think, roughly after I left, which is fair. but so many products were bootstrapped thanks to that. It's absolutely amazing, I think. So that's why those really shitty internal tools approach, they do work because you get speed out of it and you might validate very quickly that actually you don't need this feature or product in the first place and you're going to save a ton of engineering time, which is usually the limiting factor.

1:31:28So is it fair to say that what you're saying is if you're at a fast growth place, instead of, you know, reading what other fast-go companies did, their blogs, their best practices, maybe you're better off just looking at your most pressing problems, solve that, and then do the next thing. Or like also, or as with anything, there's a balance here. Or just be more like be skeptical whether, you know, the company like Uber or now we're going to hear about OpenAI or all these like very successful companies when they say why they were successful. Maybe that's not the full story.

1:31:58Charles-Axel Dein:Yeah, I would say because usually the limiting factor is the engineering time, I would say optimize for leaving the user in control so that they can change things on their own without having to involve engineers too much. Put the guardrails in place so that it does not catastrophically break your system, but leave the user in control because you will get so much better product adoption. By user, I mean usually internal operations team, internal business team, your product manager, lean on this direction rather than having a super constrained product that takes a lot more time to ship and that might not even solve the problem perfectly because you are missing this or that ops contact, particularly the case when you build a product for the physical world.

1:32:43So one of the big changes we're having, obviously, is AI, AI coding tools, AI use everywhere. What do you see? How is AI already changing software engineering, especially for the work that you're doing at CloudKitchens? And what are your thoughts on how it could change as it just becomes better?

1:32:59Charles-Axel Dein:Yeah, I mean, so much has been written. Every week there's a new article, new analysis, new prediction. We did our own study. We found it was moderately useful. It is useful. So the thing I can add is for myself, I think it's particularly useful when you have a well-defined problem that you're trying to solve, navigating the code as a coach and as a trainer. As a manager, I use it a lot to review my document. As a matter of principle, I never copy-paste text prose that was written by an AI. I want to make sure I stay in control of what I write. Because, yeah, writing is thinking, right? Like, it's the same process.

1:33:42Charles-Axel Dein:So I don't want to outsource my thinking to an AI. Like, I would be really, really sad if that was the case. And when you're saying with engineering, you found moderately positive impact, Is this to do with kind of using autocomplete, using it to generate some parts of the code? Did you go further in terms of like, let's say, code reviews or trying to automate some workflows? Yeah, so we use it for all of those use cases. I mean, obviously, I don't write as much code as I would like to and as I used to. Well, but now you could. Actually, it's a great point. I did a lot more coding lately because with agent in particular, it enables you to multitask.

1:34:21Charles-Axel Dein:You have 10 minutes before a meeting, you issue a specific prompt, you have your meeting, you come back, and then you review the code. Yeah, I would say coding-wise, refactor is really good at that. Refactor, well-specified APIs and interfaces. Anything that is more complex, I feel like it's still failing. I'm sure it's going to get better, but I think the design component will always be a human being thing. The other thing, every single time we have one of those revolutions, and AI might be the biggest yet, don't get me wrong, but every time we have those revolutions, the press and everyone speaks about replacing engineers.

1:35:03Charles-Axel Dein:We're still there. You remember when machine learning really picked up, people were saying, yeah, so we won't need software engineers anymore. The truth is, actually, it's a data scientist that was really a job role that was under a lot of challenges because of machine learning and ready-made models. But yeah, I don't think we're going to replace it. For example, your team is still hiring, right? Yeah, we're still hiring. I think it's going to enable engineers to do more, to focus on more interesting tasks. We talked about migration. I think migration is potentially a great use case. We hate doing it.

1:35:41Yeah. And it needs to be done.

1:35:43Charles-Axel Dein:Yeah. So migration, great use case. So for instance, at Cloud Kitchens, we had a migration and the teams that was owning the internal platform put together a prompt that you could copy paste to do the migration and still review the card. Because there's always things that go wrong. What about security or an increase of, would we have increase of attack surfaces by using AI? Yes. I think that's the most obvious thing as well. Security engineer is going to be a great job role. There's going to be so many things. Lately, the main source of, a big source of vulnerability was like the supply chain attacks, right?

1:36:22Charles-Axel Dein:When you take control of one of the dependencies of a project by typo squatting or by just taking control of the repo, for instance. And then this way you can get remote code execution on remote virtually thousands of repository. Well, agents are that at scale. So it's going to be an incredible time for security engineer. So far, so good. there haven't been so many problems, but we're going to have so many. It's absolutely obvious. We need to prepare for that. What about engineering management? I mean, you mentioned how you use it to help with some of your documents, but do you think it changes the engineering manager's role?

1:36:57It helps, it makes some things easier? Does it make some things harder?

1:37:00Charles-Axel Dein:Yeah, I use it as a coach. So for instance, where I would have potentially pinged my manager for advice and maybe more basic stuff. Now I get a first review from an AI. it's hit or miss. Like sometimes it's extremely valid feedback. Sometimes it's totally useless. And it's a good first pass, I think. But I would say never copy paste text from AI. I think we're going to lose skills if we do that. Well, one thing I think more people are going to use that I talk with engineers. One thing, you know, engineers, a few things engineers don't like to do that much. Meetings, migrations, performance reviews.

1:37:39Charles-Axel Dein:Yes, performance reviews. So if they listen to this, I have reached out to some of my skip level engineers and I told them, hey, don't use AI to generate the feedback for your manager. Don't only use AI at least. Yeah, he's going to find it funny, but there's even one engineer who took some of my Slack messages and then took our competencies and asked AI to generate my feedback review. It's unavoidable of happening. But as you say, there is a point of writing as thinking, and there is a reason companies try to force a little time to reflect on, you know, your relationship with this person or what you think of yourself, etc.

1:38:22Charles-Axel Dein:There's also one thing, which is nobody wants to read AI-generated text. The moment I tell you that this text was generated by AI, you lose interest immediately. So I would much rather, what I tell people who might be tempted to do that, is give me the prompt. If you don't have time to write good English and stuff, which, by the way, is not that important, tell me what you would have asked the AI to generate. And that's what I'm going to read because this is where the data. There is this irony, right? Like it starts with a very small prompt. It's bloated into a massive text by an AI. And then somebody else will use an AI to summarize the content.

1:38:59Charles-Axel Dein:Why don't we just exchange a prompt directly? We're not going to lose any information that way. I feel it's an interesting, very interesting contradiction with AI, specifically with text generation. I wonder if we're going to see more. We might see some other things in software engine slowly repeat as we learn, right? Because I feel this is, yeah, you just want the original information, the prompt. Yes. We'll see if software might or might not be the thing because people don't really care about the software. You know, users don't care like what exactly code runs under the hood. So maybe it'll be different.

1:39:32until there is a problem, right? Until there's a problem.

1:39:35Charles-Axel Dein:That's why, the other thing is, I think you talk a lot about that on your blog as well, the importance of code review. The first time I used AI to generate code, I did not realize that it would be my code. So I did not really, really proofread it that carefully. And then I put the PR up and I got a lot of feedback and I was like, yeah, I agree with all the feedback. This is really bad. And what I did after that is make sure that you read the code that was generated so that it becomes yours. Because at the end of the day, if something breaks, you're going to have to do this anyway. And that's probably the most interesting aspect, which is reading code that you did not write and making it your own is much higher cognitive load than writing it in the first place.

1:40:20Charles-Axel Dein:And as a consequence, that's why you are still going to need engineers. Because at the end of the day, the moral agent, the person making the decision and putting their stamp is still a human being. I wonder if an interesting second order effect of AI, it's very good at generating a lot of text from a prompt. It can generate pages of text. It's also very good at generating a lot of code based on a prompt. You give it a prompt, it generates the code. As a result, it's pretty obvious we're going to see a lot more code generated. If you think about the professional engineering teams that you worked on at Uber, at Cloud Kitchens, at the other startups, we spend a lot of time in engineers to not write verbose code, like on code reviews we'll push it down we don't want to copy for example functionality that exists elsewhere we would rather use the abstractions what do you think the impact might be if we just continue the thought experience that we will have a lot more code generated, a lot more wordy code than need be, where will this lead?

1:41:15Charles-Axel Dein:I think the startup might fail or slow down right? Yeah because if they're only doing vibe coding with very little code review it's going to become unmaintainable even for an AI. So I would say you have to stay in control. You have to have strong code review practices in place. Yeah, one thing I often see when I ask AI to generate, it's trained on Stack Overflow content. It's trained on not necessarily the highest, best quality code. It's whatever's on GitHub or whatever's open source. I mean, some open source is high quality. High quality, yeah. But a lot of stuff that's open there and can be used for training is not.

1:41:50Charles-Axel Dein:Yeah, so for instance, you will see that it's reinventing a feature that you know already exist in the standard library or in this or that library. Very often a failure mode, I think. So I think this is one of the risks is you end up with so much code that you actually default on the code being so overwhelming that when something bad happens, you cannot debug it yourself. You have to involve other APIs that will fail. Yeah, we might relearn some lessons that we've already learned as an industry a few decades ago. And as closing, what is a programming language that you like most? And which one would you like to learn?

1:42:30Charles-Axel Dein:Yeah, difficult question because I love programming languages. I love this quote from Jan Stubut, the creator of C++, right? There's only two types of programming languages, those we complain about, and those nobody uses. So in the first section, I would say Python. why? I mean, it's so effective a language. So versatile. A really great, by the way, interview choice. If you have a coding interview, it's a really smart move because it's so effective, right? You can get so much done. It's a professional language nowadays. Might I remember, yeah, remind people that most of Uber was built on top of Python in the beginning.

1:43:10Charles-Axel Dein:So it's a really, really powerful language. TypeScript is pretty good also in that destination. I usually prefer dynamic languages because they leave me much more in control. I love statically typed languages as well. And then learn. I learned Clojure. So that's the second, I guess, category. I never got the opportunity to use it that much. I think it's a beautiful language. Lisp dialects are really cool. Love functional programming, but sadly I haven't had the opportunity. Rust is pretty good too. I learned it as well in that section. Yeah. No, great, Charles. This was a really fun conversation.

1:43:47I'm glad we had it.

1:43:48Charles-Axel Dein:Yeah, same. It was a long time overdue. I hope you enjoyed this candid conversation with Charles. One of the things I like most about him is how he keeps trying out new stuff and is an absolute productivity geek. I hope this came across the conversation as well. For more tips on how to lead projects as a software engineer, check out the deep dives into Pragmatic Engineer that I rolled in the past, where plenty of lessons come from working with Charles. They're linked in the show notes below. If you've enjoyed this podcast, please do subscribe to our Favourites podcast platform and on YouTube. A special thank you if you also leave a rating for the show.

1:44:18Thanks, and see you in the next one.