The CTO must now think like the CFO to survive | Sancus Ventures’ Lake Dai

25:17What if, for example, you cannot get hold of the supply? What will happen to the main business? So now engineering leaders have to think from the lens as a CFO. You have to answer to those questions. That's why the compute cost is another impact. And then the third one, I'm sure a lot of your guests have already talked about it, is uplifting your engineering organizations. That means that as we continue to build more AI-native infrastructures, we're switching from maybe in the past or more of a user interface or applications or infras. But now we'll switch more of a, let's say, called agentic AI-friendly or agent infrastructures.

25:59Yeah. like orchestration, API layers. So you have to think about those new architectures. You have to think about skill sets. You can think about, you know, a lot of organizations talk about with cursor and vibe coding or co-pilots, you do not need as many junior developers anymore. So what about junior developer you already hired? What about the senior developer now needs to upscale to understand the new technology and new infrastructure? So what do you do? And maybe even you are managing a team of human and agents simultaneously. Yeah. I think you end up in this world where you have these hybridized teams where part of the team is a person and part of the team is a compute layer.

26:42I know. And then imagine that would be more complex when you have vendors also have that infrastructure. Of course. And you're interacting with other versions of that ecosystem within their own org. And you're kind of blind to it. You have to trust that what goes in is going to go in and get sorted correctly with how it gets used. We actually, we're talking about matrix now, right? So we talked about in the past, we have collaborations, internal and external, that's it. So now internal and external, you talk about human to human, human to AI, AI to AI, right? So that's the three relationship I was talking about in the future.

27:15We see more of that evolving because human to AI relationship, in the past we thought about human or training AI. But now we're talking about AI co-pilot. Yeah. And as we have more and more, for example, students or junior developers need to do training, maybe we need a simulated environment to train the junior developer to be senior developers. Oh, that's fascinating. We're creating this safe space for them to figure out how to work with these tools. Almost like a flight simulator for a pilot before they start flying. Did you get enough hours in the simulator first before you were flying a real plane with real people on it?

27:51The idea of applying the same thing to engineering is fascinating because the thing about AI is it's so powerful, which it's a force multiplier, you know, for both good and bad. And you have to have really good process and really good alignment within your org or it can really go awry, right? Right, right. I actually think AI simulated training environment is one of the key to obscure your organization. So that's the human to AI relationship, right? Let's talk about AI to AI relationship. That's also fascinating. Yeah, it is. So we're just talking about my agent to work with your agent, but my agent can also have sub-agents, right?

28:26So this is the whole, remember you play the game that you have 10 people line up, you say something, sentence, then you go to the end of a line. Play a game of telephone. Simply change, right? Yeah. So, okay, so now imagine how we actually measure the outcome of the model is that we provide training data set, we measure, we do the kind of evaluation of the outcome. That's okay. But then they start producing sub-agents and that demand and their outcome evaluation will continue to pass. How do you know what continue align with initial intention? Exactly. Right. Right. So, and then you have vendors, agents, and then you have, you know, we're talking about collaboration, but there's also, for example, if you have AI criminals attacking your system, you need AI police to protect you, right?

29:21Right. So there's also the opposite positions. So it's quite interesting. Like using AI defensively within your organization. Yeah, we've seen, I know you mentioned that we also run a VC firm. We invest in early stage AI infra and native apps. So we're seeing a lot of innovation now, even in the cybersecurity space, that using reinforced learning. Then you have a red team, you have blue team training, use reinforced learning training to have much, much better solutions than the current human, I would say human dominant. Right. They can work together in a way where they better understand each other.

30:01Right, right. So this is a getting like similar to like GAN. GAN is basically you have generative models. You have discriminative models. One side is continue creating new solutions. The other side is continue to say, yeah, like say this is right, this is not wrong. And after running a reinforcement learning for many runs, then both models would become so much better. Is that sparring partner? You make it spar with itself. Exactly. So then what's happening in the cybersecurity space is that you do want to have a sparring partner here. Then you can be so much better. Something that you mentioned about engineers using agents and those agents have sub-agents.

30:38This is a world I was recently in. So I recently did a hackathon with Block Open Source and they have Goose, you know, they're a tool for Vibe Coding. And we had a Vibe Coding hackathon and I was one of the participants and I had a partner. It's a two-person hackathon. We were building entirely with AI and AI agents. And the point was they use sub-agents as part of that process. And it was so hard in that hackathon environment for me and my partner to get our agentic teams aligned with each other on building the same thing within a really limited time frame. It was actually really hard. And at the end, you know, they asked me, like, you know, what would you have done differently now that you did that for it was a four hour hackathon?

31:15Like, what would you have done different? And I said, you know, I would have used all four hours to plan. I would have just used all four hours to talk about what we wanted to do because that ultimately was the missing piece of building that intent of what we were trying to accomplish before picking up the tools, before assembling the teams. Because like you said, once you start going down the layers, you start to lose context. You're a little blind to what's happening. So it's really important to have that top-down alignment. Yeah, so this is actually quite interesting because it's related to my early work at search.

31:48Because in search engine, remember, you know, early days, either website or mobile, you're putting the search keywords. The hardest part of the search engine, unlike what people think is crawling out the information, is actually what we call the search intent. It means that people have explicit intent or implicit intent. and to do the intent mining is so difficult because quite often you say A, you meant B. Actually, you need C and then we give you D. Exactly. I love that. I love that. That's a really great point because especially in a world like search, you're kind of like guessing ahead of the user.

32:25And in a way, that's where you start to build an experience that's delightful and you get that widespread adoption. And that's why things like search are fundamental blocks of our world today, right? And I want to also dig in about your discussion on operating metrics and about understanding that cost and being able to communicate between your financial leaders and your engineering leaders and doing forecasting even on how you're using these tools. I think that's really fascinating and a problem that a lot of engineering leaders are still grappling with and figuring out. We talk about that a lot on Dev Interrupted.

32:57Dev Interrupted as well with Linear B. Our whole mainframe is about understanding those operating metrics and really boiling it down so that you get the impact of what it is that you're getting across the finish line. Are you shipping more code? But is that code better? Is it safer? When it hits production, are things failing? Actually having that full picture is a really important part of the operating costs understanding because if you have a lot of tools and then they get introduced and then maybe a few months later you have a really bad outage or something happened and it's because you've got this proliferation of bad that maybe you didn't see you know so i think it's a really interesting problem space and i'm curious like what advice do you give to engineering leaders to be really confident going into that like financial conversation around the compute that their org is using and justifying it this is actually a great question because we are actually literally talk about mindset shifting right so um i think i at my early talk and audience is asking what can you give me a basically recommendation what are the operating metrics that we're looking at so interestingly it's really case by case let's say if you're e-commerce company and then you're apply AI in your internal process.

34:14And as a CEO of an e-commerce company, I will say, well, let's say, Andrew, you adopt AI. So let me know. So using this specifically, whatever technology you're using, how much have you lived conversion? Because e-commerce conversion works, right? So let's say we need to also buy a lot of, purchase a lot of ads for user acquisitions. how much more efficiency has increased that? So those are very specific questions related to user acquisitions and also customer conversions into, you know, or upsell percentage. But not being blind to all of those little multipliers that are across your organization where you're tweaking and making it more efficient because that all boils down to that metric that you're trying to track.

35:01Yeah. So in another way is that, well, if I'm a CEO, think that way. And then I will ask my engineer leaders that I don't care about which model you're using. Right. nor do I care how fast you code. Or how many lines you made. So what's the impact? Exactly. Yeah. What did it do? Did it ship something? Did it make something safer? Did it save a lot of time in a really like eye-opening way? Like, oh, I didn't even know we could do this this way before. Those are like the real force multipliers where you find that like hidden unlock. And a lot of times when I sit down and I talk with an engineering leader, like where was the biggest unlock in your org as people started experimenting with AI?

35:38A lot of times it comes from the non-engineers and the folks who have never picked up those tools before. Someone in finance figuring out how to pick it up at a workflow and build something out to make a cost calculation way more efficient or way more accurate. And when we look at those on an aggregate level, that really compounds and adds up. And that's the impact of the non-technologist in the conversation, right? Yeah. Which is really profound and interesting because I think we're entering a world where everyone's going to become this native technologist. And we're all getting closer to expressing our intent in words and then getting machine computation output, getting algorithmic things.

36:16We talked with guests here about like personalized software, idiosyncratic things that you spin up and throw away, stuff that you would never have built in a world before AI. But now it's actually easier to build it with AI than it is to do anything else. So as an educator, how are you thinking about that? How are you equipping the future? Oh, there's so much to talk about. I think a couple of things you said is quite interesting is when everyone starts building, who is creating value? Who's capturing the value? And I think for education, now I'll put on my professor hat on. When I think about education, particularly for higher education, So we're really looking at providing students three things.

36:59Knowledge transfer. I would say 80 % or 90 % of the university are doing knowledge transfer. Yes. And then there is the branding of a university like Stanford, Yale, you know, Hover, Branding Association. And there's a whole network of friends you went to college with. Right. Will go to your wedding, go to all the things. Of course, invest in your company later on. Like all those friends that you make and you stick with. The biggest shift is that now the 70-80 % knowledge transfer, university is not the primary, may not be the primary source anymore. Because you can get it, you can just talk to ChangeGPT.

37:34Right. You can go to Coursera, there's online courses and university providing that as well. So knowledge is not hard to find. So what is the purpose for university? What kind of skill set we should transfer? Right. And I would say it's ask the right questions. Because even today, you have access to chat.jpg. I have chat.jpg. But how come our results is different? Do you know how to ask the right question? Yeah, exactly. You have to be in the moment. And you have to be plugged in. And you have to have taste and instinct about what's going on around you. And you're right, though, that it's like you can use those things as building blocks to get that base knowledge.

38:17but that if you're not curious and ask those deeper questions and dig into it more, that's what really makes the knowledge set in and you can build upon it, right? And I think that that's like an interesting scenario that you've laid out that, you know, the knowledge transfer, maybe university is not the vehicle for it anymore. I agree with you. I think like my background as an educator and working within the space has always been around empowering people through like career and technical education. and a big part of that is just being driven and being constantly curious to learn right and those folks really they set themselves above because if you're willing to learn and you don't give up you can pick up new skills and you can pivot in this world that we live in and it really opens a new door for like a new type of thinkers to have a lot of impact in the world we live in yeah so and i i know a lot of people talk about education everyone's like have a little more negative tone oh my god you know oh yeah they're using it the cheat no one's gonna write anymore yeah but on the other hand i'll give you examples so so when vibe coding just started like i i think within a week i already demoed lovable in my class yeah so this is a master of engineering at carnegie carnegie mellon university and i also have a group of high school students join us to audit the class.

39:35So when I demo that, and actually the guest speaker came in was Ted Nyman, who was the CTO of GitHub. And he shows that. And I can see the face on my students. Oh my God, this is something I used to do colder in months. Now you do it on seven minutes. Yeah. I can see the look on the face, but I can see also on the face of the high school. Oh, does that mean I can do X, Y, Z now? And it does. And that's what's so exciting is that it pulls them up at a way earlier age to participate in the world and build the world that we live in. Yeah. So, again, if we go back, if you have the beginner's mind, drop the package, and you go back to your high school mind, you're like, oh, my God, the world is completely open up to so many possibilities.

40:24Yeah. And that's a possibility. Yeah. Yeah. I'm curious, too, from your perspective, you have a very, very high level perspective. And, you know, you gave us some insights at the very top about these pervasive conversations within boardrooms about measuring and understanding AI impact. Curious from your perspective, what do you think is the biggest blind spot that engineering leaders have right now about it? AI governance. Yeah? Yeah. So AI governance, we talk about operating metrics, but I think there's another new metrics that will come in. that has become so critical to the leaders, not only just engineering leaders, is that the governance elements.

41:03Because we did talk about major trends. One of the trends I didn't talk about is the regulatory trends. There are so many AI policies, compliance and regulatory requirements. And it's very difficult for both sides. The policymakers try to catch up with the development of AI and also for AI innovators to try to catch up for the compliance requirements. Yeah, or build something that's not going to get regulated to death in a year or really they'll think ahead about how is policy going to impact what I'm building. Yeah, so I think that is very important to have both sides to have a fluid conversation because I think everyone has the best intention.

41:42Yeah. Right? So as an engineering leader, I think to understand where the trends is going and create a framework really allows you to continue to innovate. Meantime, protect your organization, also protect your consumers, customers, and your partners. That's super important. So I'm going to quote, so one of the really good talk coming from the founder of BotAuto, Xiaodi Hou, he said really perfectly, he said there are three structures, there are three frameworks. The first framework is the unit risk. Yeah. Means that when you're coding level, The second is system risk, which means that how you think about engineering development process.

42:26The third is the ethical level is how you organize and culturally think about this problem. Yeah. And you said in the unit level, it's really difficult to eliminate it because as long as a software, you have a box, right? Right. However, systematically, how you think about organizing and looking for the potential bias of the models, hallucinations, misinformation, disinformation, safety, or potentially, for example, IP infringement in the process and document it. And that's very important because it doesn't have to be perfect, but you can start with something. Then just like the, you know, the surgeons have a checklist before the surgeons, this is the one, two, three, four thing you want to check before you conduct the surgeon.

43:14I think that's really essential. Yeah. Building those policies, those playbooks, those internal frameworks for how you take something that is late in a latent space and you make it more deterministic and repeatable and measurable over time. And you continue to perfect the list. Yeah. But have a list is so much better to have no list. Absolutely. And then there's ethical elements being that company culture, what do you think of this? Yeah. Right? And then all of this, I think, needs to create their own. Each organization is different. You need to have your own, I'll say, handbook. And it's good to be documented.

43:53Yeah. to protect you and also to support you to continue to innovate. I'm curious too how you think about as a leader how do you tackle the problem of technical debt in a world of AI? How does it influence how you think about it within an engineering world? Well that's such a big question. Right? When you say technical debt there's a lot. Yeah. So I think you need to make a comparison that it's faster and easier to patch it up. or just completely use something new. Yeah. And we see that quite often in infrastructure because infrastructure takes time to build. The ones who are more successful and have a larger infra, more complex infra, it's actually sometimes, you know, harder to make this new, I would say adopt the new technologies.

44:43So we're going to see this whole wave, maybe like the tier two is catching up because they are, they have less baggage. They're like, forget about this. someone to start from everything from new. I'm not going to suggest the names, but I know even for us in the cloud space, we can see some tier two cloud providers not making really good progress. Yeah. I don't think they think they're tier two, by the way. I know what you mean. I think that's an interesting way to look at it too. And I'm curious about how, so we're all in this world where you have this new compute layer that's taking up a huge portion of mindshare and physical, like actual tangible costs within an engineering org.

45:25And we're in a world where that access to compute is largely subsidized. It's at our fingertips. We're all still figuring out how to use it. I'm curious what your perspective is, especially from just really high level that you have about how AI and access to AI might evolve over time and how engineering leaders can get ahead of that. There are some that like try to be more insular, have their own models, spin up their own compute layer. That doesn't necessarily scale. You have others that lean entirely on third-party providers. They get all of it inbound, and that adds risk to their org. So how do you navigate that?

45:59Wow, that's really good questions. Each question can write a paper. We'll spin this up into a paper, and then we'll get it published. So I think there are multiple layers to answer this question. So first thing is that when we think about what's going to be the next generation changes, I think engineering leaders should pay really close attention. What are the fundamental movers? Yeah. Right. So I didn't mention a little bit is that, for example, the agentic infrastructure, that's something takes some time to build. Right. So if you're not paying attention to what's happening and it's harder to catch up.

46:38So you kind of I know everyone's really busy schedule. But in the meantime, you carve out some time to watch the most important technical trends that come in. They may not arrive in the same year, but all these trends are started with the research and then move to industry and being adopted very, very quickly at scale. So, organic infrastructure is one. And the other thing is we see, you know, we're moving from pre-training to post-training. We see a lot of inference now is moving to the edge side. And that was something that's very interesting. What does it mean? I moved to Edge side, right? We talked about Edge computing for a long time.

47:21We've sat down with Google DeepMind and dug into Gemma. You know, they're small, like, on-prem, small embedded models. And what does that even mean to pick those up and use those, especially combined in a world like robotics, too? Yeah, so the first question I would ask you, why would people think about Edge? Why Edge? Why can't it be everything just, you know, in the cloud? Oh, because of the cost? Because of the risk of, if everything, I guess, is in the cloud. Like, let's say that your entire compute layer relies on making those API calls into Cloud Code or into ChatGPT. Yeah. And then, you know, you could be like a frog in a pot of water that's slowly getting warmer and warmer as those prices go up and up.

48:01And then that fights against the whole operating metrics that you're trying to be super efficient on. So you end up kind of trading one thing for a new problem, kind of. It becomes difficult, I think, for leaders. Okay, so cost dependency, as you mentioned. There are also other elements. So for example, privacy reason. Yeah, privacy reason. So some of the information you do not want to put in. That's right. You want to put on the devices that give you privacy, and then sometimes the response time as well. Something you need to make really quick decisions. You want the models to be running. It needs to be at hand.

48:38Yeah. It needs to be right there. So that's another reason. Yeah. And I also say that the framework models in the future is not only for large language models and multimodality. And sometimes it may actually run on the devices would be more efficient. And how we look at all the information or connect each other, for example, let's say, for example, let's just take a traffic. You don't always have to call to see where the traffic is. And if you have all the cars in the same region, car communication could be more efficient. Right, right. You get these different communications. communication models.

49:12Yeah, it has nothing to do with the rest of the cars who are not in this. You don't got to deal with those. You don't need to query some big master list of all of it. You can just work with that localizing. So it's also about, from your opinion, localizing the knowledge and structuring that knowledge to where it's close at hand when it needs to be. Yeah. Yeah, that's really cool. And, you know, some of the last things I want to dig into here is how do you prepare new leaders to build something in AI or in engineering? If maybe it's their first time really entering into the scene, do you have strong advice for how engineering leaders can use the opportunities in our market and our industry today to really get ahead?

49:52Because there's a lot of transformative opportunities. Is there a good advice that you'd want to give to a new engineering leader, someone who's picking up a problem? It's almost too much too fast. Then you have to be like Matrix, then you do the slow move. Oh, yeah. Do the go down with the bullets, go on by. It feels that way to be an engineering leader right now with the way the problems are coming at you and how you dodge them. You should be slowed down, not going fast. Yeah. So first of all, I want to say something a little bit unconventional. And I can't wait for us not to talk about AI because we don't talk about internet anymore.

50:28We don't talk about mobile anymore. Nobody come out to say, I'm building an internet company. That's right. Or I'm building a mobile company. Right. No one says that. Right. Because there's only a technology that facilitates and supports you to creating something new. Yeah. So now we talk about we're creating an e-commerce company, we're going to this company, we solve this problem. I think hopefully I will see this hype of talking about AI. Decrease. It needs to cool off. It needs to just embed itself into our world as this new platform. We talk about the real problem. the worst solution we try to provide.

51:04And when it slows down and cools off and it's kind of like the Earth's crust and it cools off and then things can finally start growing on it. Yeah. And I seriously, I think you should just slow down. If I want to catch every AI news and all the names, I'll be going to ask. I try to do that every week right here on Dev Interrupted. They all know I'm trying to keep up with the news. My head is spinning some weeks. Oh, I'll tell you something really funny. So I did the research. I was like, okay, So this quarter was the biggest trends that being toppling in the media and gave me 10 most insightful papers to read.

51:40And I just have the best model to run it. And I came back and like three out of 10 papers were written by ChattGPT. Yeah. Yeah. I was going to say, I can imagine what it pulled from. Right. So you basically, you look at it as completely written by HHGPT with no new insights. Yeah. So this is interesting. As we retrieve the information from those models and the models pulling the information, the human are generating more content with the system of AI. You're going to that loop. It's a feedback loop. Yeah. Junk in, junk out eventually. Yeah. Yeah. So that's what I'm saying is like, want to catch up on everything is hard.

52:27Yeah. Yeah. And go slow. Go slow, people. If you're listening to this, AI is moving real fast and slow and steady is going to win this race. Think about real fundamentals because you have a few quiet moments, then you can think about what are the true fundamentals? What are the trends that continue to repeat itself? Exactly. We see very clear trends whenever there's a technology, major technology shift in terms of internet and mobile. Look at those histories, how it repeats itself. You can learn so much from the past. I learn so much. History is very cyclical. Even when the Macintosh hit the scene and then people were suddenly creating graphic arts and you have this revolt from traditional media artists.

53:10And then even before that, when you had computer-assisted design and CAD software, and it displaces the work of so many architects who were then freed up to do higher-level architectural work, right? And it's a really interesting reaction, but then also a harnessing of that power. And then in those trends, you will see something like very consistent, which whenever there's new technology, there's a lot of so much like a passion to create some applications. You're going to see the first wave are people try to, oh, I can use the technology to do ABCD. And but because they didn't have a really good infrastructure that support that.

53:46So a lot of them actually died off. And then everyone is like, okay, let's build the infrastructure for this new technology, right? And then you're going to see building tech, uh, infra, and then you see overview of infra. Yeah. It comes in stages. And then, but because it's overbuilding and then drive the infra cost down. Yeah. And all in a sudden the real boom started. So we actually seeing the AI right now, right? Because we see a little bit of, a little bit like applications very early, like two, when Chagipiti first came out. But at that time, we didn't really have a full AI-native infrastructure.

54:19And then the last three years, a lot of companies are building it and improving it. So now we see the cost of building AI apps in terms of tools, compute costs. Everything has actually become so much easier. And then we see more people building. So I hope to see the app store time for AI apps. You know, Lake, I think we're going to have to sit down with you again in the future. and take another temperature test on how AI is evolving. And maybe by the time we talk, it's cooled off a little bit. People aren't talking about it as much. We can talk about that next platform, that next level evolution.

54:55But I have enjoyed our conversation so much today. And you bring so much amazing insight from your perspective and your role as an educator and an advisor. So I really want to thank you for sitting down with us. And I want to ask if there's final things that you're working on right now, places where people can go to to learn more about Lake and the work that you do. Can I recommend my sub stack? Yeah, please. So my name is Lake Dai, D-A-I. So I have an A-I in my name. So I created this sub stack called Lake D-A-I Unbundled. A little long. That's not a really good name. Now I think about it. But what's important is that I continue to monitor the trends.

55:37And like I said, trend watching is really important for my investment and for my teaching. Right. So I write maybe once a month or once every other month, only when something important happens. Right. You write when it matters. I know. So I think that's, because I have too many people asking me, what do you think will happen next? And I figure this is actually a really good way to show people just read the article. Amazing. Well, we're going to plug that in to our show notes. Make sure folks can go check out your substack and follow the work that you do. And again, I really want to thank you for sitting down with us on Dev Interrupted.

56:10And to those listening, we'll see you next time. See you next time.

56:21Outro Music