I gave Claude Code our entire codebase. Our customers noticed. | Al Chen (Galileo)

0:00Al Chen:The minute I realized I couldn't really do my job was when I was trying to reference our public documentation and trying to provide an answer. It just still wasn't coming up with an answer that my customers were looking for. They don't want the docs answer. They want the step-by-step answer of how all these services cascade together. What I realized is that I can actually pull all of these repos into my VS Code and I can now use Cloud Code to ask our entire code base questions. Did you just say, Claude Code, write me a script that pulls all these? Yeah, yeah. I'm opening up the script right now.

0:32Al Chen:It's like, what, 16 lines? Didn't have to write this. I just said, help me figure out a way to pull the latest main branches into my local repos. The reality is we can now all live in a little bit more chaos because the AI navigates all that information for us across systems, right? So you can be in your code querying Confluence. We'll find the information. You have to be less precious about where and how you store the information. Throw it into Confluence, throw it into Notion, throw it into Slack, whatever. That ends up being context you can provide to Claude when you are trying to ask it a question about a customer or about your code base.

1:06Let's give Claude code a little spiff every time it answers a question correctly. You got to split your quota with Claude code.

1:13Al Chen:Yeah, it gives you better answers the more bucks you give it or something. Coin-operated Claude, that's going to be my new skill.

1:22Welcome back to How I AI. I'm Clara Vo, product leader and AI obsessive here on a mission to help you build better with these new tools. Today, we have an episode all about harnessing your code to make your customers experience way better. Al Chen, who's on the field engineering team at Galileo, shows us how he uses their 15 repositories and cloud code to answer every nuanced customer question that comes across his desk and use that to make the entire customer base and his entire team a lot happier. Let's get to it. This episode is brought to you by Orcus, the company behind Open Source Conductor, which powers complex workflows and process orchestration for modern enterprise apps and agentic workflows.

2:04Legacy business process automation tools are breaking down. Siloed low-code platforms, outdated process management systems, and disconnected API management tools weren't built for today's AI-powered world. Orcus changes that. With Orcus Conductor, you get a modern orchestration layer that scales with high reliability and brings humans, AI, and systems together in real time. It's not just about tasks. It's about orchestrating everything. APIs, microservices, data pipelines, human-in-the-loop actions, and even autonomous agents. So build, test, and debug complex workflows with ease, all while maintaining enterprise-grade security, compliance, and observability.

2:44Orcus. Orchestrate the future of work. Learn more and start building at orcas.io. Al, thanks for joining How I AI. I am really excited about this episode because we've seen a lot about using your code as documentation. You know, we've heard engineers saying, you know, docs and code should be in the repo. Product managers saying code can now be my documentation for internally facing assets or as I help draft PRDs. But you're going to show us how you can use code as an asset to create customer-facing things and solve customer-facing problems. So tell me, what problem were you facing when you decided, I'm just going to clone the repo and fire up CloudCode and solve some of these problems myself?

3:33Sure.

3:34Al Chen:So working at Galileo on the field engineering team, I'm on the front lines in terms of working with our enterprise customers who are typically developers themselves and asking very in-depth technical questions. and the minute I realized I couldn't really do my job was when I was trying to reference our public documentation and trying to provide an answer to my customers even using Cloud Code or ChattapT or whatever and trying to take all these different help docs and trying to come up with the answer. It just still wasn't coming up with the answer that my customers were looking for. and I just background I'm not an engineer I've never held an engineering role but I think I know enough to just be dangerous and I realized that our product LAO's product word observability tool for AI applications if you look at this image here I'm showing an architecture diagram high level of all the different services that make up our platform this is all like back-end images that customers have to deploy onto their Kubernetes cluster.

4:44Al Chen:And I realized that all these different services like UI, API, Auth, Z, Comet, they are all individual repos within our Galileo repo. We're not a monorepo. We have multiple different repos. And so what I realized is that I can actually pull all of these repos into my VS code. Initially, it was just more for me to like, I want to understand how our code works, and how our code is structured. But then when I threw it all into VS Code, which looks like here, you notice along the left-hand side, I'm on VS Code now. And most of these directories correspond to one of those services within our architecture.

5:25Al Chen:So one repo corresponds to one service. And by having all of these repos in my VS Code, I can now use Cloud Code to ask our entire code base questions that are not answerable by our public documentation. And so sometimes I'll get really questions about, well, how does this feature actually work? And so I'll ask Cloud Code, look into the API repo, look into the AuthZ repo, and help me come up with an answer. If you can't find the answer, reference other repos within my directory, my root directory, and help me figure out the answer. And so that's the key on lock was when I figured out I could get way more in-depth, way more technical, and at the same time, myself, I can learn how our code base works.

6:15Al Chen:And how this has really helped me is I don't have to constantly ping our team engineering channel with, hey, what's the answer to this question? The customer just pinged me about this. And you can imagine engineers being really frustrated when I'm trying to post these questions, and then the customer asks me a follow-up, and then I'm posting that follow-up in the Slack thread. So I'm sure many of you who are working on the front lines of customers understand how that feels. But I've basically reduced all of that almost down to zero by pulling all these repos into my local VS code. I'm really empathetic to this problem because I used to work at LaunchDarkly, leading product in engineering, very technical product.

6:53We, too, had an architecture diagram that looked very similar to that. And, again, as more of a, you know, people think that CPOs, chief product officers, or CTOs are internal facing. saying, no, no, no, we're salespeople. We're always salespeople. You trot us out and you put us in front of the customer or you put us in front of the prospect to answer the technical questions. And we had a diagram like that. And I would constantly get these very detailed questions that required very detailed answers. Like, how does your caching work? And, you know, when you have seven layers of caching in your app, you can give the high level docs answer.

7:29But when you're sitting with, you know, an architect in the room or somebody highly technical, they don't want the docs answer. They want the step by step answer of how all these services cascade together to build a resilient caching mechanism, for example. And I just think how powerful is it to be in a meeting or in an email back and forth, and not just sort of give this high level, but be able to query the current code base and really understand at a detailed level how it works. And I think current is very important because, you know, and I know this is always evolving over time. So even if you got the answer right, you know, a month ago, maybe your team shipped an update or maybe, you know, that method is actually out of date or the docs are a little bit out of date.

8:17And so I do think because the code base is, you know, at least your main branch is always the source of truth, it becomes a really reliable context set for you to answer questions about how the product operates.

8:32Al Chen:Yeah. And to quickly address your comment about how your code is obviously always evolving, I mean, we're pushing out multiple features per day, multiple releases. And one thing I've done, wrote this with cloud code, is I have this script at my root directory that says I just do something called pull all. and I'm not sure if this is how other people do it but it just pulls the main branch into my repo for all the repos in my root directory. So if I do this every day I kind of get the latest code across all these directories on the left hand side of my VS code. So the alternative which I was doing before for a few weeks and I realized this is just asinine that I'm doing this is doing git pull orange and main on every single directory and it was just not scalable because there's now like 15 different repos I have to pull the latest from.

9:23Al Chen:So that's kind of how I solved the code base is always evolving problem to make sure that I'm always getting the most up-to-date information for my customers. And I have to ask, did you just say cloud code, write me a script that get pulls all these? Oh yeah, yeah, yeah. I have no idea. I'm opening up the script right now in my VS code and it's like, what, 16 lines? I didn't have to write this. I just said, help me figure out a way to pull the latest main branches into my local repos. And I just did it in like one shot. Yeah, the other thing I want to call out for folks as I'm looking at your screen is I don't think people use this trick enough, which is in VS Code, in Cursor, in whatever your IDE is.

10:07Loading a project at the multi-repo level as opposed to at the individual repo level, if you're trying to answer questions across the product, is really important. So, you know, there's some like context bloat stuff that can come into sort of querying across all those repos and all those files. But it would be very painful if you had to go into each of these repos one by one and like query and then go into the other one and query. And so I like this idea of opening them all jointly in your IDE so that when you're querying it with Cloud Code or you're querying it with something like Cursor, it can go across, traverse across repos and really give you highly contextualized answers.

10:53Al Chen:Yeah, our code basis happens to be in multiple repos, but I just pulled them all into this giant Galileo directory here. And so everything is at the same parent. but if you're in a monorepo could be actually I don't know how this would work with a monorepo because I've never done it with monorepo with CloudCode but at least for us this is how it works well I have many monorepos and yeah you just open it at the right I would say my advice to folks is open CloudCode or open you know your IDE at the right level and sometimes it's narrow and sometimes you need to go up a directory and I think really thinking about that.

11:34And you can even do that contextualized to the problem you're trying to solve, right? And doing that, I think, is really helpful. Can you show us, just using Cloud Code, what kind of question you could answer with this code context?

11:47Al Chen:I will give you an example of, I guess, I'm a big believer in using shortcuts, too. So I use a bunch of custom Cloud Code custom commands to help me do stuff. One thing I do a lot is helping my customers deploy Galileo into their VPC. I have a custom command called DPL. It actually references our first thing it does is it looks at our confluence because we have a whole bunch of confluence pages about how to deploy into Kubernetes using our different images and stuff like that. So I'll say DPL my customer cannot use CRDs and they are using Google Secrets Manager and want to deploy the wizard image.

12:42Al Chen:Give me a step-by-step process on how to do it. This is actually not a super representative query because they're way more detailed than this and I provide a lot more context. But I want Cloud Code to focus on looking at Confluence first, because I know that we have a whole bunch of deployment stuff there. And then from there, if they can't find the answer, it will go off into all the different repos along the left-hand side of my Cloud Code VS Code to find the answer. So right now it's just using the LASC and MCP to pull information from Confluence, and then marries that with our code base to answer a very in-depth deployment question.

13:24Al Chen:The one thing, I'm not sure if we should talk about this now, but I started doing this in Confluence where we have a, we call it a customer quirks page. These are all kinds of, all of our enterprise customers, you typically have air-gapped environments. So they have all these security measures and we have to abide by them when we deploy the product into their environment. And so I literally have a page that looks exactly like this where I have the customer's name at the top level and then a bunch of bullet points with like, you know, here are some things about how they store their secrets. Here's how they do namespaces.

14:00Al Chen:You know, here's how they handle side cars and service to service encryption. Things I know nothing about. But as I'm meeting with my customers, I'm putting this all into this one Confluence page, this ever-growing Confluence page. And then this is actually one of the core pages that goes into this DPL custom command, which is look at the customer quirks page. If I'm mentioning a customer that's on that page, look at all their quirks. And then in the response from cloud, it's highly customized, highly tailored to their environment. Because I've seen from working with our DevOps team that we can provide a generic answer about Kubernetes or about ClickHouse or about whatever.

14:44Al Chen:for the customer, but it's like something you can just find online by Googling or using AI. But when it's tailored to specific security requirements and deployment requirements, it's way more effective and just gives the customer more trust that we know what we're doing essentially. What I love about what you showed here, which is, you know, kind of combining the repository with the Confluence MCP and then both like team generated general documentation, as well as you generated like micro documentation at the customer level is I've heard so often in my 20 years in enterprise SaaS, like, what is the source of truth for this information?

15:24Like, I'm sure you've heard this too. Like, what's the source of truth for how XYZ works? Or what's the source of truth for this customer? And people have spent so much time like, you know, pruning these confluence gardens and organizing their Slack channels and trying to get people to, you know get naming conventions right and like the reality is we can now all live in a little bit more chaos because the ai navigates all that information for us across systems right so you can be in your code querying confluence it will find you can kind of point it in the right direction it will find the information you have to be less precious about where and how you store the information bullet point list of quirks you know like really official docs whatever it doesn't matter because ai is just so much um more effective at traversing all that information and pulling it in and making it actionable for you and i don't think that's anything like any human was really proud that they were good at they're like i'm really good at finding like the right confluence doc uh that was never never the value add yeah yeah i mean i think um even if it's as simple as

16:29Al Chen:hey you you came up come across a really great answer in slack like in a really engaging slack thread, throw that into a Confluence page, or save that Slack thread, because I also use the Slack MCP to be able to summarize threads. So if you have just some random, this ongoing stream of consciousness of documents you want to have Cloud Code scan, I would just say throw it into Confluence, throw it into Notion, throw it into Slack, whatever. And then that ends up being context you can provide to Cloud when you are trying to ask it a question about a customer or about your code base. Well, and the other thing, and this is maybe going back to how I introduced this episode, which is people use AI so much to compete on the field of the product and engineering velocity.

17:14And what I mean by that is like, we're all using Cloud Code to ship more product. We're all using AI and codecs to build, you know, better user experiences or more resilient backends or any of that stuff. But there's also a completely different competitive field, which is how you show up in your relationships with your customers. And I think, you know, what you're showing is you can actually use AI to invest and compete on customer experience. And, you know, my hypothesis is when you're very complex enterprise customers, have you show up and you don't just say like, here are our general docs to deploy this.

17:52And instead you say, I heard you. I understand what your needs are. And here are your custom docs on how you specifically need to deploy this. And I've already pre-thought about all the problems you've already told me about. you know, just looking like in a competitive sense, that's got to come across as a much more enjoyable customer experience on the receiving end and allows you to position yourself not just as great product, but as a great team that's going to service your customers well.

18:22Al Chen:Yeah, I hope so. I mean, I think our customers, I think my customers are hopefully enjoying the answers I provide and the in-depthness that I provide. I think I've thought about taking this to the extreme, which is I have certain customers who are very in the weeds. They want to know things right at this very second. And I'm literally taking their question and then just saying, my customer then asked me this because they can't see your code, but me, Al, I can see the code. Help me get the answer. And so if I take that to the logical conclusion, it's like, why can't we just share our repos with the customer?

19:00Al Chen:Because then they can just start querying our repos directly to get the answers they need instead of me as kind of like the quote unquote middleman. And, you know, the issue is that like our code is proprietary and all that kind of stuff. But I have seen, there's actually a case study from Langchain. And since a lot of Langchain's repos are, you know, it's open source, like their support agent bot actually does a lot of things I do, but it is able to query, you know, all the public open source repos. and any of you out there who are trying to use LangChain or LangGraph, you can just pull all those repos down to your local machine and then ask questions, of course, using CloudCode or Cursor or whatever but I've gone through that thought experiment of I'm still kind of a bottleneck in terms of answering my customers' questions because I kind of hold the keys to our code but if they somehow had a sanitized version of it, then maybe they could just self-answer their questions too, because they're also all using VS Code and Cursor and Clawed too, but they just don't happen to have our proprietary code base.

20:06Yeah. I was going to ask you, are you worried that the owl bot is coming and you're cut out of it? And I'm just curious how you think about then when, again, the highest order of you is not to be a pass-through, and I don't think you think of yourself self is that. And so where does the human in these relationships powered by AI, you know, add, add the value?

20:30Al Chen:Well, to, I don't just blindly copy and paste the answers I get from cloud code to my customers in Slack or email or wherever. I still try to proofread everything. And I actually do like try to make it sound more human. And you can then say the argument, Oh, why don't you use cloud code to make your answer sound more human. And I think all of us know when we get an answer that's from the AI.

20:55Al Chen:You'll see a bullet point saying, in summary, here are the things you need to do to make sure your click house works. Removing things like that that just make it seem like it's from a bot just makes it seem more human. This is going behind the scenes of how we work, but we've been dinged sometimes where the customer will say, can you just not give me an error response and just give me like a human proofread of it and tell me how it applies to me? Because typically the response is way too verbose, it has way too much information and the customer just wants to know, give me like the bottom line up front, what do I need to know to like deploy this image onto my cluster?

21:37Al Chen:And so that's where the human, I still see myself as a human providing value and calling that down to what they actually need. And I would say, even for some of the more in-depth technical questions, I still try to get an engineer's perspective on it to make sure cloud code is not hallucinating or not saying anything out of the ordinary. In my system prompt, in my cloud code, I say things like, don't make anything up, always cite your sources, pull me to the line of code where you're getting this information from. But even with that, if I don't fully understand how this function works or whatever, I'm still paying the engineering channel to say, hey, this is what Cloud Code told me.

22:19Al Chen:Does that jive with what you're thinking? And there are times when I'm wrong or Cloud Code is wrong because our engineers have been thinking about refactoring into this new model, which is not captured in our code base anywhere. It's just captured in a meeting node somewhere or just hallway conversations. And so those are the things that I'll never be able to query, let's say, in Claude. Yeah, I would say the other thing that, you know, where I see humans adding value, and I say this all the time, which is like Riz is the only moat, which is at some point, you know, people just want to have a face and a trusted personal relationship, you know, with the folks.

23:01And this is like my enterprise showing, but like with the folks that are selling them software, you want to know that you have somebody to call. You want to know that you have somebody that can gather the right folks around your team and your deployment. and you know you want to enjoy working with that person and I will just say I get a lot of um it is very fun for me to build with these tools with AI tools but I wouldn't say my AI colleagues are like the most fun to hang out with which is like I'm not like always looking forward to like my my Claude code session like I'm gonna really chit chat with with good old Claude um and I do think you still have that relationship with you know your human partners your human colleagues all that sort of stuff.

23:43And so I think there is a piece of that that's just not going to get cut out. And honestly, I gave this talk, I don't know, two years ago, I said PM is dead. And people are like, well, what else should we do? And I was like, get into sales. Like that's not going away. Customer facing stuff is not going away. So for anybody that wants to survive, you know, the incoming apocalypse, I do think customer facing roles and spending more time customer facing is a really important part of everyone's job.

24:10Al Chen:Absolutely. If you're working enterprise sales like that is all people handshakes um lunches dinners so that will never be replaced i think by ai anytime soon well you know and there might be a generational shift though here i think as as we sell as we sell we'll see you know i used to say my my joke in enterprise sales and the biggest um the the biggest headwind to enterprise sales was i was starting to sell to millennials who like wanted you to text when you showed up at their door. They didn't want you to knock on their door like there was this abortion. We'll see. We'll see how enterprise sales changes generationally.

24:49This episode is brought to you by Times, the intelligent workflow platform powering the world's most important work. Business moves faster than the systems meant to support it. Teams are stuck with repetitive tasks, scattered tools, and hard to reach data. AI has huge promise, but struggles when everything underneath is fragmented. Tynes fixes that. It unifies your tools, data, and processes in one secure, flexible platform, blending egetic AI, automation, and human-led intervention. Teams get their time back, workflows run smarter, and AI actually delivers real value. Customers now automate over 1.5 billion actions every week.