Streamlining Cloud Infrastructure Deployments with Jake Cooper

20:27What about like a canary rollout? Yeah, so we can do canary rollout. So you just you can just do it on the command line if you just do railway up. So as an also kind of like a piece of a fill in, we have the obviously the canvas dashboard or anything else like that. But we also have a command line because people like to interact with services in various different ways, right? So the command line is really useful for a bunch of different things, including that. Okay. And then how do I, I think one of the challenges companies typically have with whatever sort of cloud reasons they're using is they'll sometimes have essentially provision too much or potentially too little, which will lead to problems.

21:00Like they're spending too much or maybe they provision too little and then they run into like challenges with like throughput or latency or something like that. How do you solve for that? I think that's a really important thing to solve for. I think it's also a thing that is a core differentiator for us versus other platforms in the sense that the aforementioned orchestration engine that we've built allows us to basically only bill for the what you're using perspective. And so in a production workload, that's pretty cool because assume that you have 2x standard deviation or 10x because you're on the front page of Hacker News or anything else like that.

21:31That's fine. We can go and handle that. We can go and scale up the instances. We can scale them down. We can go and do that for you. Right. The part where it becomes actually like, I think even a little bit more interesting is when you end up with pull request environments that can be served like serverless. Right. And I use the server, the word serverless in like quotes. I know we're not on like a video or we're not going to share the media or whatever, but in a way that basically allows you to send a request to it, the request will spin up the container, the request will be filled and then the container will be finished.

21:59Right. And so we can actually go in and kind of construct that parallel environment of yours with a copy and write database volume. pretty soon, which is super cool. We'll roll that out in like Q1 to 2025, as well as like serverless, serverless spin up of those parallel environments, right? So they instead of spinning up something that's like, Oh, you know, I need 32 gigs for this service, and I need four gigs for all this other service, and you spin them up, and you leave them around over the weekend, and you just incinerate money for like these things that are idle, we will actually only spin them up for the time that you need and only charge you for the kind of usage that you have, right?

22:30And so ultimately, that means that like, you're going to avoid those like random errant, you know,$1 ,500 bills, because somebody forgot to like, you know, actually Terraform apply off of master instead of like their staging environment that they were testing, right? So, and I think that having that posture in place kind of by default means that companies get a lot more cost control. They get a lot more benefit, right? But they still get the ability to move extremely quickly by having the ability to create copies of their environment, right? How does monitoring, logging, observability, these types of things work?

22:58We have templates that people have built that allow you to kind of like exfil logs to Datadog. We have a template for spinning up Grafana or like Victoria Metrics, so Prometheus compatible instances. So you can do all of those things. We've also built from the ground up a observability system inside of Railway. And since we have the edge network, we automatically can kind of add a request ID. So we can kind of give you distributed tracing by default through all of your microservices. We can give you alerting for if things spike or stay high or anything else like that, right? We can give you information that you wouldn't have in other environments without doing, for lack of a better word, a ton of plumbing.

23:38So that's the thing that we've built from the ground up internally at Railway. And I think that obviously Datadog is a massive business and there's tons and tons of stuff in there. So we're straddling more of the 80-20 of let's just give people the baseline amount of things that they want. And over time, we're going to go and ask them, hey, what else can we give to you? But we have people who are just kind of using Railway entirely in terms of build, deploy, observe, scale, all of those pillars and are actually extremely happy on using just that. And I think it's bare bones in terms of where we want to take it right now.

24:09But it's super exciting to say, hey, listen, we have all the building blocks right here and we just need to work with our users to kind of scale to the things that they really, really want. What would you say are some of the constraints or limits today? Constraints or limits? That's an interesting one. I would say that there's like, I mean, it's going to sound weird, but there's not really any constraints or limits right now. And that's kind of like, we've really tried to solve that like Heroku problem of, oh, you know, I'm going to outgrow this thing, right? And so I would say that we're almost limited by trust, if that makes sense, right?

24:40And so, you know, you have AWS, you have GCP, and you have Azure. Those are the big clouds, right? Like, barring like anything, those are the big clouds, right? And you have like Cloudflare that's like trying to do things, right? But Cloudflare is like a$30 billion organization. They've been around for like a decade plus. They've accumulated trust. You know, they're the meme whenever Cloudflare has an issue, it's like software engineering snow day, right? It's kind of like they've continued to build that trust over time and they're still working on it, right? And so for us, I would say like the main thing that's kind of the limiting reagent right now is not like what the platform can do, but it's almost like how much you can trust it, right?

25:15Because when it comes to software infrastructure, it's your livelihood, right? Like there's not much more that you can kind of like trust to people, right? It's your data. It's the fact that if that thing goes down, especially with your data, you're SOL until these people like get back to you, right? And so you're kind of hanging in the limbo, etc. Right. So it kind of pulls more towards the quote of like, you know, nobody got fired for buying AWS, nobody got fired for any of these other things, right? And so I think that's the main thing that we're kind of like consistently working with companies on and saying like, yes, you're going to get all of these benefits, and we're going to give you like a higher order level of reliability and we're going to give you better service sla turnarounds than than the kind of like larger clouds and that ends up being kind of like a very very difficult battle to have with people because they just say that sounds like bs right and you have to just show them over time it's like no like we're going to continue to kind of like work on that and like we will be available for you should anything occur and we've also designed this system such that the there are less and less fault points as you kind of go i mean as a like if you're founder of a business and you're building out a new product, then it can be a lot to sort of bet your product life on another, essentially startup, going back to sort of the trust challenge that you're talking about.

26:23I think actually the startups really, they like are what we call from our like growth master master plan right now is like we're stretching market, right? And so startups like our current ICP, like the normal distribution of that, that kind of like go to market is actually 15 to 50 person teams. Those people like seem to love us, right? They seem to be able to want to move a ton of different stuff over. Maybe it's not literally all of their infrastructure footprint, but it's a large swath of things that are no longer legacy. And they're basically saying, listen, we want to move really, really quickly on these things.

26:53It ends up being those larger organizations that move a little bit slower and want that higher order trust bit really, really flipped. And so when you start going to like organizations, it ends up being most of the sales motion at that point ends up being this kind of like, how do we get past any sort of like trust or compliance or whatever objections, which we've gotten really, really good at and show you the value to kind of like tie it to maybe one of your like top eight, like velocity initiatives where like we just want engineers to be able to ship faster and get get more done, you know? So if you are a larger organization, let's say that you you're able to establish that trust with them, like what is the starting point for them?

27:29Like, you know, obviously, I'm not going to just if I'm not, you know, hybrid cloud or I'm on cloud today, I'm not going to go and like replace everything. Like, how do I get started essentially in a way that doesn't require me to boil the entire ocean? Yeah. So, I mean, the nice thing about Railway is like you can incrementally adopt it. Right. And so if you have services that you want to go in and spin up internally and you want to like pair them with services that already exist, You can do that using like something like tail scale, or we like have a wire guard binary that allows you to like mesh it into your instances over there.

27:56We can also do potentially dedicated instances if you want after chatting with you. So there's a variety of different ways you can get started. But the main point is that people just kind of like incrementally adopt it. They'll start with something they'll basically start with usually kind of it's an EM messing around on the weekend to basically say, all right, how do I go and explore a couple of these things that I know are being pitched as like these faster alternatives to X. but I need to be able to know that like, you know, it's good. Like it's, first of all, it's going to satisfy that like faster initiative to X criteria that I'm looking for.

28:23And two, that, you know, it'll be solid for us to like go in and make a case in the future. Right. And so what we do essentially is we go and we pull telemetry from people as they're, they're signing up and we say like, Hey, when you start getting to points where you want to be activated, we basically just say, Hey, if you want to go and chat with us, you can chat with us over Slack. You can chat with us over email. Right. Like we want to be really, really available without being kind of all up in their face, you know? And then you've open sourced like a significant portion of the railway production stack.

28:48Like what was the motivation behind that? I think the motivation for us is that we want that trust. Right. And so if we can give you this kind of like ability to introspect the service, maybe even like self-hosted in the future or anything else like that, then realistically, there's not much more you can trust us if you can see if you can see the guts and the internals and anything else like that. Right. So that's kind of the main motivator, I would say, in general. it's also obviously excellent to have members of the community be able to like go and contribute we have like people who are like submitting like nix packs prs all the time to like go and add new versions or new providers or new languages or new like you know dependencies or anything else like that and so getting kind of that tailwind of like open source not like tailwind in the like but like the benefit of the open source community to be able to like go in and help us like build this together you know that's i think a big key and i also like kubernetes ends up being open source right and so it's like do you want to potentially you kind of have to go and meet people where they are.

29:41And if they're self-host, if they're kind of self-hosting some of these things, at some point you need to be able to allow them to self-host at least the data plane, right? Does that help also overcome some of the trust issues? I believe so, yeah. Because you can see everything that people are doing, right? You can see what they're doing on the instances. You can see what code they're running. You can see all of the other things, right? So I think ultimately that really helps with the trust issues. Maybe not issues, but like problem there in general, right? But at the end of the day, I think the main thing that you can do to make sure that people trust you is to once the stuff is up, keep it up and keep it running.

30:17Right. And just make sure that it's like a bulletproof experience. Right. So we've been like working, especially over like the last like six months to make sure it's like, we're starting to get, you know, multiple nines on the board of like, okay, cool. Like, this is like a level of reliability, especially with our new bare metal instances. We haven't had any issues so far. So obviously knock on wood there, but like building something that you have higher order control of so you can get that level of reliability so people can say actually i've used it for x i don't really have a ton of problems with it right so and then we've also scaled the like cloud version of it to like you know we're doing tens of billions of requests per month on the edge proxy we have like you know the orchestration engine in the cloud is managing like two plus million microservices on us on like i think four clusters right so that's kind of like been built out so that whenever we go to like have a conversation with larger companies like uber or anything else like that.

31:035 ,000 microservices, that's a lot of microservices, evidently, right? But the cloud version of what we built is like scaled to like 2 million, right? So it's like a few orders of magnitude off. So we can say like, hey, if we were to go and like do a self-hosted version of this for you, we can do that like level of scale. You previously worked at Uber. Did any of that experience sort of like inform or motivate you to start Railway? Yeah, definitely. So there was, you know, kind of this like walled garden of like platform teams where, you know, some people were responsible for like getting code deployed and stuff like that.

31:31And I would go and interact with these things at work and it would be like fine, you know, and then I would go and interact with them at home and there'd be like varying levels of less than fine, you know. And so there was just, it really seemed that, you know, Uber was a high growth company at the time, still is, you know, it's massive. And, you know, they had the ability to retain the best engineers to go in and build this thing. And I still could potentially see ways that they could be like significantly better just in terms of like unlocking developer productivity. So I would say that that is definitely like a driving function for like making this system.

32:05So you've been working on Railway for what, almost five years? Yeah. Yeah. About five years. I think four and a half now. And as like a first time like founder, you're running 100 % remote. Like was that a conscious decision? Yes, it was a conscious decision. So I've worked remotely since 2015, maybe. And I've always like really, really strongly enjoyed working remotely. It's definitely not for everybody. I think it requires people be almost like extrinsically motivated about the problem, right? So people have to be like, they have to really, really like the thing that they're working on. And they have to kind of like see the vision, they have to see all of this other thing, like the where it goes, and everything else like that.

32:41So you can't kind of like get a lot of the benefits out of, you know, being in an office and, you know, kind of being able to breathe down people's spine and about saying like, Oh, we got to like do all of these things, right? So I was chatting with somebody, one of my friends the other day, and I was like, I think maybe my one of my most controversial opinions is like, I think despite running a remote company, I'm like, I think it's a terrible idea for probably about 90 % of people, right? Because you have to be extremely deliberate and you have to like hire people who are really, really excited about the problem space.

33:07We're going to like self-manage, we're going to go and do all of these things. Right. And so that's what we've done so far. We've got like 25 people. We're like remote spanning all the way, you know, like we have some people in like the Western hemisphere of Canada and I'm from like Vancouver Island, as we like previously mentioned, right. I'm in San Francisco now. And then we expand all the way, We've got Thailand, we've got Dubai, we've got Japan. So there's a lot of different time zones that we're spanning in general. And so we've had to hire people who are autonomous and who can push these things.

33:35So that's been excellent from leverage. But it does also mean that there's a specific class of individual who does really, really well at remote companies. And there's a specific class of individual who does well in in-person companies. It's kind of more of a chocolate or vanilla. But yeah, it was definitely a conscious decision because I do enjoy the benefits that remote has not in like a, you know, you can sit around and kind of twill your thumbs. But like you can almost like meet with people at almost various different times in the day. And, you know, their morning can be your evening. And so you can almost have this like almost like time compression handoff of saying like, oh, we've really got to go and do this.

34:08And then you go to bed and then you wake up and it's done. And you're like, damn, I love working with excellent coworkers, you know. And so it means that you almost get twice as much time in the week if you can get these handoffs like 100%. In terms of hiring people that are extremely motivated by the problem and also are able to self-manage, I feel like that's kind of a requirement for any relatively small stage startup. Because you just can't afford to have to micromanage people in order to do their job. You have to hire people who are motivated to be there, believe in the mission, and also can essentially just get things done and understand what needs to be done.

34:41I totally agree by the 100%. And the corollary of that is that like most people don't become like good managers at the start, especially like first time founders. You'll hear this from like, you know, Emmett from from Twitch has talked about like how he was not a great founder originally. I think Brian or Joe have also talked about it from Airbnb. Right. And so it almost like forces you to develop those like skills while you're also trying to assemble the airplane. Right. And so you don't get a lot of that kind of like benefit that you can kind of like smear this over time and learn those as you go through all of these stages.

35:10you really have to kind of like learn and compress a lot of those learnings and you have to do it in there so like that's the only kind of like thing that i i generally see is yes you all have you'll have to do all things eventually but we also talk a lot about you know internally about focus right like how do we stay focused on just doing the things that we can be like top number one in the world at right because that's where we're going to have all of the compounded returns right so those are kind of like some interesting benefits and drawbacks to remote that i think that maybe people don't necessarily consider, you know, they consider the, yeah, sure.

35:39I can go out and, you know, go for a bike ride in the middle of the day and work slightly later and stuff like that. And that's kind of like copacetic with my, you know, operating cadence or, you know, we have, we have people who have families, you know, and they spend the afternoon with their families and then they'll come back and, you know, polish up the stuff once the kids are to bed. Right. And I, I personally, I'm a big fan of treating people. It's going to sound a little bit condescending, but like treating people like adults because everybody is adults, right. It's like, they're going to manage all their time and they're going to like go and do all these things.

36:06Right. And you shouldn't kind of be around and basically say like, oh, I'm going to approve all your PTO requests. I'm going to like go and do this. Right. Because again, those kinds of people that you mentioned, like they're not going to be successful at startups in general. So we've almost kind of erred on the side of like, let's just pretty rapidly remove any of the guardrails in terms of onboarding and say like, hey, this is how we kind of operate. If you really, really like it, like here we are. And if we don't, then like, you know, we can, we can hopefully, hopefully find a really, really awesome spot for you to like go and land in the future.

36:32How do you test for that? Like, how do you find these people that are going to be a fit? Yeah, so there's a couple of problems that I like to ask people. There's a couple open-ended kind of like technical problems in terms of like design. I don't think you test for this using like lead code, you know? There's a lot of problems with using things like lead code. Oh, yeah. And so like we prefer to go for kind of like maybe the Montessori school of management of like open-ended kind of like problems of saying like, hey, listen, like how would you like solve this class of like real world problems, right?

36:59So I think that's like one way to go and do it. I think also sitting down and chatting with people about what drives them, you know, are they passionate about X, you know, stuff like that. And it doesn't need to necessarily be dev tools, right? Like it could literally just be like, I'm so passionate about networking, right? Like it's just networking. I got, I got a wrist of switches in my basement and all these other things, right? And stuff like that, right? And I think if people, they have that thing that they're really, really passionate about, and you can almost like see it and stuff like that, it's going to sound like super realistic.

37:26But like, you can almost tell when like people really, really care about these things, because the moment you poke them, they almost say it kind of like expands. And you're like, Oh, my God, like, that's so much stuff, you know, like, where did that all come from? Right. And where it came from is this kind of deep passion, this life experience, all these other things, right. And so that's like one way to go and test it in the interview. And then as part of like onboarding, what we do is we rapidly kind of will just remove the guardrails as you go, right. So like, we have six weeks of onboarding, which like may seem long, but the goal is almost pushing out the funnel on what you can do, right.

37:56Because by the end of the six weeks, we consider onboarding to be like, how do we get you from good to doing great work and then being able to be fully autonomous? So that's the whole goal is we do two weeks of tasks. So it's like five tickets. You just have them in linear. They'll be really straightforward. It's like, hey, this thing is actually broken. It's like, you go and you make a few lines of code changes. And then that's your first two weeks. And then you move on to the problems, which is, hey, our current experience sucks. Right. Very different class of problems that we've just given you.

38:23Right. It's like, well, what sucks about it? Like, what are the problems? All those things. Right. And then people have to like go in and, you know, maybe they'll ask their coworkers, maybe they'll go and ask the users, maybe they'll go and put together a document that says, I think these are the things. And then people will be like, no, I think, what about like, we could probably drop this requirement. Right. And then they'll go and solve those problems. And then the third one is like opportunities, which is you've been here a month. What do you think the company needs? That's the only prompt.

38:45Right. And so you get to that open ended kind of like line of thinking by the end of it. And, you know, that's that kind of pushes people more towards like, oh, I can actually do pretty much anything here. Right. And so it's just a matter of what. Yeah, yeah. I think there's a certain like, you know, back to what you're talking about, where you're trying to look for, you know, what is the person like really passionate about or interested in? There's a certain obsession behavior that I think people need to be successful in startups. And it doesn't even need to be historically an obsession that relates necessarily to what the company is doing.

39:17But you have to kind of be manically obsessed about solving something in your life. Because a lot of it is really about knocking down doors and solving problems. And the level of abstraction also hits you much faster earlier in your career, I think, at a startup because there's just less people around. To your point, by week six, you're asking questions about what could the company be doing better. If you're at a really large organization and you've worked for some, I've worked for some, Like you could really be in that first stage of like, here's a really small problem that you need to solve.

39:49And that could be like the first two years of your life in existence there. Yeah. Right. So we aim for trying to find those people because we think that I think even when it comes down to like solving problems excellently, right? Like that last kind of like 5 % of the problem is kind of where most of the progress is. Right. And if you're not like focused and you know, you're consistently oscillating and bumping between all of these different things, right? You're probably not going to like get actually to like the meat and potatoes of like what that problem actually is, right? I'm a very, very big believer of sit there and run a ton of different revisions, right?

40:23And then figure out like why these things were either better or worse than your previous revisions, right? And make sure you have a clear goal that you're kind of like entering towards, right? And I think that doing that and doing that well, you know, that focus and that passion is like, it's almost like a necessary precondition. It's like, it unlocks like pretty much all of those other things, right? And I don't see how people can potentially do it without it. I've seen it once in a blue moon, but it's also very, very rare. So, you know, it's we aim for kind of saying like, OK, well, like how do we generalize this class of problem of like finding these extrinsically motivated people?

40:54Right. And saying like, this is kind of like in general, the archetype of individuals that we've seen be quite successful here. Yeah. I mean, there's a quote about it. I don't remember the exact quote, but it's like, you know, we've completed 90%. So now we get to start on the other 90 % of the project, essentially. because exactly it's really that like last 10 % where all the hard work the meat and potatoes has to exist in order to actually whether it's a product or whatever it is that you're building that to do that that's where the polish happens that's where you know you run into your scale problems and so forth it's also important like not just from like an individual perspective but like from working with other individuals perspective because that last 10 % it's the hardest like that last extra mile of like going and doing the thing it's like yeah well like this is like good enough right like it'll work right and if you work with people like who have you know outsized talent or outside standards or anything else like that, they'll basically say like, no, we can do better, right?

41:44Like we can push this thing a little bit farther, right? And then they'll have some like tools in their toolkit, they'll have, you know, some skill or some emotional acumen or some way to kind of like judo your thinking and basically saying like, what have we thought of it like this? Like, I think this is really, really close. And this is where the reason why it's really, really good. Like, can we try this? Right? Versus, you know, I think the conventional thinking at larger companies is like, okay, cool, it's done. Like, let's just move on to the other thing. Awesome. Jake, thanks so much for being here.

42:07Cool. Awesome. Yeah. Thanks so much for having me. This was great. All right. Cheers. Cheers.