1. Podcasts
  2. Agentic Conversations (formally mlops.community)
  3. Multi-Cloud LLM Routing and Context-Aware Coding Agents
Agentic Conversations (formally mlops.community)

16 Nov 2026 · 29m · 12 chapters

Episode artwork

Multi-Cloud LLM Routing and Context-Aware Coding Agents

In short

How AI gateways and multi-cloud LLM routing power context-aware coding agents, plus how agent “experience” is evaluated and used for automated PR review and security/compliance checks.

Guests

Matt (Cloudflare, works on Flu/agent stack; built an internal AI code reviewer on Flu); Scott (Cloudflare/Miro discussion partner); additional Miro/Cloudflare engineers are referenced (Sanket, Svetlana, Murad) and “Agent Experience” team members at Cloudflare.

Key claims

AI gateways combine model calling with business logic (e.g., Germany-only European model routing). Routing becomes multi-dimensional (multi-region, multi-cloud, per-model RPM/token limits). Agents should use company-specific context pools (e.g., “engineering codex” for PR quality) and require agent-experience evals/benchmarks.

Notable examples

“Easelbot” (company-context PR triage using app logs), normal Claude reviews, and an automated AI review bot that learns from approve/reject relevance; Oplane security/compliance checks for PRs; Cloudflare “Cloudflare OS” and Flu framework; internal benchmark of frontier coding agents deploying Cloudflare workers from scratch; open-source “close the loop” example (Astro reaching zero open issues).

Written by AI. May contain mistakes.

Chapters

0:00

Introduction to AI Code Review

Discover the capabilities of an advanced AI code reviewer.

“The biggest use case internally is we have this, I think, the best AI code reviewer I've ever used.”
1:04

Defining the AI Gateway

Learn what an AI gateway is and how it functions within Miro.

1:20

Multi-Cloud Model Routing

Explore the evolution of model routing across different cloud providers.

“the last conversation I had with the Booking.com team, they were talking about the MCP gateway that they put together.”
3:02

Introducing Miro's SideKicks

Understand how Miro uses AI agents for complex routing tasks.

“We want to deploy this not only in all the regions that we can within AWS, but guess what?”
4:50

Cloudflare OS and Frameworks

Get insights into Cloudflare's OS and the Flu framework for building applications.

“as I said, it allows us to do a little bit more complex things on the back end.”
7:15

Personalizing Code Review Agents

Examine the difference between customized and default code review agents.

“So yeah, we're running a bunch of production workloads internally.”
9:27

Evaluating Agent Performance

Discuss the challenges of evaluating the performance of coding agents.

“Maybe it's a ponytail skill or something.”
12:08

Internal Tools for Code Review

Learn about the internal tools used for code review and their impact.

“They help with other interfaces, so they help with the MCP server to interact with things, and we're constantly experimenting on that front.”
14:04

Exploring PR Management and Automation Tools

Learn how PR management and automation tools like Oplane improve code review processes.

“That one's a really good security if you guys have discussed it.”
17:52

The Evolution of Open Source Contributions

Discuss how open source contributions and maintainership are evolving with technology.

“I went to a talk that he did and it was talking about open source contributions for companies, right?”
Show all 12 chapters
23:01

The Role of Software in Diverse Backgrounds

Examine how diverse backgrounds in engineering can enhance software development and innovation.

“of the idea that i wanted to get after with you guys your hot takes you got any good ones for us I can give you mine.”
28:00

Philanthropic Perspectives in Tech

Exploring how tech can address large societal issues and personal passions.

A man walking through an airport, asking his phone a question

Ask about this episode

Get the gist, explore an idea or ask about something said.

Claude

On Agentic Conversations (formally mlops.community), what are the key points of this episode?

Searching your listening

Transcript

Automatic transcript. May contain errors.

0:00Scott Crooks: The biggest use case internally is we have this, I think, the best AI code reviewer I've ever used. I have no idea when we're ever going to share it externally or if we ever will, but it is amazing and it's built on flu. Is it primarily for humans to read?

0:12Matt Carey: It's only for agents to read. When people approve or reject, our AI review kind of looks at that and says, OK, this is not only what I think needs to be fixed, but what is important to people. because that's the more interesting statistic that we found with our internal tool is how many PRs get opened, how many get merged, and how many get closed. PRs are cheap, man.

0:34Scott Crooks: PRs are very cheap. Yeah, it depends on your model, but yeah, potentially they're cheap. And maybe this is super sad. I think agents are bringing us into the terminal and away from people. You've got to get out and meet people and do things like this, yeah?

0:45Matt Carey: I think my first hot take is almonds are the only nut that you should eat with chocolate.

1:03Matt Carey: we're back i'm with matt and scott how you guys doing yeah pretty awesome awesome dude i want to talk about so much with you there you're at cloudflare you're at miro there's so many different things that we can go into i think we were just talking about ai gateway right yes and it's funny the last conversation I had with the Booking.com team, they were talking about the MCP gateway that they put together. What do you define as an AI gateway? Yeah, I can tell you how we define it. I'll give maybe the quick understandable version, which is similar to, I think a lot of companies did this. It's, hey, we need something really quick.

1:43Matt Carey: Let's just throw it together. So it was whatever use cases were amalgamated to create that. But for Miro, what that means is it's two things, really. It's an AI gateway that actually does the model calling, right? So like there's the back end, let's say, of models. In our case, we have a good mix of self-hosted versus just regular LLM calls. And then the other part of that is the business logic. So Miro has some very specific things that we need to do in order to make sure that customers are happy with our product. A good example of this is you're in Germany. You say, I only want to call European-based models, right?

2:22Matt Carey: That's a very simple use case. We code that in. And so these sort of things is what's kind of self-contained. Is it also routing? It's also routing, yeah. So one of the things that we've seen is an evolution in how the models, particularly with the cloud providers, has gone. We started with AWS. You would have models that were really only in certain regions, right? And so as they were also figuring out their infrastructure, we said, okay, let's route everything to this region, get everything working. As we've gone along, we've seen sort of dimensions come along. So we've not only seen multi-regionality, because we deploy in multiple regions, but also multi-cloud.

3:02Matt Carey: So this is like, we have Opus 5. We want to deploy this not only in all the regions that we can within AWS, but guess what? you can also serve that through Azure, through GCP, or Anthropic Direct in certain cases. And so it becomes this sort of like three-dimensional problem of like multi-region, multi-model. In the end, what our model gateway does is, you know, we have a pool of providers that allow us to essentially define, okay, for this model, what kind of requests per minute can we get? What kind of tokens per minute can we get? And all of that just gets aggregated. And then we make routing decisions based on this.

3:39Matt Carey: Dude, I totally forgot back in the day where you could only get open AI through Azure. Exactly. And you saying that reminded me of like, oh yeah. And when you remember when you get rate limited, like after two prompts, you're like, what happened all the time? And so now, yeah. Okay. You've got these, as you call it, multi-dimensional and you have not only for the same service, if you're dealing with the anthropic models, but then I I imagine you've got these other models that you're dealing with. Yes. And so maybe you've got some open source ones. And are you routing depending on the complexity of the tasks?

4:14Matt Carey: We are, yeah. So Miro has a program or a capability called SideKicks. It's a fancy way of saying AI agents. And it does exactly this. So we have default models defined. And then there's sort of a layer that also does an analysis and says, okay, the customer requested this, but they also, let's say, want to maintain costs, and they talk to us about this. So we also give them some logic around, yeah, like kind of an auto mode. Nice. So yeah, this was also something that came about pretty quickly, and because we have the model capabilities and the pool, as I said, it allows us to do a little bit more complex things on the back end.

4:54Matt Carey: You guys built something like this too? Yeah, we have an AI gateway for customers, yeah.

4:59Scott Crooks: Yeah.

5:00Matt Carey: And you came out with the OS recently too, right? Can you tell me more about that?

5:06Scott Crooks: We have like a full sort of like agent stack. So the AI gateway can route to external model providers and can route to like the workers AI hosted models. And then we also have like the application layer on top. So we have the agents SDK, which is for people building their own stuff. We have Flu, which is like a big old framework for building agents. It's kind of taking a new devx around it's like from the from the makers of astro it's really really interesting actually and then we also have this brand new thing i was gonna go through all them with this brand new thing which is like this full application that's open source that you can deploy yourself and it's like the uh uh you can think of it's like the off the a uh agent office suite for your company and we call that cloudflare os so i have a version of it deployed for like um literally like me and my my girlfriend as well and we like we have it in in our house uh which is kind of fun uh but then you can also what is in the house oh it's like just like shopping stuff like like like literally food shopping is like the easiest one but um people like come like there are a bunch of companies i have no idea which ones i'm an outer name so that have it deployed like internally as they're like all in one office suite you can make slides out of it you can build these things these apps like blueprints for these apps so you can have like your finance team building like special dashboards and then you can save them as blueprints and then you can other teams can spin them up you can have like uh skills via by like job title and things like this and like job and it's a really really fun uh and it's built on this uh capability model where the agent only gets access to the app that it's looking at um and like all internal to the cloudflare network and what's the astro one that you're at flu uh the flu framework yeah so um we released that well the the makers of astro sort of went out on a tangent actually and released that and it's it's really really cool um it's a dev experience it's kind of like a react like functional dev experience to build uh agent control flow uh and it just natively deploys to anywhere the javascript runs so it can you can deploy it to like a node.js server you can deploy it to um like for cell fluid compute or you can deploy it to Cloudflare workers.

7:14Scott Crooks: Okay. Have you played? Yeah, Flu. Yeah, yeah. The Flu. I work on Flu a bunch. So yeah, we're running a bunch of production workloads internally. We hack on it. We build stuff with it. We build applications on it. And we give applications to our customers on it. What have you been building? I think the biggest use case internally is we have this, I think the best AI code reviewer I've ever used internally to Cloudflare. I have no idea when we're ever going to share it externally or if we ever will, but it is amazing and it's built on flu.

7:48Matt Carey: Why built on flu with the code reviewer? Is it just like reading these files? It's reading your commits?

7:57Scott Crooks: Yeah, all of the above. And it's got its own custom tools. You're building your own harness. Like some people might want to run like code code in a container, or you might want to run like open code on a durable object or like Pi or like you can run all of these things, but it's very much like a lower level thing. Or you can, Flu is a whole framework. So you write in JavaScript, but you write like Flu semantics and then it'll deploy wherever you want to deploy JavaScript. And you can specify all of these details. Yeah, like you can specify tools, prompts. It's really the innovative thing with Flu is like the control flow.

8:34Scott Crooks: so it's very easy to do things like lazy loading of tools and like more interesting harness engineering stuff because of the way that uh fred and the astro team have done the control flow yeah i see it's it is really good fun it's really fun and it'll run anywhere that javascript runs so it's not just like tied to cloudflare you can you can run it wherever yeah even runs in the

8:54Matt Carey: browser oh nice dude well as you're saying that it just makes me think like man i haven't gone deep on personalizing my coding specifically like the pr reviewing agents because i get so much lift with the ones that come out of the box how much of a difference do you feel like there is between those two when you're going in and you're tweaking everyone versus the uh just like yeah It's a skill that you get off the shelf. Maybe it's a ponytail skill or something. Yeah, I think it's all about like

9:32Scott Crooks: how much collective context you have about your company and about like what your company denotes a good PR, I would say. At least that's the way we started. Clubhouse was cool in a few months ago, well, actually almost a year ago, we started developing this internal engineering codex, which I think is public now, but it's like how much, it's like what denotes what tools we're allowed to use, what tools we should be using, how to make exemptions to this, how to like think about approaching an engineering problem at Cloudflare. And a lot of other companies have similar versions to this. Like I know the very famous ones like the Airbnb style guide for code, right?

10:10Scott Crooks: Like people used to write JavaScript in the Airbnb style guide. Now I think you can build whole applications with the Cloudflare codex. I think that's how we're trying to think about it.

10:18Matt Carey: Is it primarily for humans to read or for -

10:20Scott Crooks: It's only for agents to read. yeah it's only for agents to read so it's built by um agents based on like extensive research about how the company operates and we're trying to like write down how the company operates and like trying to be aligned across the whole company like cloudflow is four and a half thousand people i think and trying to be aligned about everyone's coding agents should know how the company works and how the company operates where to go for different things um yeah it's a massive task but when if you can do that and you can generate this like codex uh codex is such a loaded word now uh but if you can generate this like this like uh um context pool then you can have really good success with internal agents because the internal agents can also do um can do like

11:02Matt Carey: best practice yeah one thing that i find super difficult is just knowing how well my stuff's performing versus like what it should be at or what is the top end of this performance and so like evaling it and going through and even just something as simple as documentation and the agent's ability to understand and read the documentation and evaling that you know is something that you wouldn't necessarily think about when you're making it for humans yeah because it's a human all right like i want to write the best i can i want to make sure that these docs are well written but you're not like having to go and then like say can we eval these docs

11:47Scott Crooks: right it's this whole new field of like agent experience yeah um and i think we had liad uh uh presenting here in mcp agent con um and their whole company is around like aura is around like agent experience there's a bunch of other startups are trying to crack this like agent experience at Cloudflare, we have a team called Agent Experience, and they do awesome stuff. They maintain the dashboard agent, but they also maintain every other interface. They help with other interfaces, so they help with the MCP server to interact with things, and we're constantly experimenting on that front. And yeah, we have evals.

12:23Scott Crooks: The coolest thing I think they did recently was a full benchmark for how well do the frontier coding agents, how well can they use Cloudflare natively? How well can they use the APIs? Like, do they know our open API specs? Can they deploy a worker from scratch given like literally nothing, just internet access? I think that's really fun. Have you guys experimented with anything like this?

12:43Matt Carey: Yeah, we have an internal bot that does exactly this. So it seems like there's a common pattern emerging. So we call it, so Miro is the canvas, so we call it easel bot, right? Nice. It's pretty cool, actually. Yeah, so what we noticed was with the code review problem, we actually have like kind of three checks and they all are similar, but they all they seem to hit different areas, and I think that's why we've kept all three of them. So we have Easelbot. That is more of getting the whole company context, fixing errors where it finds them, but it's also our application that gets access directly to our application logs.

13:19Matt Carey: So when you wake up in the morning, typically on our bigger, more active code repositories, we'll see two or three PRs. Hey, I saw this has been appearing like 10 times in the last hour. This is what I think is the potential fix. So we have that level. And then for the individual reviews, we have just normal Claude, right? It's like, hey, Claude, can you review this, please? And then we have our own AI review bot. This one is also automated in our pipelines. It's kind of a combination. So I think it takes a lot of the potential fixes that have come from Easelbot. So when people approve or reject, our AI review kind of looks at that and says, okay this is not only like what i think needs to be fixed but what is important to people because that's that's the more interesting statistic that we found with our internal tool is how many prs get opened how many get merged and how many get closed yeah um and so we find like there's about a 50 50 split so right now we're still trying to trying to find that sweet spot because yeah our prs can be open these days i'll tell you what yeah so we and the good news is that like we have still like engineers who are looking into this and and you know we have on duty people saying yeah this is relevant this is not so it's kind of like a you know we're experimenting through reinforcement learning like this pr is not relevant close it and then that just feeds back into the system yeah so it's nice because we have uh not only multiple streams for when some some when you're just opening something and then you get the review but then we also have on the back end like looking at the contents and see if it were if it was relevant or not um so we get it from both sides.

14:55Matt Carey: In addition, we also use Oplane. That one's a really good security if you guys have discussed it. Has it come up before? No, I don't know what this is. Oplane. O-P-L-A-N-E. It's kind of like a security check, let's say. You open a PR. It gives you kind of like a breakdown of compliance and it can really range from anything. It could be something as simple as we use GitHub, so we have GitHub Actions. It could say, hey, you haven't shoppinned this action. And so it'll give you an explanation of like why this is important. And then after the fact, it'll also get more into detail about like your specific change.

15:35Matt Carey: So it's like a kind of like an all encompassing security review so that like when you merge it, you feel confident like, okay, like I've hit or the agent has hit like kind of the basic. I'm not exposing any secrets. I'm not exposing, yeah, exactly. So it'll give you like recommendations on like how do you redact data? Like it really is kind of an all encompassing tool.

15:54Scott Crooks: so it's great i think like the upshot of i guess this conversation i didn't realize how much you had internally that's so awesome but like everyone is building these like software automations i think they're still with like the intention of them becoming like full factories yeah like everyone's trying to close the loop at some point to become full factories uh and i think the cool is one of the fun examples that people can go and have a look at because it's very public is how some of the open source repos are actually doing this like close the loop um astro the um the people I talked about earlier making this JavaScript web framework, they got to zero open issues today for the first time ever, like from like literally thousands of open issues, they got to zero.

16:34Scott Crooks: And they are, they are going for inbox zero from now on, which is insane. Like, because up until now, I think like open source repos have really had this like deluge of people. I want to make a, I want to make a contribution. I'm going to throw my agent at this potential problem and see what, see what it comes up with. I'm going to scan the repo and find these like security vulnerabilities, which are potentially.

16:56Matt Carey: I know the guy who created superpowers that pack, Jesse, he told me that for his open source projects and superpowers, he was like, I get a lot of PRs for superpowers. And a lot of times it's very low quality because obviously, yeah. but he built out ways to just like automatically sift through the low quality ones and then the ones that should get the human's time get bubbled to the top by having like some very firm rules

17:29Scott Crooks: yeah i think everyone everyone is doing this like in the extreme cases for a lot of the mcpsdks i can't speak for all of them but we've actually turned off external uh non-collaborator prs and people just make issues and they can if they want to have a go at making a patch they can link the patch in their issue and then we can go in and do it i think there's this like whole there's this whole thing about how can you reduce that maintain a burden and how can how can you actually automate each of the points but have some type of human review and like whatever people are comfortable with because prs are cheap man well they're very cheap yeah it depends on your model but yeah potentially that that they're cheap uh but also a lot of the time i think open source maintainers have a very like strong idea of where they want the project to go and so them knowing about issues is very useful but then actually the implementation has to fit with the direction that the project is going and that's often that previously involved like a lot of backwards and forwards uh between maintainers and the person wanting to fix the issue but now it is just cheaper the backwards and forwards is sad and maybe this is super sad like i think agents are bringing us into the terminal and away from people um you don't gotta get out there and yeah you gotta get out and meet people do things like this yeah that's true we've spent like the last two days in mcp maintainer meetings and it's just been so fun just to like riff on ideas and things when normally it's uh it's just it's it's very much agent to agent in the world right now you're talking to your

18:53Matt Carey: llm too much and makes you delusional sometimes you gotta touch grass we'll say i hope that this also has another an opposite effect like a few years back at kubcon it was the one that was here in Amsterdam and Kelsey Hightower, who of Kubernetes fame. I went to a talk that he did and it was talking about open source contributions for companies, right? So it was specifically like, okay, you're coming to this talk because you're at a company, you want to open source something, like let me give you some advice on that. At the end of the talk, what he said was like, you know, you have to think not only about the monetary implications, right?

19:29Matt Carey: Does this make sense for your company to put time into open source, but it's also a reputation thing. And so he said, you know, I used to work, he mentioned a company, I won't mention it here, but he said, I used to work and consult for this company. You know, they were very famous very early on when the microservice kind of trend was taking off. You know, they released a lot of products, a lot of libraries, things that took off, and they didn't maintain them. I hope in the future that this will also kind of encourage companies to take something that they really are developing internally and say, you know, this is something that should be public.

20:04Matt Carey: The more recent one, the talk from Spotify just a couple hours ago, this is exactly they're like, hey, we have this MCP gateway that we want to internally have people use. The fact that they're developing this, they already had success with Backstage, and so I think people are now like, please, yes, like release it. So having this agent kind of world also I hope enables that to get more quality products out the door. And there's certain companies that it feels like it's in their DNA a little bit. I know that before backstage even, Spotify put out, was it Luigi in the data world? I don't know if you're familiar with that.

20:43Matt Carey: It was like an Airflow competitor. And I think the guy that wrote it, do you all know Modal? Eric Bernharsen? Yeah, yeah, yeah. I think he created, I want to say the name is Luigi. Somebody can correct me in the comments. But it was that. I've heard this story. I've asked him about this as well. He was working at Spotify, and then he put that out, and then Airflow went out a little bit before or a little after and then kind of took all the air out of the room. What I'm trying to say, though, is you've got some companies that are very much like, yeah, cool, let's release it, let's put it out there, and let's make it a thing.

21:20Matt Carey: And then other companies, I think they're learning how to do open source because maybe it's not as big of a priority.

21:27Scott Crooks: and i guess i would say like the lms are making um allowing people to generate vast amounts of code very very quickly whether whether your company wants to support that code in the long term is interesting and maybe the value of like fun ideas goes down or like fun fun demos goes down like previously you'd see a fun demo and maybe they didn't release the code and you were like maybe a little bit bummed out by that because you're like oh i like want to see that but now like that's so true i can just make myself now like from the front demo you can make it yourself so maybe they don't need to release the code maybe it's still helpful but if they do release the code then it's worth like marking it as like this this is like static in time like this is a yeah this is a this is a moment and like we're not going to carry on from that and i think a lot of companies who have like an open source direction like cloudflare we have like a bunch people working on home source i work on the mcp sdk like half time um we it's very hard to work out like what what you're going to support a lot longer term uh and also how you deprecate things in a nice way for customers like how can you be like this was a time and a place but actually the whole industry has moved on right now and uh this is going to be the last thing guys yeah how do you set that expectation we donate it to someone if people are still using it or can we have like like we don't want to support this anymore um i'm not saying i'm not gonna name specific projects but like that is definitely a thought process it's a life cycle of a lot of different projects or can we make something stable so that people who are using it come they can carry on using it

22:56Matt Carey: but like we're not going to continuously develop this yeah well speaking of which that reminds me of the idea that i wanted to get after with you guys your hot takes you got any good ones for us I can give you mine. Go for it. Wait, I have a whole list of them on my phone around here that I wanted to share at some point. I think my first hot take is almonds are the only nut that you should eat with chocolate. Walnuts, no good. That's so hot because hazelnuts. Yeah, hazelnuts, no. It's too much chocolate on chocolate. I don't like it. I guess does the whole Dubai chocolate craze fall into that?

23:38Matt Carey: Exactly. So that's out. I'm not a fan of the Dubai chocolate. Again, almonds are the nuts. That's my hot take.

23:46Scott Crooks: Yeah, that's too hot for me. Hazelnuts work so well.

23:49Matt Carey: Yeah. No, I'm out. I do have a hot take, and I have to give credit to some of my fellow Miro engineers. There's a lot of us here. So it was, let me see, it was Sanket, it was Svetlana. Shout out. Yeah, and Murad. So we were, hot take, it's more of like a, we're coming to this conference and all of us are from you know different backgrounds different countries and i think the the one interesting thing that we all sort of came to the conclusion of is is that because now software is so easy to write um the time you you're a little bit free to think of other things and one of the things that we all agreed on was the combining of something in the physical world with with software now becomes just that much easier and and will make life that much better um not only that but being somebody who is maybe from a different background in software so for example i was my background is chemical engineering oh nice right mechanical yeah so so same it's you you come from this background it's like well what does that have to do with software and it's like well some would say nothing but but others it's like yeah but you're also learning kind of like generic concepts that you can then apply to to other domains right And that's the power of it.

25:04Scott Crooks: Yeah, we mean to write this article about the scholarly engineer, which is like, I guess, I'm still a software engineer, even though I write less code than I did previously. I also actually went through a phase of being like, oh, but I just read all the code. And that's how I imprint my mark on it when the LLMs are writing. And now I'm not even that pro that idea. I read actually much less of the code, but I still understand the ideas. And then there's going to be a point where maybe like I understand less of the ideas. And so that's the bit that I'm like struggling with, where if I'm not reading, if I'm not writing the code and I'm reading less of the code, at least I'm doing less code review than I was before.

25:46Scott Crooks: And I'm leaning much more on like my own understanding of how the system works. then actually in those previous two times when i would have been like like hunched over a laptop like writing and reading uh i should now actually just be learning more about the concepts of my industry the things that i can build upon like knowing how to prompt is actually just like knowing more about the world that we live in and like knowing more about like given this situation i should be thinking about this as an idea like that the scholarly engineer is like something

26:16Matt Carey: i'm really like it's the higher level concepts that you need it's almost like the abstraction is going up and so you're going to be much more effective if you understand what you need to use the right tools you need to use or the right words you need to say in the right moments

26:33Scott Crooks: something along my team calls it uh the his he calls it his gentleman gentlemanly scientist era which i find hysterical and it's like this vision of this uh like gentleman scientist with a big top hat and maybe a pipe just pontificating about the world and like reading like massive

26:53Matt Carey: books and yeah philosophizing about philosophy yeah they're yeah they're like uh the their

26:59Scott Crooks: impact and then doing very very small experiments that actually have like exponentially large

27:05Matt Carey: results yeah exactly that's the key is like all right well gathering a bunch of inspiration from random places around the world and around like the the educational sphere and then implementing that into what you're working on because uh i can't remember where i was reading it but it's pretty obvious that like being able to take ideas from outside of your little bubble and then implement them in your bubble is going to give you much richer results yeah perhaps it'll be in the future where you know instead of having like one career path and saying oh i'm going to be this uh for my for my career it'll be more like what what problem do i want to solve or problems yeah and then it'll be looking at the space and saying what information do i need in order to get to solve that problem yeah kind of combining things at the very end you heard of the idea of like

27:56Scott Crooks: general managers uh general managers of a problem i'll send you afterwards but it's uh it's more of a philanthropic viewpoint of like there is there are these large problems in the world like i want to be the general manager of this like singular thing yeah like yeah i don't know um i i wish the lms and i hope the lms move us more towards spending our time upon these like more philanthropic is maybe the wrong word here but like that's sort of good for the world like ideas yeah um you get to

28:23Matt Carey: level up and that's the cool part is that you're not spending your time being there which some people love i will say that like so if you love it it's just like you know working on old cars whatever or your motorcycle it's not totally necessary but if you love it do it go for it otherwise yeah you can abstract away and uh hopefully get these bigger ideas and and find be able to chase what you enjoy so i think with that man that's a great place to end it It feels like a good note.

28:55Scott Crooks: Nice. I feel warm and fuzzy inside. Yeah, it was good.

28:57Matt Carey: That was awesome for doing this. Thank you so much.

From the publisher

In this episode, we sit down with Matt Carey and Scott Crooks, engineers at Cloudflare and Miro, to look at how two very different companies are building their own AI software factories, and what happens to engineering when the code mostly writes and reviews itself.


We start with the AI gateway: why a model call is no longer simple when the same model is served across multiple regions, multiple clouds, and direct from the provider, and how Miro pools rate limits and data residency rules (like EU-only models for German customers) into a single routing layer that also powers Miro’s Sidekicks. Matt walks through Cloudflare’s agent stack, from the AI gateway and Agents SDK to Flue, a JavaScript agent framework from the makers of Astro, and Cloudflare OS, an open source agent office suite he even runs at home for grocery shopping.


The heart of the conversation is code review. Matt describes what he calls the best AI code reviewer he’s ever used, built on Flue and powered by an internal engineering codex written only for agents. Scott breaks down Miro’s three-layer setup, including Easel, a bot that reads production logs overnight and opens fix PRs before anyone wakes up, and explains why only about half of those PRs get merged, and how that signal is fed back to teach the system what people care about.


Along the way, we cover agent experience as a new discipline, how open source maintainers are handling a flood of cheap AI pull requests, Astro reaching zero open issues, when companies should open source internal tools and how to retire them gracefully, plus a few hot takes on chocolate and nuts. We close on the rise of the “scholarly engineer,” and why understanding ideas may matter more than reading every line of code.


Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmMatt Carey: https://www.linkedin.com/in/mattzcareyScott Crooks: https://www.linkedin.com/in/jshcrooks


Timestamps:[00:00] Cold open[01:03] Meet Matt and Scott[01:20] What an AI gateway really is[02:28] Multi-region, multi-cloud model routing[05:00] Cloudflare’s agent stack and Cloudflare OS[06:43] The Flue framework from the Astro team[07:32] The best AI code reviewer[09:29] An engineering codex written for agents[11:48] Agent experience and evals[12:44] Miro’s Easel bot and three layers of review[15:55] Software factories and Astro hitting zero issues[18:58] Companies open sourcing internal tools[22:59] Hot takes[25:03] The scholarly engineer[28:50] Wrap up

More from Agentic Conversations (formally mlops.community)

All episodes