AI agents are knocking. Is your API ready to answer? | GraphQL’s Matt DeBergalis

26:19And as people experiment and, you know, we're building agents, we're giving agents tools. It sounds like we're entering a world where API consumption is going to go up, maybe go up a lot. Do you feel that way? Yeah, absolutely. Yeah, and so we're the GraphQL company. Just to set the stage here, we are excited about GraphQL because it's a query language for your APIs. What it means is in a world of microservices and cloud systems and SaaS, maybe 10, 20 years ago, you only had a handful of APIs that you built your software on. But now you've got a typical app, maybe sits on 50 or 100 different APIs.

26:57And you need infrastructure. You need machinery to help orchestrate how all those API calls get made. The idea that you're going to handwrite code doesn't really scale when you start talking about systems of this size. Right. And so we've been working for years now on how to drive more value from APIs that people have. One of the things a lot of GraphQL users first come to GraphQL for is they've got an API that was built for one use case, but now there's this desire to ship a different kind of user experience. Maybe you're building a mobile application that drove a lot of the early excitement around GraphQL.

27:34In our customers, there's this push toward meeting the end user more where they are. Like if you think about a modern hotel experience, there's probably a kiosk in the lobby where you can check in. That's just another set of API calls. It's a different kind of software. Maybe you can unlock your room from your phone. Maybe when you go in your room, your name's on the TV and there's a welcome message. and maybe you can ask for a late checkout during your reservation flow online. Right. And that's an example. All of those are systems that are being repurposed and kind of recombined or composed into this new kind of experience.

28:10I think AI is just going to be like the mother of that, right? Because it's so flexible. And if you imagine what it's going to be like in a year or two, talking to an AI or using an AI first application. Yeah, it's going to make a lot more API calls. It's going to do it in new ways with new combinations. I think the standard software is always kind of limited. You can only ship so many versions of your app every year. We don't have that limit anymore. So we may also find that the pace of all this stuff just hits a new level that we're going to have to get used to. And that has implications too, for how quickly this stuff gets added.

28:48Yeah, and definitely a compounding speed factor. It seems to go faster and faster, the more sophisticated it gets. And, you know, your position and what you've been doing at GraphQL and working on Graph is, it puts you actually in this really unique spot of expertise, because you already know that you can't just throw an API on top of everything, you need to have a structured, orchestrated way of accessing your data that is flexible for all the different experiences you want to provide. And then now that's evolving, like you said, into an MCP type experience. And you see some companies that see this and just try to map like one-to-one API to MCP.

29:24People talk about how that doesn't necessarily work. And it reminds me a lot of how what GraphQL does on top of an API, because it allows you to ask for specifically what you want. It's a structured query language. And that semanticism is what is so important to it. And I think that that has a lot of parallels with MCP. Do you feel the same or do you think that it's kind of evolving in a different way? Yeah, that's a good example of where agent experience is going to be a little different from the human experience or the developer experience. It's really clear from work we've done and what we've done together with customers that the semantic precision of GraphQL is a great fit for the models.

30:08The models are really good about reasoning. and GraphQL, you know, if you haven't used it, it's strongly typed. It's self-documenting. So you can put English language doc strings, and that's been a best practice in the GraphQL world for a long time. There's even more than that. There's a place to put comments. And we found sort of along the lines of, you know, an llms.txt file, you can talk to the model in the definition of the graph in a pretty helpful way, it turns out, to help the model navigate. And at the end of the day, the graph is a set of objects that mean something. The big insight of GraphQL is instead of looking at your APIs as a set of separate endpoints that just returned JSON, you can look at all those APIs together as a network of objects.

30:53And the properties of each object, maybe that's a product object or a review object or a shopping cart object. If you think about an e-commerce example, the properties of those objects are provided by different APIs and they're scattered all over the place. That's just the nature of what happens when you build software. Bringing that all back home and giving the model a precise construction of the meaning of all this stuff has profound impact on what it can do and what can it understand. And so that structured and queriable intent behind the data unlocks better outcomes for AI systems as people experiment.

31:31Because it's just like what you mentioned about you go to a hotel and you have all of these different unique experiences that are all probably their own apis somewhere managed by some team but they all are very different from each other but they work together to create a cohesive experience of i'm at the hotel this is convenient i can get what i want right and it sounds like what you're hinting at is that the layer that we're building now with agents it has that same amount of personalization and customization right it's like you you really want to be able to take all of your systems, all of your APIs, all of your tools and products, and put it into one spot where the LLM can decide with its own reasoning and logic for what it wants, what to pick and choose and call.

32:11And then that again, going back to what you said, is why graph is so powerful for this, because it's going to allow it to access exactly what it wants. And this is actually, it sounds like going to unlock scaling. Because going back to what we mentioned, like API consumption is going to go through the roof. We talked about this recently with a guest on the show, Sagar Bachu of Speakeasy. And as I talk with more leaders, it seems to feel the same from them as well. And so if you're building tooling and you're trying to orchestrate this, you know, what does that look like in the future that's different from today?

32:47I think this is a great example where engineering leaders need to rethink some of the conventional wisdom of the last generation because of AI. Like take microservices as an example. So there's been this era of development where we said, okay, the way you scale an organization, the way you organize your teams is you build individual services that can be called independently and composed together. And it goes back to the Jeff Bezos memo at Amazon, right? Yep. Everything's got to be an API and there's no other interface into your system. It's a great model. And part of what made that model work is that it created this really clear responsibility.

33:28Your job as a service owner is a very clear, tightly scoped responsibility. We could talk about SLAs, we could talk about performance metrics, and somebody else, some human or some other team is going to make use of that alongside a bunch of other APIs to create a piece of software. AI changes that a bit because you can't just rely on the AI to do all that work that humans did. That's pretty high level architectural work, right? I mean, AI can write some code that'll call an API for you, but asking it to build a whole system that correctly and always the same way combines 10, 20, 30, 50 API calls and handles all of the stuff around like errors and retry and all the subtlety that goes with that.

34:16That's a bit much. So if you're an API owner, if you're a team leader for an API, one of the things AI is going to do is it's going to force you to think more broadly about how is this all going to fit together? And what do I need to do with my API to make this part of a a working system when a lot of the software is being built with the assistance of AI. I think that's going to force people to take more of a platform view of the world because we're going to need infrastructure at all different levels of the stack. AI is very, very good when you pair it with a semantic declarative piece of infrastructure.

34:59Lots of people are using AI to help write React components, for example. There's another declarative architecture. But I don't think anybody's seriously talking about dumping React and just asking an LLM to automatically fit all the components together, right? That's the job of the infrastructure. The pressure to combine stuff in new ways and the fact that the AI is pretty good at parts of that and pretty turned around with other parts of that, I think forces a lot of us that are tech leaders to really zoom out and view the whole picture. The nature of LLMs being probabilistic, right? Going back to what you just said about, you can't necessarily rely on them to make all of the calls correctly in the right way, handle all the errors, because there can be lots of deviations.

35:42And when you're in the world of APIs, like you don't want to deviate from what the API is giving you. And so, yeah. And even how they combine, like we shipped an MCP server today for Apollo. This allows you to use AI to talk to your graph and I wrote a blog post and I talk about this. You know, imagine a bank. You don't want one of your customers, if you're asking the AI agent for your recent transactions, you don't want one of your customers to get back five and the next one to get back eight. And you don't want, you know, sometimes it includes like the remaining balance along with the transaction and sometimes it doesn't.

36:18You know what I mean? Like it's more than just how to combine the APIs. It's all these decisions that we used to make in the code about exactly what we want, how we want it presented, exactly what combinations, who's allowed to see what, that is so important. Because at the end of the day, this all comes back to like, we're making software for people and we want people to have a great experience. And it's really important to hit that note. I don't think there's a single Apollo user that I've talked to that isn't building an agent, isn't thinking about what these experiences are going to be like.

36:50But the flip side of it is, all that stuff is still on the way to production. we're really early in actually rolling this stuff out. And the reason is because getting this stuff right is part of what has to happen if you're serious about putting that kind of software in front of people. Yeah, I mean, if you're going to ship these kinds of experiences that are so sophisticated with an LLM, you need to have insight into how they work. You need to be able to reliably understand their outcomes. And it goes back to like LLMs, they're pairing well with strong semantic infrastructure and orchestration, things that help keep them on the rails.

37:27Like the LLM can be the creative kind of composer of all these different tools and pieces, but it needs to be guided by a rigid system that tells it yes, no, and keeps it in line. And so it sounds like that pairs really, really nicely with semantic data that you get from a graph. It feels like a natural ingredient within that mix. Yeah, it's a left brain, right brain thing. and they work very well together. Yeah. And so as these things get more sophisticated and we have more ability to get nuanced information, make nuanced requests in a chat or otherwise using a tool with natural language, it kind of starts to open up a world where you can kind of solve everything with a conversation.

38:15Do you think that we might evolve into a place where most applications and ways of going about doing things on a computer reside within a kind of like a chat type of interface? Or do you think that chat is limiting and that there's maybe an additional way that this is all going to be evolving? I think we're going to find out when mobile came out. I mean, in some sense, a mobile app is just a computer app on a smaller screen, right? But it turns out multi-touch was important. And some of the user interface things that work really well on a desktop don't work at all on mobile and vice versa. I think mobile gave rise to a world of map-based experiences, for example.

38:57It's not like we didn't have maps on the desktop, but it's very different when it's in your hand. I think that's going to play out with AI. We know some of the answers. We know there's going to be this natural language back and forth. I suspect a lot of that's going to be spoken. but we also know these generative systems are really good at building graphics so i think we might find some of this stuff turns the application inside out we'll still have ui elements and a lot of what's on the screen today but it might be all sort of surrounded by a much more free form interface i think one of the really neat things about ai is that it's so flexible and and it's like personalization turned up to 11, right?

39:42Yeah. Everybody, if you go to your banking application today, you and me get the exact same app. That's just how it has to be. But maybe it turns out you prefer a different way of seeing your accounts and all the things you can do there than I do. And I wouldn't be surprised at all if we get to the point where the typical banking experience is one that learns from your preferences and adapts to you. But underneath it, to your point a minute ago, there's a structure and a consistency. So we're not saying different things when we talk about the meaning of a wire transfer or a deposit into your bank account.

40:21I think we all want that stuff to be common, but there's so much room on top of that to adapt to customers and to what they want. And I think that's going to attract people. I think people want that kind of an experience when they're using software. Yeah, it's just gonna like be wearing different clothes, like just the fit specifically what you're looking for. We already see some of this in applications where sure, everyone gets the same app. But maybe when you land on the app, you get like a different set of personalized topics or features or assets or you customize your home on the app, right?

40:53So it's like, there's already a world where users are invited to customize their software and kind of like collaborate with it. And now we're going to turn it up to 11, like you said. I really like that as a way of describing it, because it's going beyond even the bounds of what we thought that this stuff could do. And your prediction, too, it reminds me of one from past guests. He's been on the show a few times, Rob Zuber of Circle CI. He talked about how he made a prediction at an event that DevEntrept had hosted back in December about how maybe in the future, when you make a website request or you try to query that data instead of getting a website and then the website's data is hydrated for you on demand, the website itself could be created on the floor to fit what you're looking for.

41:38So do you feel the same way? Like that might be a future? It'll be something like that. I'm sure we've got all the exact details wrong. I mean, it's telling the future is so hard, but personalization is definitely part of it. I think AI is probably the next big push to elevate not just the amount of API calls that you were getting at before, but just think about how important APIs become. Because APIs are the capabilities of a business. It's the stuff that a company or a system can do. And Uber, we all use the Uber app, but maybe the app gets a lot less important. And there's already hints to this, right?

42:16You can call an Uber from Apple Maps, or maybe it's Lyft, or I don't know. There's all kinds of car sharing and map partnerships out there, right, as an example. But the real value of Uber in a real sense is the network of drivers and all of the machinery that knows how to schedule, knows how to price. That's the actual asset, I think. So it's companies that have something like that and can expose it in an agent-friendly format. That's just a fancy way of saying good APIs, I think. That might have profound impact on companies that we think of as really valuable or that have a really important experience.

43:00And I think there's going to be a race in a lot of different industries to go in that direction. You probably won't be able to use your user-facing application surface area as your calling card in the future. Right. It becomes, you know, having that killer app that has like all the good experience in it. It's still valuable, but it's maybe not like the clincher that it would have been in the past. Because now it's about the asset of, you know, what is the service or the data that you're providing as part of your company? you want to consume. With Uber, like you said, it's the scheduling, it's the availability of the drivers, it's the networking of all of those through their APIs.

43:40And so in a world, there's like a world where you would call your Uber from like a conversation or another app entirely. And so you get almost like a commoditization of the services that people are providing. And so if you're an engineering leader right now, like a lot of our listeners are, and maybe you already have one of these very valuable types of data sets or APIs or orchestration that you're sitting on top of that you that you ship and put out through an app or a website or a SaaS whatever whatever it may be what are some steps that you think that that leader should be taking today to make sure that that asset is unlocked tomorrow well I think they should go to apollo.dev and uh I mean I so it all starts with the graph for sure but yeah I do think platforms are going to be really important.

44:32So I think every engineering leader should have a really clear platform strategy in terms of what's my stack, what are the components of that stack, and in the areas where I'm maybe relying on bespoke software that we build ourselves, maybe look for a standards-based, platform-based alternative to that. I feel pretty good about that prediction because we know that AI-generated software, when constrained... I mean, a good platform basically provides guardrails and rules and laws of physics, right? So when you have that, we know that the AI can go very, very fast. My guess is that humans aren't going to be writing much of that software in just a few years, right?

45:19It'll just be the AI and maybe there'll be another AI that does your code review and a third AI that does your, you know, CICD sequencing. And I'm sure there's a human somewhere in that picture, but it's going to start to get pretty abstract. But I think there's going to be this growing need for a higher level understanding of the overall system and the architecture. And that's exciting. I mean, I think like we don't write assembly code anymore, right? This whole industry is just a series of increasingly powerful abstractions. To use them well, you have to understand the layer underneath that abstraction, right?

45:56You weren't a good C programmer if you didn't understand assembly, but you didn't have to write assembly anymore. You could say the same thing about other modern programming languages. I think the same thing's going to be true about how all this stuff gets glued together. You'll have to understand Kubernetes, but you probably won't have to write operators too much anymore. You'll have to understand how Graph works. I think that will be the orchestration layer for APIs. But you probably won't be writing GraphQL schemas by hand or queries by hand. You'll have to understand how React works, but I think the days of handwriting React components are probably not much longer for us, right?

46:30Right. That's my guess. And I think you can work backwards from that. We find at Apollo just internally, we've really leaned into the culture around writing and documentation because I think it's a good way to do that kind of higher level thinking and design work that you need to do. I just think there's a lot of interesting implications for how you build an engineering team, what skills you emphasize, what skills are maybe a lot less critical than they were before. So strong communication skills, strong understanding of the engineering fundamentals, those are the things that are more important in doing things like actually writing the code takes more of a backseat because of what you described, this future where LLMs are going to be doing a lot of that process.

47:15Even thinking about it just from the point of view of how you talk to a, if you're using cursor or LOD or any of these things today, right? What I find is if I know what I want and I explain it clearly, I get good results. So those are the two skills, right? You have to know what you want, which means you have to have a certain just higher level knowledge of the broader landscape and you have to be able to explain it well. And it's just interesting. These just, they're different skills than, you know, certainly, you know, we emphasized a generation ago. And that shouldn't be a big surprise to us, right?

47:51The technologies change and the skill set changes with it. But that's the kind of thing we're thinking a lot about. And it's exciting because I think it opens the door for a lot of really strong talent to do this kind of work that maybe didn't have some of the more, you know, domain specific knowledge of React or, you know, whatever else it may be and didn't feel like they could build the software today. Yeah, I really identify with that myself because I have a humanities background before I moved into DevRel, which was kind of like my bridge into tech. So I used to be a teacher. I studied like classical languages, right?

48:25And so it doesn't have really have anything to do with programming. But as I've gotten more, you know, I've learned to code. I was in web dev for a while. And as I've moved more into like developer marketing, now I see this world where these communication skills that I spent so long building and using it to understand the underlying concepts of something and communicate what I want. Like those are becoming even more important. And these are soft skills that engineers have, you know, maybe in the past not prioritized as much, but now become so much more important. And it opens the door for a lot of people to then develop their own software.

48:58We've talked about this in the past about, you know, you'll get the rise of like personalized software, disposable software that you just make once to do something or for yourself or for a project. And, you know, we're seeing this really fast right now on the web and with web dev. We talked with, we talked with Vercel, Lee Robinson over there about this because, you know, nothing moves faster than the web. And so they're right now kind of seeing this where people make stuff on demand and people from all sorts of non-technical backgrounds are able to get in there and make something. But at the end of the day, they have to know what they want to make.

49:32And so having that higher level understanding of what is my goal in going on cursor and asking it to build something. It kind of contrasts the whole thing going on right now where everyone's talking about vibe coding and everyone has a different reaction to it. Some people have a very negative reaction to it. And some people think that it's like an accessibility thing for many people to be able to get into programming. But at the core of all of those conversations, everyone always arrives at this same point of you need to know what you want and you need to know when the LLM got it right or wrong.

50:04You need to have that understanding. What's your take on that right now. Yeah, I agree. I think the other big gap, and it's just an age-old story, is there's a huge difference between prototype and production. And I think that was true long before VibeCoding, right? Like, lots of things can get built in a sprint that, for real, take the better part of a year. And AI is going to accelerate all of those constants, but I don't think it's going to change the fundamentals where the software we actually work with every day as people, the stuff on our phone or the stuff that we use when we log in for the morning, there's so much more to it than like what was on the screen and getting a first version of that to work.

50:50Yeah. And I think sometimes that doesn't get the attention that the flashy stuff gets, but it's no less critical. And I'm sure we'll see, this seems like a pretty safe prediction. I'm sure we'll see a lot of energy going into that part of the development life cycle in terms of where AI can help and what ought to change about the stack. But I know for sure that the typical team that we talk to has a mandate to ship. They are being asked to prioritize AI. They've got some instincts for what the experience ought to be, but there's a lot of open questions. It's not just the AI API orchestration problem, but there's a lot of questions about what the rest of that stack is going to look like.

51:36If you're into dev tools or infrastructure, it's a very exciting time because all this stuff is going to get sorted, I think, over the next year or two. And the thing I hear over and over again is you can't just YOLO an AI experience and not worry about the part where it, you know, leaks sensitive information or just think of all the things that could go wrong, right? And people are well aware of that. So there's still a lot of work ahead of us, I think, for all of us to get that stuff squared away. Yeah, we're hinting at this world where the AIs are writing the code, they're testing it, they're reviewing the PRs, and now they're also at the heart of accessing your data, understanding the structure of your data.

52:20You know, what kind of prevents this from becoming the Wild West? How do we really keep a handle on it as engineers and understand what's going on when all of these systems are interacting? Yeah, I mean, I fall back on system design for a lot of this, right? If you think about the overall stack, The LLM is just one part of it. And if you think about the responsibility, the role of each piece of the stack, and if you think about the sort of inherent limitations or safeguards that the boundaries between those things create, that's part of the answer, right? So like, I may not understand exactly what the LLM is doing, but I know that if I give it a very specific set of MCP tools and I understand what those tools do, that creates some amount of safety, right?

53:02I don't think the AI is gonna reach around MCP in the near future and go get what it wants directly out of the database. Maybe there's a dystopia that's coming for us here, but I think the data is safe for the moment. And so it's stuff like that that I think covers the bases around a lot of this. I think analytics and measurement is another huge unanswered question, right? What's the right way to measure the quality of a agentic experience. It's going to be a lot different from how we think about CSAT and software. And my guess is that a lot of that will get fed back through some kind of an offline AI process.

53:45Like one of the things we're thinking about is you don't want to give AI unfettered access to all your APIs. That's just a non-starter. So the way that we approach this and what we talk about today with our MCP tooling is that you use the graph infrastructure to create a set of specific MCP tools, right? Maybe there's a tool for adding something to your shopping cart. And under the hood, that might call five or 10 different APIs, but the way those are combined is done on the graph layer. So the AI is not making something up here. But the question becomes, how do we know what tools to build? And how do we know that when we put a customer in front of the agent, they're getting a good experience?

54:24Because you've probably had the experience where you called a human agent for something and they were like, I can't really help you. okay yeah so i i think there's going to be a whole flow around feeding the whole interaction through a different set of processes with a human in the loop maybe this is what a modern product management job looks like or even a development job it's hard to say right i think there's going to be some new job titles here but there's a there's a process where we learn from the interactions in real time, turn that right around into sipping a new capability for a customer.

55:01And it's interesting. This is what I was getting at before. If I opened my app on my phone, say my app for my bank, and it was different every day, I don't think anybody really wants that. But agents can get better every day and it's just good. It just means they're more capable. It means they make fewer mistakes. It means I have to fall back to the handwritten software or maybe an interaction that involves a human less often, which means I get it done faster, right? Like that's just net good. And so I think that we've removed some of the natural bottleneck on how fast this stuff can improve. And that's where I think there's going to be a really rich vein of AI-enabled products and technology that help us do that well.

55:47I completely agree. I think there's so much opportunity that we're right on the cusp of, of taking advantage of. And you've kind of outlined in this conversation a lot of really great ways that teams can get started with it and some really salient predictions, I think, about where all of this is going. And for me, you know, this has been like a really fun chat, Matt. You know, you have such good insight on what's evolving in the industry right now. But before we start wrapping up, where can our audience go to learn more about Apollo and the work y 'all are doing? You mentioned you recently dropped an MCP server.

56:17Yeah, we're at Apollo.dev, so you can read about the underlying graph infrastructure as well as what it means to use that with an agent. And the MCP stuff is cool, I got to tell you. Like, if you haven't had the experience yet, you know, one of the reasons GraphQL flourished is that it's a joy. It's just, it's really fun to write queries. it's the tools are solid but there's something even deeper than that it's just like being able to to explore your apis in this way and and navigate that stuff i think any developer that's had that experience knows what i'm talking about there's a new one now which is watching an llm crawl over your graph and write a query it's just it's amazing and this stuff's really easy to do the knock on GraphQL for a long time is that it's incredibly powerful, but it's hard to get started with because you have to convert APIs into this GraphQL format before you can take advantage of all the stuff we've been talking about.

57:19That's not true anymore. You can bring any REST API to this world. You just write some configuration. So I think for any developer, just trying this stuff out on your own API is worth the half hour and it'll give you something interesting to think about. I think that's a fair promise. And then, you know, look, I'll just go back to what we started with. I think I've been doing this a long time. I've just never seen so much energy. I've never seen it across the developer landscape. I've never seen it across the, you know, C-suite in terms of the urgency of doing AI stuff. Like, what a time. And I don't know where this is going.

57:58I mean, I said some stuff. I think most of it's probably directionally, right? But the only way to really know and to find out is to get your hands dirty and shape the world. That's why I do this stuff. And I just think it's going to be a really interesting year to come. I couldn't agree more. The joy of experimenting right now and being in the space, it's a really unique time to be exploring what we can all build. And I know after this conversation, I'm going to go check out what y 'all are doing. There's been some really interesting tidbits about how teams can take advantage of this now. And so we're going to make sure we include this stuff in our show notes so that, you know, our listeners can go and check it out as well.

58:34And to you, our listener, if you've made it this far to the end of the conversation, then you clearly loved what we talked about today. Be sure to give it a subscribe if you haven't already on wherever you're listening to the podcast, maybe even give us a like or a rating. But if you're not reading our sub stack, which I mentioned every week, definitely go and check that out as well, because we're going to be dropping Matt's interview there, along with the links and some news related to this topic, because we cover things every Tuesday on our Substack. So be sure to check us out. And thanks for joining us today on today's Dev Interrupted.

59:05We'll see you next time.