Goblins in prod, the messy middle of AI adoption, and everything is a harness now

0:04Andrew:I don't know, Andrew, are you getting a Codex pet? I feel like it's only a matter of time until we have NFT-based identities for all of our bots. Oh, no.

0:14Ben:That is not the direction I want this thing at all. I actually think it's funny to see things like Codex pets. For those who haven't seen it, it's a plugin for Codex that lets you add a terminal companion or a terminal friend as you're using the tool. You know, we've obviously been seeing this before. CloudCode had buddy mode a few weeks ago or a month ago. I know you were lamenting the loss of yours. That was devastating.

0:39Andrew:Yeah, yeah, poor Trixel. I'm sure you're looking for a new pet.

0:43Ben:Trixel's time in this world was too short because the CloudCode buddy feature only lasted about 10 sweet days. And so it's funny to then see the Codex folks over there and like, oh, you know, this is a grass is greener kind of moment where I'm like, oh, they're having fun over there with their pets. But you know what? I'm actually not really leaning into the whole trying to make it a personality or trying to have it as this like thing that I talk with a lot of times that my sessions are, you know, very ephemeral and they're anchored in durable sources of context and information. But like the sessions and agents themselves don't really glom much of an identity.

1:20Ben:Like even in the open claw world, when that really hit the scene, there were parts of that that I stole and wanted to use for my own harness. And I certainly did. But, you know, the growing personality over time just wasn't really one of them.

1:33Andrew:Well, that's not you're not entirely true. Your agents do ask you to build them a profile picture at the very least. It seems like that is true.

1:43Ben:That is true. That is because all of my agents are represented as a little fish with a cowboy hat. For those that aren't in the savvy know, I post about them on LinkedIn because they certainly all have the same kind of shape and face. But actually, that is just because I have a really simple template that I go to Nana Banana and I make them a little fish with a hat. It's zero cognitive burden for me. I don't have to be like, oh, what's your name? What are you? I'm not trying to put any kind of face on them, but it doesn't mean that I can't have fun with it. So yeah, I'll go slap a fish with a cowboy hat on that.

2:15Andrew:That's the kind of learnings you get here at the Friday Deploy, brought to you by LinearB. I'm your host, Ben Lloyd Pearson. And I'm your host, Andrew Ziegler. Yeah, and this week, beyond Codex Pets, we have OpenAI's Goblin Invasion, Organizational AI Learning Crisis, Spex Maxing, wow, and AI Productivity Myths. Tongue twisters aside, where are we going to start, Andrew?

2:40Ben:Let's talk about the story where goblins are overrunning our chats. So this is a really fun post-mortem that came across our desk in the last week. If you've been on social media, you've probably seen fun prompts and outputs folks have shared from their usage of chat GPT, where it becomes obsessive in mentioning goblins, gremlins, and other creatures in its responses, rendering them in places where they don't belong. and Goblin usage and Goblin directed conversations spiking over 175%, which is an amazing thought to think that there's a dashboard or a metric inside of OpenAI that's tracking Goblin.

3:17Ben:And it forced a lot of folks to add explicit anti-Goblin instructions, literally like a barricade on the town walls to keep Goblins out of their outputs. So, you know, where did this come from? This is a really interesting dive into the reality of how LLMs are trained and where they get their performance from. So the problem on this originated from a personality training for a, quote, nerdy chat GPT preset. Where, you know, things like a creature references to a bestiary might come up every once in a while. It's the idea of having a personality on Claude where you'd probably be playing Dungeons and Dragons in a basement with it.

3:57Ben:So this output from this particular personality training ended up spreading to other models because it's a pretty standard industry practice to use model outputs to train future models. So all it took was the existence of this nerdy chat GPT preset somewhere deep in the training architecture of the models to slowly be pumping out goblin-obsessed outputs and otherwise playing an infinite game of D &D in some virtual basement somewhere. And it really just dramatically polluted the output downstream for folk. So it really reminds us that this is a really big Ouroboros, right? It's eating its own tail in terms of the kind of like performance we're getting.

4:38Ben:And it really calls to mind the importance of provenance and understanding the data that goes into your model. What did you learn from this one, Ben?

4:46Andrew:Yeah, well, the biggest thing that I'm coming away from this is just wondering like how many other goblins, so to speak, are out there. Like these things that have leaked into the core training models that have created an unknown tendency that impacts everything we do with them. So, and I think it's a really great representation as well of intention drift. So a minor shift to word structure can sort of cascade into massive downstream differences. The more that you iterate on this, you know, we've seen this all the time, quite frequently, where we introduce like a minor thought into our agent harness.

5:24Andrew:And maybe it implies a little more intention than we intended. Excuse the redundancy on that a little bit. And retraining AI models on past iterations is effectively, you said Boroboros, but it's like a game of agentic telephone as well. You know, every iteration is slightly different than the one before it. And so the final outcome is sometimes is nowhere near what it started what you expected it to be. And this thing, you know, happens at the micro scale as well, when you're working with an agent harness, you know, a minor idea could be injected early, and that could morph into some sort of strategic imperative within the agents like mind.

6:04Andrew:So, you know, for example, consider the line like, it would be nice to build feature x, but we would need to build a new API integration for that feature. So you would add that to like roadmap along with the API integration. Later, your agents are working on it. They come across the part that says that you need this API integration, but they miss the context that it's only in the context of this future feature that we're not building at this moment. And it could mistake that and think that this API integration is now a critical component of what it needs to do. And, you know, and this happens at scale when you're dealing with agent orchestrators, you know, that drift is the constant battle that you have to work against, whether it's goblins or whether it's agents that violate your testing policies or do any other number of things.

6:58Andrew:So, you know, AI loves over-indexing towards things that seem really important. And yeah, I can understand this goblin channel very deeply myself.

7:09Ben:Yeah, it's really good advice, like what you said, to go back and revisit your harness and your prompts and your skills and make sure you're not over rotating on things that are polluting downstream context. If you find yourself iterating a lot or making the same kinds of revisions on outputs that you previously had crystallized as like a skill or some kind of process, I think that's a good reminder to go and revisit it because there might be goblins lurking in that prompt.

7:34Andrew:All right, Andrew, let's move on to AI deleting databases. Is it the AI that did it? Is it you that did it? Who's responsible for that, Andrew?

7:44Ben:Okay, well, another week and another tragic incident of a production database or some sort of production environment getting totally wiped out or sideswiped by a rogue AI agent or a bad endpoint or a combination of the two. and really this is just once again your weekly reminder that ultimately the permissions and the scopes and the hooks and the protections around your agent are your responsibility and understanding the power and the leverage that you can have with your agent and the responsibility that comes with it is really crucial when you start entering into interacting with a lot of systems at scale like Even going back to the whole goblin problem we talked about a moment ago, imagine this drift now on top of something that is working with your production data, with your customer data, with your product data.

8:37Ben:It becomes more than just, oh, it's annoying, a goblin is popping up in outputs I don't like. It can become catastrophic when you combine that with tool calling. And so it really kind of highlights the danger of not having those guardrails and reminding us that it's a system-based thinking. And part of that is closing the environment. What is the world that my agent has access to? And understanding that really, really deeply before you start to put tools in its hand. And it really just is a strong reminder that like if you're working in production environments and you're working with agents to make sure that you have these structured layers and these protections in place to protect them from doing harm to data that might be irrecoverable or cause downtime or interruptions for your users.

9:26Andrew:You know, Andrew, you and I have spoken to enough people firsthand that flew a bit too close to the sun with their agent orchestrators and have come away with scary stories that we should all learn from, you know, in addition to all these ones that are getting public attention. And I feel like I'm actually kind of getting to the point now where I'm starting to feel almost like a safety officer. Like when I hear about like coworkers that are like, I just discovered this new agentic workflow. And I'm like, okay, are you being careful about all the ways I can go wild? And so, you know, but the technology is just so exciting that you want to explore like that because it's fascinating.

10:07Andrew:fascinating. But, you know, there was a line in this article that really stuck out to me about how we use terms like thinking and reasoning when we're talking about these AI systems. But really, those are just like marketing terms that have been put on top of them. The reality is that they're just coming up with structured ways to generate tokens that simulate those, those like thinking and reasoning. So we have to keep remember constantly that AI is a simulation of our ability to think and reason. And I'm really like 100 % aligned with this author. Like we need to be responsible for our own agents.

10:46Andrew:You know, and I think it also shows really just the importance of blameless culture. You know, failures happen all the time. The important thing is that you respond to them and make things more robust for the future. And so, yeah, and I'm going to get back on my soapbox about agent permissions for a moment. And, you know, I kind of do think that these AI companies, the vendors are, they're putting some unreasonable expectations on users about how to use them safely. You know, if your AI is willing to go to the end of the world to solve a problem that it thinks it's responsible for solving, it's really hard to prevent it from going rogue.

11:22Andrew:Yes, we need to have safe practices by default, but what was safe in a human driven world is no longer safe in an AI driven world. and mistakes are now happening in the blink of an eye and just like just unraveling. And, you know, this is where we really still need determinism, like determinism plays an even more important role today. Absolutely. We all need to have checks in place to prevent those rogue AI agents. Like, you know, at the very simplest level, we have AI, we have hard caps on API usage for our agents in the case that they decide to go spend a year's worth of budget in an afternoon.

11:58Andrew:But you need to have that same sort of protection at every single stage of your SDLC. Like you have to have that awareness and that check in place. You know, at Linear B, like we've been focused for a long time specifically on code reviews because they are, you know, one of the most important quality checks. And they're the single most important or the single most frequent bottleneck in most engineering teams. And, you know, we spend a lot of time just like helping teams like eliminate hours of toil around code review specifically, you know, and the goal is really always just to get developers back to more high impact work and to make sure that they can work quickly without, you know, creating additional problems and headaches for their team.

12:38Andrew:So that's a great place for to implement a lot of this determinism to protect your organization. Like, you know, CICD is kind of old news at this point. Like it feels like this. Gass, scandalous statement. But it's as relative as ever, you know, and it's kind of more fun. It's more foundational than ever before. Like you can't expect human attention on every single detail of the code that's entering your code base at this point. So, you know, these failures, like they need to be caught early. They need to be contained. You need to have rollback procedures in place for when it's necessary. And then you use failures to strengthen your automation.

13:16Andrew:And if the opposite is happening to your organization right now, like failures are cascading, they're becoming more frequent, they're impacting your ability to deliver. That's where you need that visibility to understand what's happening so you can fix those problems before going to broader AI adoption. So, you know, my advice is focus your tokens on solving the process first, and then you can go and solve that like AI adoption problem. Absolutely. All right, Andrew, let's talk about when everyone has AI, the company still learns nothing. I really love this article, but why don't you summarize what was in it for us?

13:54Ben:Okay, so this is a fun article. It really cuts to the heart of a problem that's happening in a lot of organizations. And it talks about how individual AI productivity gains don't automatically or even realistically sometimes translate to organizational benefits. And a lot of companies are now entering this messy middle phase where the AI use is certainly widespread. It's undeniably there, but it's uneven, it's hidden, it's disconnected and siloed. You focus on all different parts of the AI adoption journey, and their ability to communicate and collaborate with each other is pretty staggeringly hampered by the differences in their outputs and what they need of each other right now.

14:34Ben:And really the organizational work needed to understand how to connect these different types of learners and people of agentic ability with their outputs and their organizational goals is so early days that we don't really even have the language as teams to talk about how to share and distribute, how change and process management has to evolve. And just like the new reality of how fast some folks can work and how some other types of roles are stuck in a more traditional throughput. So because of this, you might be in a familiar story, especially like if you listen to this podcast, you're probably a little more on the advanced side of your agentic journey.

15:15Ben:And you probably have, in some cases, encountered where you meet folks that are on another end of that journey. And communicating and collaborating and getting on a shared baseline can be really hard. So this article outlines the three new capabilities that we all need to be thinking about on like a meta and organizational level. And this is about how to like orchestrate and operate agents as an organizational scale, but also things like understanding the intelligence of a loop, like what makes an AI workflow effective, unscalable, and what are the capabilities of the things inside. This kind of literacy around AI is really critical and it's applied, right?

15:56Ben:It can only come from having lived in the experience and being able to communicate it on your own company's terms. So it also even veers against some of the token maxing stuff that we've talked about recently. Some organizations, when challenged with this problem, what they threw at it was what we talked about last week. And those were the Goodhart's laws to the extreme token maxing boards that were tracking all of the engineering teams and their token usage. And then when you peeled back the hood or when these engineers, these companies revolted and threw back the, you know, the tracking, you really started to peel back then the learnings and realize that you weren't really understanding anything about your organization's impact or what you were shipping or how to get this one guy over here who has like, you know, a thousand agents running.

16:41Ben:How do I distribute that to everyone else? How does everyone even understand what's going on over there? Like none of that was elucidating. So really, there's a competitive advantage available for folks here. So this is the leadership takeaway, if you're listening to this, about how can you shift on an organizational level the ability for people of different AI competencies to interact with each other, to share their gains, but then also just find a new collaborative normal. And accepting the fact that not everyone's going to be some super agentic 30 ,000 terminals open. You're in the minority report, and you've got the glove on, and you're throwing things across the screen.

17:16Ben:I don't think anyone's expecting that of everybody. But there is a baseline expectation for all of us to be curious and to throw away assumptions of yesterday and to find our new place in this matrix.

17:27Andrew:Yeah, even if you could operate with that matrix machine, you know, you're not always operating with that at that level. You know, it really depends heavily on the type of task and work that you are focused on. But I really love articles like this that just sort of paint a picture of the moment in time that we're all experiencing right now because, you know, this is a very emergent, just foundational skill. Like working with AI is not like we've all purchased a new tool and now like have to like use it or something like that. This is actually like skills that we have to like almost retrain ourselves on how to operate within our day-to-day lives.

18:08Andrew:and the messy middle is it I just love that that phrasing because it really does feel like we're going what I'm experiencing right now and what I'm seeing you know because it does feel like we're kind of transitioning to this super awkward phase of AI adoption but there was a quote that that I really love that it focused on the key thing that many organizations don't understand about measuring AI adoption. And that is that AI collaboration stretches from tight synchronous co-driving to looser asynchronous delegation. So the point there is that too many companies are focused on whether or not people are using AI.

18:46Andrew:Like they're just looking at like, how many times did you accept co-pilot's suggestion? Or did you put AI code into your poll request this week? But you should really be more focused on whether or not your teams know which loop to use and where they need to put resistance into the machine, but also like which artifacts survive the loop and then how those artifacts become something that the organization can learn from. And we're seeing a lot of this on our own engineering team. You know, someone discovers a loop that allows them to consistently achieve like high quality output. And sometimes it can be translated into broader organizational improvements.

19:24Andrew:Other times it gets used for a a single project and then basically just scrapped and thrown away. And then we just take the knowledge that we gain from that into the next one. There's just such a wide range of how our experience with AI is unfolding right now. And I do want to highlight the three capabilities. The top one that was listed is that to help navigate this messy middle is agent operations, and specifically things like which agents are allowed to run. What systems can they touch? What data can they see? What actions require explicit human approval? You know, all the things that we've solved for the human workplace, but not for the agentic workplace.

20:05Andrew:And then there's just the bigger question of like, who even owns this? Like, you know, the cloud era led to like platform engineering becoming a thing. And, you know, we've seen that sort of morph into like developer experience. And now we're seeing like a lot of terms like AI enablement, AI innovation turning up within organizations. There's really no established precedent for how to operate AI within an organization at scale. And anyone who's doing it is basically building that knowledge from scratch. So, yeah. And, you know, personally, that's where I love where we're at right now. Like, you know, both with Dev Interrupted and Linear B, we're always working with those people.

20:42Andrew:And we really have as a part of our core mission is to help that audience really navigate the uncertainties of this. So yeah, go check this article out because I think everyone here will find a lot to empathize with.

20:57Ben:For sure. And I think it pairs really well with our next article too, which talks about the usage of Cloud Code and whether or not it's making your product better. And this analyzes a very similar reality of the world that we live in right now, this messy middle. And that is of this K-shaped productivity curve. The idea that senior engineers are showing measurable output gains and throughput by using agents. And they're able to kind of up-level their abilities in a very new way. Meanwhile, on the other end of the curve, trending down, you get experiences where engineers with less experience or with less domain expertise are flattening or declining in their productivity.