Multi-agent orchestration in Slack | Saleforce's Kurtis Kemple

0:05Today, I am thrilled to welcome our guest, Curtis Kempel, the Senior Director of DevRelrel at Slack. Curtis, welcome to Dev Interrupted. Thank you so much for having me. It is a pleasure to be here. We're really excited to have you here. You and I, we met at Dreamforce last year. And when we met, I knew I had to have you on the show to pick your brain because we chatted for a while about some pretty cool concepts. One of them really stuck with me around the future of work. And I really want to dive into that with you today. Because when we chatted at Dreamforce, You shared this idea that really stuck with me about how Slack is evolving from a place where work is discussed to where the work is actually done, as if like those words are starting to move into action.

0:51And I think that's really interesting to explore. It makes me think of like supporting practices in the code world, like DevOps. You know, maybe we're entering a world where you get something like chat ops. And so this is kind of part of the future of work that Slack is taking us and everyone who uses Slack, which is a lot of folks, into the future. And I want to dive into that vision with you and talk about how those core problems have evolved. So, you know, what do you think about that premise? Do you want to dive into that today? I absolutely do. I absolutely do. And, you know, before we hop directly into that future, I just want to take one second to talk about the past and how we got here.

1:29Because, you know, Salesforce has really created the push for agent force and agent interaction into Slack. That was our real first approach, right? Like we dropped it in there. We learned a lot. And through that process, we started to understand and develop, like, what does it take to support, like, that type of experience, getting agents directly in first through there, but now through anywhere, right? Like third party directly into the Slack platform or first party customers building their own agents and integrating. And so we just hit, like, kind of that, you know, perfect storm or shelling point, if you will, right?

2:11And essentially, it made us really just stop for a second and put on a beginner's mindset and say, like, what is a platform that supports like any kind of agentic workflow? But does it in a way that is like structured, consistent, grounded? You know, that is a very difficult tension to think through. And so I just want to preface that. And we've been working with a lot of customers to figure this story out, right? Like Anthropic and Vercel have been at the forefront of this. We've got all kinds of companies really just helping us replant another one that stands to mind. Just tons of these across different industries, all noticing and saying like, hey, we can deploy AI here because we've got collaborative environments.

3:01We've got context, which is what we're going to talk about here. And so, yeah. So, sorry, just the main intro. I just wanted to bring us because that's how we started thinking about the future, right? Like, look at where all these things are heading. There are some similarities, some things that are overlapping. And when we think about truly having humans and agents working together and collaborating, like, what does that look like in reality, right? Not even just at the code level, but literally handing off. At the interaction point. I love how you framed it. I'm really excited to dive into this because you're right.

3:38I mean, Slack becomes the place where all of that context lives. And that context is messy. It's the real communications between real people getting their work done. It's not this neat, orderly structured data that can flow in and out of systems. And so it creates this perfect intersection between the systems we're building to be more productive and how and where the work is getting discussed. And it's exciting to think out how all these other companies to see the opportunities with their conversations and want to tap into that to make their own work better. And really, it comes down to this context, right?

4:14It's because context has evolved now into being a first class citizen of the AI world. Before we were all about prompts and prompt engineering, and then it evolved into context and context engineering. And, you know, I can't think of a better source of context for a lot of the things that happen at work than maybe some Slack channels. So, you know, can you expand a little bit on this context gap and how it really is needed to help models perform and meet companies where they want to use it? Yeah, absolutely. So I'm going to walk you through super quick something that I refer to as leaky prompts, right?

4:46when you only own half of the experience, meaning that I can't control what a user prompts, right? And they might start off with a very perfect prompt with what they're trying to accomplish. But literally proven through science, like any conversation, whether that's with something digital, another person, a group of people, will actually slip into chaos unless it is actually managed, like triaged, right? And we see this, actually, you do this right now. When you are interviewing people and you got engaging conversation and we're chatting, that takes effort from you and energy. You are literally putting in a ton of work to ensure that we have this very good, fruitful conversation that stays on track and has important insights and talking points.

5:38So, you know, that work is also required when you're engaging with an LLM. surprisingly enough, right? But the issue is, is we can't control how somebody else is doing. And so it puts us in a place where the only way that we can have the best chance of ensuring that that intent is in alignment, we're staying on task to their goal, is that the context, the representation of what we give to the LLM on the user's behalf is as best a representation of what we can think they're trying to do. You're almost adding like a second order need of understanding. All right. And it's like you have to understand how the user is going to interact with the LLM and ensure that you can just provide the right context.

6:26I like to think of it more as information architecture at this point. And if you can do it well enough, it makes it a lot harder to have those conversations get off track and that misalignment on a tent makes a difference. And like you said, Slack is a wonderful home for that context. We've got threads and channels and messages. And that's where I see the secret sauce at. Totally. And so when you're talking about basically this triage, this harness to keep the conversation on rails, we're at this point where we all acknowledge that AI is very powerful, but it's a force multiplier. It multiplies the good and the bad.

7:07And it's going to make bad situations worse in terms of not having the right kind of prompt, you know, leaky prompts, as you described it. Also not doing your own kind of like data hygiene on what you provide and what you're asking for. Also having clarity on what you're even trying to achieve when you ask it. All of these things are powerful things that the user brings to influence the outcomes and the experience of using the tool. But context, as you say, becomes this experience that the producer of the experience, the provider, the one that's trying to give the end-to-end service can actually use to keep on rails.

7:41And I'm kind of curious to know from you, how does Slack turn the messy reality of all of those conversations into that context harness that keeps users from hurting themselves with their own agentic conversations? So we're approaching it in a couple different ways. And I think number one first is like understanding the needs of app developers, right? And people who want to integrate into the Slack platform because that's actually going to largely inform what type of context we should be exposing to them and at what degree, right? And helping them understand how best to use it through SDKs or APIs.

8:22And so tactically, how that actually manifests is something like, okay, very common need is to do some sort of deep research or deep synthesis of context, right? And that will be broader than a specific channel or thread or something like that. So how do we accommodate that? We build like a real-time search API that is purpose-built to interact with LLMs as opposed to end users, right? And so then you can build these, you know, better search integrations, your perplexities or other things. I'm working on an app right now called Trendy that we might talk about a little bit that does deep research.

9:00Right. And so, you know, these things, you know, require one specific type of context. But then we've got where maybe you're a design team and you're working with your marketing team and you've got a design for a new landing page. So you pull up the Vercel V0 agent and you're working back and forth and it's able to take the context from that thread level and actually go off and generate something for you based off of that. And that's great. But then also, what about the scenario where you've got generally most Slack workspaces have some sort of knowledge or answer, you know, or Q &A channels, right, where you go to look things up.

9:43So you probably want to be able to have an agentic experience at that channel level that's able to tie into related, you know, canvases or lists and the messages within there and help answer questions faster. I can think of about 15 different, you know, verticals or use cases where that becomes immediately applicable. And so, you know, last time when we talked, I'll pause right after this, but it's, you know, it's about having the micro, microscopic and macroscopic and just like finding those right integration points at the platform to enable what it is AI app developers and these AI platforms are wanting to, you know, bring to their users.

10:26And so it's a lot, but yes, it's all context. Everything I said is about like data management or context. And in this world where you're managing and creating this context that produces these more deterministic outcomes, you're ultimately rendering a conversation into a tool somewhere and allowing it to apply actions. Like the Vercel, the V0 one is a really powerful example. People could be having a conversation or dropping a Figma link or crosslinking things in Jira, right? Because these are also places where context lives. And so when you talk about tools being able to grab and use and interact with that data, just like we as humans can, you start talking about a new kind of, it's like an integration layer where human intent and machine ability can meet.

11:15And that's an exciting opportunity for Slack. I think Slack is uniquely positioned to tackle that problem. It sounds like from the way that you frame it, y 'all already are, you know, really like headfirst tackling this problem. I'm wondering how y 'all think about it, too, because Slack is notoriously a multiplayer experience. No one uses Slack by themselves. But AI is relatively single player in terms of how we think about it and use it, right? We maintain our own context windows, our own chats. We have our own silos. We go to chat, GPT, whatnot. not. But, you know, I wonder from your perspective, how does that change and evolve when you start getting multiple people interacting with these bots in a shared communication environment?

11:58Yeah, you know, I really wanted to experience that. And so we've been building it, right? You know, and I even built a full on example of just a little chat app for me and my family that integrates AI just to really experience multi-term collaborative with AI in the flow of that. And, And, you know, I think the only reason we don't see more of it is because I think it's pretty difficult to really build up that user interface, you know. But we've been doing that for a long time. And, you know, I think the sales force to agent force to Slack integration shows up a lot there. And I bring this up because we see companies who are like saving literally like a 4.8 million in annual benefits by offloading stuff that, yeah, agents that they were able to just click and create to help us deal pipelines so that it's doing the intermediate toil triaging, not making decisions, bucketing, categorizing, flowing, deciding where it goes.

13:00Right. And that is completely different. And now we're seeing more and more verticals bringing that. Like you can code apps fully through OpenAI Codex or GitHub Copilot or do both. You're an engineering manager, GitHub Copilot, go dig through all of my top PRs, prioritize them for me. Oh, Codex, now please go through those open top five in a sandbox environment for me so that I can work through them or check them out, right? And then you close them all up. And, you know, the thing for me, too, is that I think about Slack like as a saving use on the productivity tax. Yeah. And so when you're doing AI well in Slack, it's doing the same thing any other app does.

13:45It's me when I vibe whatever and I spin that off. And now I'm off doing something only I can do. Only me that I need to spend my time on. And so I've just like got this like a web of agents around me now, you know, that I'm using. and they're doing everything, like I said, from deep research projects, pulling in my calendar and organizing my day, looking at issues and things that I might need to triage and address. All the stuff that was just like manual labor, nobody else was gonna do, right? Like it falls to me. And then my other favorite place is I love to do it to apply AI to where I otherwise wouldn't have time.

14:26I have 20, 30 minutes for AI. let me see if I can vibe code up a good enough solution for this I end up with nothing it was 30 minutes of my time I end up with a success I now have something that I wouldn't have had anyway because I only had 30 minutes couldn't have done it alone in 30 maybe I'll get a good result if I can sprinkle a little AI in so I just you know when I talk about integrating AI in the flow of work I think I'm like at a point where people are like you know we're talking about like handing off just a Figma here and there. I'm talking about like building the Figma and then handing that off to V0, who's generating the page and Slack bots writing up the canvas for me to go share with the marketing team so we can get ready for the GA and submitting the AI created workflow that lines all of the social media stuff for us.

15:16You know, it's like, Wow, that's like a powerful handoff experience. You're talking about this world where you're effectively chaining these agentic tools together using Slack as the medium. Slack becomes the integration layer because it's the means by which you're communicating with the bots. What do you do when you communicate with any AI tool? You're providing words that are context. Slack then just kind of becomes this place where you're conducting an orchestra, basically, between all of these agents. As the way that you described it, you're orchestrating because you're basically a pipeline, you yourself.

15:49And you are the human in the loop deciding what's the next stage of the pipeline, but you're effectively handing things between the tools. And I think that's a powerful way of working. I also think that opens up a whole new level of, you know, even the examples of like using GitHub Copilot to get your top issues and feeding it into codex. Like that's an immediate, powerful, atomic example. And all of this comes down to how easy it is to build and tinker and explore. You talked about companies deploying their own tools to reduce a lot of toil and sales pipelines, you know, saving them literally millions of dollars.

16:25And those are just easy one-click wins. So, you know, I want to dig into, I think this is a good point to really kind of dig into the how of how Slack is empowering these teams to build these really cool agentic experiences to actually unlock those savings. And, you know, a big vision like that can only work if the platform is easy and delightful to build on. So how does Slack achieve that? How do you educate your devs for all of this complexity? be? You know, that was the first thing is like we had to reassess our entire platform. So a couple months ago, we were looking at the direction of where things are going and where we had spent investment over the last two years.

17:03And a lot of that was into the automation side of the house, which was good. We needed it. Workflow Builder is amazing. It gets you so far. Also, just released a bunch more like conditional, nested conditional branching to customers. So go check that out. If If you're not using Workflow Builder, you should be. But it gets you so far. But when we think of building these AI experiences, they're going to be tying into Slack and all kinds of surfaces, right? Like this app I'm vibe coding now, Trendy, you can DM it, you can pull it up through the agent's sunroof, you can at message it in channel, and no matter what, it's going to help you build that deep research report, right?

17:42And so like, you know, we have to think about, and that's still being narrow. We've got slash commands, message actions, you know, all kinds of events. There's no reason AI can't sit behind any of those events coming through Slack, you know. And as a matter of fact, my next project app I'm building is called Tidy. And it's going to go around and help make sure that your workspace is just all tidied up for you and exactly what you want. Giving you reports, what's happening now. These channels might need to be archived, you know. It will write up canvases and archive it and clean it up, recommend workflows.

18:20And that's just scratching the surface. Yeah, because all of that context also will need its own kind of agent janitor to keep it useful. That is it. That is it. And so, you know, we are also hard at work on making that click to create agent experience, just getting them right in there. Super simple. the vibe coding Slack app experience. You'll be able to vibe code Slack apps like with Heroku and stuff like that and just deploy them right into your workspace. Very seamless. But it all starts with the developer experience. And we've invested a ton over that. And so we did that by consolidating back onto Bolt Apps, which is our main framework.

19:01We consolidated onto the CLI. So you CLI, you know, Slack create, pass it a template if you want or start blank. It's up to you. We've got all the options. You're on Slack run, and now you can run it in a workspace and tinker with it and run Slack deploy and deploy it to your platform of choice. And that's where we are right now in that second half, which I've put under time to value, which is deploying. Like I've got an agent, and it needs to be in Slack, and I need to have it production ready. I want that time like down, like a week, like two weeks. You should be able to databases, observability, fully integrated into surfaces, AI inference happening in a matter of weeks and be submitting to the Slack marketplace to get your app there.

19:49Last note, and I'll say I keep telling folks, stop building apps and start building conversations. Like we already have multi-turn, multi-collaborative UI and AI user experience, purpose built. Why build your whole own website on an app on top of yours? Find product market fit right in Slack. You know, that's what we're building. The infrastructure to support that. Make conversations, interaction point, reduce the overhead on creating those tools. If it's already something that they're going to concierge serve from a conversation style thing, why not just have that live within Slack? It becomes the user experience as well.

20:29And I want to know too, in this world we're describing and we're entering into, you know, doing this agentic coding, it's relatively approachable. We talked about the developer experience and how the developers can spin up their sandbox environment on slack.dev and get started. And they can also use Bolt, your SDK to quickly get an app online and connected, right? So, you know, how does Slack also think about enabling people who are not engineers, but who are also using Slack and would benefit from these workflows to be able to, maybe build and deploy things. Is that what the builder, workflow builder aims to solve?

21:07Or do you see a world for them where maybe they're using these tools as well? I think we're going to see the world. I think any tool that is built to add guardrails, most people, when they, you know, become familiar with them enough, hit the rails. It's almost inevitable. And it doesn't matter how far you move that guardrail, eventually people who are invested enough and have found enough value will hit those rails because they understand the more they integrate, the more value they're receiving, right? And so they'll find new ways that you never thought of to try to do that. So I think my point there is that like, yes, you have to purposefully build what I like to call like learning paths that are essentially are persona based.

21:55And you have to understand that right level of abstraction. And so the kind of the way I like to think of it is first, I teach them about the platform in a bit of a vacuum, not too much of a vacuum so that there's no external context. But really, first, let's familiarize ourself with the platform, its AI offerings, why it might matter to you, what features are available and how you get started. That's like clean. You know what I mean? I can work with that. I can get through that pretty quickly. And then we want to pull out of that vacuum. And now we want ecosystem integration. Because when I'm building something for production, whether I'm a customer or a partner, whoever, I'm never building my app like completely in isolation, right?

22:36We have data management, permissions, security, compliance. You know, the list goes on and on. And so this is actually where we are now. We've consolidated our developer experience to make this easier. We've updated all the initial enablement material for Slack in a vacuum. And where we're pushing with partnerships with Vercel and Replit and Anthropic and OpenAI and all these other amazing AI companies. More and more on the list, too. We're going to be everywhere. I'm very excited. But long story short is, you know, we're really pushing to just like enable all of these different types of experiences to be built directly into Slack and through multiple ways.

23:20Like we even have NCP. If you're working in Anthropic, right, or you're working in ChatGPT, stay there. Right. But now your Slack context, again, is getting to you where you are. And when you jump back over to Slack, you've got the GTP app installed. you're literally picking up almost having the exact same conversation right so yeah so it really fulfills that that that vision of the agentic operating system right it becomes the place where everyone goes to do their work i i want to peel back the curtain a little bit i i selfishly want to get a glimpse at what that is like inside of the slack world how much is slack you know dog fooding these principles around you know making it and just kind of applying ai for all of these experimental scenarios.

24:04You've described your, you're almost a council of tools that you've been building that you've been advising. I imagine they live, you know, in some magical Curtis sandbox somewhere. So what does that look like inside Slack? Like how has your engineering team really evolved to also get in the trenches and build these things? Yeah. So first of all, like we bring it in all the time. So we've got another app called Tiny that's already installed to our workspace that is used by first it will be used by everyone within the slack business unit and then we open it up to the broader salesforce company right and then i'm working on trendy now and we'll do the same thing we'll open that up and let people test it out and build with it and then we'll open it up to the entire company and i'm going to do the same thing with tidy when i build that and that one's going to be deeply embedded into the ai ecosystem like lang chain like creating embeddings and storing that in interstitial data for steps for longer jobs when it's doing all these really interesting things, being able to stop and rewind and redo stuff.

25:13And we've got like the thinking steps and all the support, like actual native user experience to support this coming as well. And lastly, we're looking at even building like an agent SDK and just saying like, let's make it even faster than using Bolt. Like what if, what can we streamline more? We're looking at adding new APIs to make it easier for partners to have the proper permissions to build and deploy these agents and apps on users' behalf, but always, of course, as secure as possible. We take that very seriously. I want to know from your perspective, because you have your role, in your role at Slack, you are very empowered and you're very technical and you're able to envision and build and deliver these things, which is really exciting because you understand what you're trying to achieve.

26:00And as a DevRel professional, I myself really resonate with that being one myself, is that our whole roles revolve around building the context for engineers to do their best work and bringing it all together. What would you say to an engineering leader about how your role has evolved and you have picked up these tools and you're seeing all the success in delivering things? and maybe they haven't coded in a while. They're an engineering manager. They're kind of like dipping their toes back into this world or they're trying to figure out what would you say to them to really get them on the right track and to be building and showing internally in the same kind of way that you are.

26:38I love that. And I'm going to give you all my tips and tricks right now. You know, number one is I'm just going to start with this again. I throw AI at what I call toil and time-dependent tasks that I cannot accomplish otherwise first. I do find places for it in my workflow, but I think the hardest part is getting started. And I think it's when we try to invest too much. Like if I went into a, okay, here's a, you know, like a seven-step program to have you reviewing your PRs today, right? It's gonna overwhelm people and it might not match exactly your workflow. And so I actually say, find the most chaotic, annoying part of your day.

27:24Sit, go through a week, maybe two, and really think about what part of your day is bothering you and invest 30 minutes into researching if there's a way that you can use AI within your system, right, the tools, what tools are available. Do any of those tools, like features, align with my problem space? Yes, let me actually invest 15 minutes in trying it out. And I think that you'll be surprised when you start saving five minutes here, 15 minutes there, 30 minutes here. Then you'll start investing a little bit more. I've got a couple of fun little scripts and prompts that I reuse. Like one of my favorites is it's a strict kind of outline that I give to AI on planning mode when I want to like familiarize myself with a new project.

28:16What dependencies are in use? What patterns are you recognizing? Where are you seeing like overlap? I'm focused on this area. What part of the project should I dive into first? You know, a lot of these type of things that can just source information for you quick and you can verify, I find to be so useful. It's why I'm building Trendy. Doing one-shot deep research projects on a topic and I can just put it to the side and come back to it later, saves me literal hours, hours a day. Deep research is probably one of my most used AI features. In DevRel, I'm in a role where I need to understand what's happening in the industry.