What an AI-Native Product Team Actually Looks Like at Salesforce

0:00You created this new term, ADLC. Yeah, if the software you're producing is not just an internal piece of software, but isn't aging, how do you build it? How do you test it? How do you evaluate it? How do you deploy it? So what I want to do is kind of build people a recipe for becoming more AI native. I feel like at the base, we've covered that tool layer. We are all in Slack all the time. That is really our primary interface on how we work together with each other. The next layer is roles, and you just talked about it, PMs, and you also referred to it with evals. a lot of people are wondering what is the future of the product trio?

0:32The PM, designer and engineer roles, they all seem to be converging on top of each other. The lines that we draw don't really exist. I think we create them in our own heads and then we specialize around them. I think that third layer now is the life cycle that all of this is under. I think the very last component at the very top of the pyramid is probably where you spend a lot of your time. How engineers, product managers, and executives who are really setting the strategy of what we're working on collaborate to build a roadmap. One piece of advice I have always is...

1:24Check out bundle.akashg.com for an entire year of Bolt.new, Airtable, Speechify, Descript, Magic Patterns, Linear, Dovetail, Arise, and Mobbin. And now, into today's show.

1:40Jayesh Govindarajan has spent over 25 years in the tech industry, rising up from being a software engineer to leading a huge 400-person engineering org at Avaya, starting his own four-person startup, which was acquired by Salesforce, and now has worked his way up to leading a 700-person org, which is responsible for Salesforce AI and AgentForce. So he has a ton to teach you about how to build an AI-native company, an AI-native engineering team, and an AI-native product team. Jish, welcome to the podcast. Thank you. It's a pleasure to be here. If someone doesn't know you, can you walk us through what is the org you lead today?

2:23What are they responsible for? What do they do? Sure. So I lead the broader AI engineering org at Salesforce. What that means is we set the strategy direction, build and operate AI systems that span predictive AI, generative AI, and now agentic AI. Basically three flavors of AI that deeply embed into the Salesforce stack, enable our customers to change how they work using predictive AI that's core, generative AI that's core, and now building agents for many, many of our customers. So you've been working on AI and AI products for over 20 years. What does it look like now here at the end of 2026 to be operating like an AI native organization?

3:07That's a great question. I'll color this based on kind of what we're doing at Salesforce, how we're organized, how we're structured. I think there's no better example of an AI-native team than one that is building the foundation of an agentic system that customers used to build their agents on. So broadly, the last two and a half, three years have been an incredible whirlwind of agentic systems as our customers have sought to become an agentic enterprise. And all of that sort of is underlying agent force, which is our harness that we built about two years ago before the name Harness came along.

3:44So from an AI-native perspective, the way the team is structured is designed to be able to build harnesses, to be able to build models, to be able to use these models in context of a harness. And then I'll talk more about this later, this emerging lifecycle of an agentic system that we like to call ADLC, which is how do you build an agent, how do you test an agent, how do you evaluate an agent, how do you observe an agent in production. So these are tools and systems that we build that our customers use to go build their own agents for the scenarios and use cases that they use it for. So how has your organization changed in the last couple years?

4:27We see everybody talking about Cloud Code and Cursor. We see people talking about agentic coding really being the first breakthrough in AI that's really transformed the organization. What has that meant for the roles and types of people in your org? Great question. So I'll take a step back first. We need to color this from the lens of the kind of work people do. So we have teams that basically build and train and tune models. Their world has changed considerably given the kinds of models that they're tuning and training and relying and working, partnering with Frontier Cloud, such as OpenAI, who is a great partner of ours, Anthropic and Gemini.

5:05That world looks a certain way. And then there's a team that is moving really fast and building the harness technologies, which is how context gets engineered into an agent, which is how tool calling happens within an agentic framework. And these are all tools that we build within the agentic system. That team has a certain rhythm of operating with the tools that they have. And then there's the very new and emerging agent development lifecycle, which is how do you build agents? How do you evaluate them in production at scale? How do you observe them? What happens when a long-running agent, a conversation is going off the rails?

5:42How do you correct it? How do you bring all of that sort of system back? That's sort of the three big layers. And if you look at my org chart, that's kind of what it looks like. I have a team that's responsible at the model layer called the AI Platform Team. And there's a team that's above that, which is building harnesses, which is building memory systems for harnesses. And there's a team on top of that, which is the whole ADLC and agent optimization team. And I think all three teams are working very differently than they were before. We are obviously very AI-pilled here. Within our team, we use Cloud, we use Codex for our development processes.

6:20We are all in Slack all the time. That is really our primary interface and how we work together with each other, how work gets done within a team. and across teams. Increasingly, we are bringing some of our customers also into Slack using this feature called Slack Connect, where they can actually work with us directly on some project. And that's kind of the communication part that's changed considerably with Slack at its core. And the great things about Slack is you can basically bring in your agents to come do work on your behalf. And we'll get into more of that. But at a high level, our world has changed in every aspect, how we design products, how we work with customers on iterating on what it is that they need to do, that we need to build, how we get feedback from them, and then how do we break that into tasks?

7:12And then how do we get agents to help us with some of these tasks? I think broadly in one sentence, I'd say we've had people change how they think about work. And that change is, in essence how do you manage a team of agents versus doing the work by hand if that makes sense yeah i used to think i had a retention problem turns out i had a messaging problem i was sending the same onboarding emails to every new use whether they activated on day one or never logged in again i had no idea who was slipping or why customer.io changed that every message i send is now based on what users actually do in the product someone hits a key activation moment they get nudged to the next one.

7:54Someone goes quiet, they get a different path entirely. Their AI agent makes it fast. I describe the campaign I want and it builds the full journey form. Triggers, timing, copy, even branching logic. And when I want to know how something is performing, I just ask the agent directly and it tells me what to do next. They also have an MCP server, which means AI tools like Claude can see directly what's happening in your customer.io workspace. Your segments, your customer data, your attribution, all of it. So instead of explaining your business context every time you need help, quad-arting, no-sing.

8:24Notion used customer.io to personalize their onboarding and hit nearly 50 % open rate, improved conversion by 6-7 % with localized campaigns, and pushed open rates up another 20 % through A-B testing. The idea is simple, customer.io helps you deliver more impact from every message you send. If you're a PM or founder and your onboarding is still one-size-fits-all, try customer.io at customer.io. Here's the problem with most AI tools. They just sit there waiting for you to prompt them. You do all the remembering, they do the typing. Mari from Ariso flipped that for me. It connects to my email, calendar, Slack, and it works the other way around.

8:59It comes to me. Monday morning, there's a briefing waiting. Before a call, it reminds me what we covered last time and what I promised to send. When a follow-up slips, it nudges me before the other person has to. Every commitment, every thread, every contact, tracked and organized without lifting a finger. And because it has continuous memory, it actually gets smarter the longer I use it. It's not a chat window. It's one AI partner that knows my whole working life, and it keeps me prepared for all of it. If you're tired of being your own chief of staff, try Ari at ariso.ai. That's A-R-I-S-O dot A-I slash A-A-K-A-S-H.

9:34You mentioned a bunch of tools there, so I want to dive deeper on those tools. And from your perspective at the executive layer, when do you decide, okay, we need to get this tool into the organization? What are the proof points you need to see externally? How do you roll out a new tool? Let's say you talked about orchestrating agents. The new craze right now is GrokBot or HermesBot mode. So when you're evaluating, okay, do I need to bring GrokBot into my organization? What is the thought process you're going through? Yeah. So firstly, I think we have to acknowledge that tools are moving at a rate that is unprecedented.

10:13There's no doubt about that. What is that? That implies that one cannot make a bet forever, right? I think you need to make bets and you need to give them some space to play out. And then you need to back something and push that sort of tool and technology through, but remain open to change. As an example, we started off using Cursor. That was the primary sort of interface of choice for engineers. Then Clawed happened. And, you know, this sort of family of agents doing work, talking to each other, getting it done, in particular, Clawed code, the harness for coding, you know, became something that engineers loved using.

10:51The way I look at this is you need to give people a choice to pick the tool they want. And often the best engineers, the best PMs pick the best tools to get the job done. And then you look at data. Then you look at what's actually driving adoption more. And then you lean towards the best practices which emerge from just real usage on a daily basis. So it sounds like you're of the camp that don't prescribe, okay, guys, we're all going to use Cloud Code now. Give people access to multiple sets of tools, whether that's Codex, Cloud Code, Cursor. If there's something new that comes up, give them access to that.

11:32And don't be super restrictive on your IT corporate security policy. Is that fair? Be very, very careful about corporate security, but be open about the tools that are given to users as a choice. So every single tool in the Salesforce ecosystem has to go through security. So we make sure that all the security criteria is met, but we give our teams a choice for a while. And then we watch to see what works. And often the best practices emerge from real usage. And as I said, things keep changing. It used to be that the IDE of choice, I don't think it changed in a decade from an engineering perspective.

12:14The tools that are PNs has used, Excel, Docs, Google Docs, Figma, these tools haven't changed a whole lot in the last decade or so, but now they have. And not only have they changed, the change isn't singular. It's going to keep evolving. So I think you need to think of not just where you are on the graph, but what the slope looks like, and try to hug the tangent as the slope changes continuously. And the way to do that is to have a process and a methodology and some trust that your teams will be able to figure it out. So long as you give them access to the best tools that are available and then watch what works, what doesn't.

12:54And that sort of served us quite well, actually. One of the most interesting things that I think Salesforce has done recently is this idea of being headless. And you guys just recently announced Cloudforce. Yes. So basically, where your users for your$60 billion revenue or$50 billion revenue business, they used to live inside your tools. They used to live inside Salesforce.com. But now increasingly, they're living inside Claude or like you mentioned, they're living inside Slack. What does that transformation look like on the ground for people internally? How do they make that shift towards living outside of the SaaS products they used to buy and then more living inside of these centralized operating systems, whether they're Claude or Slack?

13:41Liberating, in one word. I think a lot of the Salesforce magic was never in the interface, although we did that well. It was always in the layer that powered the interface. So when we say headless, all of the metadata, all of the business logic, all of that is actually captured in the API. And when Claude came along, it's really an unshackling of these subsystems that do powerful things, understanding what the flow of sales process looks like that ends up with a 70 % probability of a close or a ticket resolution looks like that has an 80 % chance of being resolved successfully with this ability to bring in a human at the right time.

14:22And these are powerful workflows that have been perfected over decades with Salesforce and its customers. They've customized it for their own needs. What Headless does is it lets our customers bring that into Cloud using skills that we wrap these around and basically use it in any context that they want. So Claude is doing this bit of bringing all of these things together into ClaudeForce with this ability to build an interface that is very custom for their need. Again, I think the magic was never just in the user interface. It was what was underlying the user interface. And what Claude basically does is it helps you code one up for your bespoke need, which is very specific to what you want to do, and you're basically using the same technology in the back.

15:14So what I want to do is kind of build people a recipe for becoming more AI native. So we kind of talked a little bit about you need to hug the tangent of how tools are adopting. Then we talked about you need to let people operate inside these centralized tools like Claude and Slack. And the first sort of thing that I worry about when somebody's using Salesforce in Claude is that it's not really as deterministic as Salesforce was. If you think about it, the way Salesforce is identifying how much open pipeline you have, there's clear data fields that it's referencing. But we all know using LLMs, they can be probabilistic.

15:53They can have variations. But you can't really have two pipeline numbers, right? You can't have 25 million and 20 million. Which one is it, right? You actually need to know. So if people are moving into these tools, how do you build the harness in the right way? Or how do you use Claude or Slack in the right way? so that you can get to a ground truth. Yeah, so with AgentForce, we saw this need arise just as soon as we launched, which is this need to bring in some degree of control and determinism to an agent's specification. What does that look like? It looks like a language that you can describe your agent in, which then translates into tool calls.

16:37It translates into context lookups, and it translates to reduced hallucination because it's relying on tool calls. We designed a language around it called agent script, and basically it transpiles to plain English. So you can describe what an agent should do in plain English. It transpiles to agent script, which then has all the logic and the metadata for tool calling for getting the right context in the right place. that reduces solutionation rates by a massive, massive amount because you're not using Cloud to generate code when there is already a piece of code in an API that is exactly what you want deterministically every time, try, test it, and good to go.

17:22You are still using Cloud to bring the right systems together for that sort of task, right? But that's kind of the magic, which is in the agentic world with agent force, you can bring in more than that. You can bring in a set of controls with AgentScript to be able to pull all of that together. So in our recipe card, it sounds like what's really important is giving the connections to the right data. But on top of that, defining how those tool calls work and building the right skill files so that when Claude makes those tool calls, it does it correctly. Perfectly. Perfectly right. One other thing I would add, which is a really important insight that we gleaned away, is if you look across my teams, we have teams that think eval first by nature.

18:13These are all the data science teams that have been doing predictive AI and generative AI, and then into the agentic world. For them, they are what I call eval native. They look at a system probabilistically. They understand that it's not going to work all the time. So the test cases or evals that they've written tied to outcomes is a superpower now. And there are other teams that are more sort of code-focused, engineering-focused. They've built the infrastructure. They've always worked in the deterministic world. And very quickly, earlier you asked, what are the big changes that you're seeing with the teams?

18:50This is one of the changes we're seeing. We're seeing a lot of people becoming eval first. and that means thinking through not just the test cases but outcomes and having these custom scorers using both llms and other models to score the output and these are not generic these are very specific to the agent you're building or the code you're writing this is the other sort of this is a cultural shift it is a tool set shift it is a mindset shift in how you sort of design software that is probabilistic and you want it to work every time. So I'd add one more, which is the eval part around it. Let's circle back to evals in a second.

19:28I want to close the tools component of this with Slack. Yeah. Because I'd say a lot of people listening who are trying to come up with an AI native organization, they're probably still stuck in the Microsoft Teams world. They don't even understand why Slack is important or what it enables. So can you talk to me about, as far as I understand it, people are spending, like you said, 50, 60 % of their time in Slack now, whereas it might have been in an IDE before. So how do you get that setup correct? And what is involved in that setup so that you're doing so much out of Slack? That's a great question.

20:08I use Slack every day, and I am just a mad fanboy. So two things. One is, what's the one place that people are available asynchronously all the time? It's Slack. In most enterprises, that's where people are. That's where work is organized. That's where conversations around work are organized. That context is gold dust. especially taken longitudinally over the long period of time that just is the amazing bit of missing piece that's needed for getting something done. Serendipitous discovery is possible in such a system. Slack will routinely find someone that has that answer in some channel that is a public channel that I had no idea existed and that's context for an agent to work on.

20:58So what does the tool set look like? It looks like a great implementation of Slack within the organization. It is a Slack where people are using it every day, which is super high utilization, which is leaving a lot of data traces behind. And then it is plugging in this agentic tool set within Slack. We ship with one that is magical called Slackbot. If you haven't used it, please do. I want to take a second to talk to you about the fourth cohort of LAN PM job. I trained 30 students in cohort one 50 students in cohort two and 75 students in cohort three and i am bringing back a program for cohort four it starts in august and it lasts three months where you're going to have intense sessions a monday morning session where i go over your resume behavioral interviews linkedin on top of that bart jaworski is going to be teaching you the pm fundamentals in 2026 how to write ai prds how to ai prototype with cloud code all of the key skills you need to freshen up your knowledge for this market.

21:58And Ankit Vermani is going to be teaching you AI product management. He is an AI product manager at Uber, and he is going to teach you how to build AI features that actually work successfully. On top of that, Prasad Reddy is going to be doing one-on-ones with you for market reviews, LinkedIn review, candidate market hit review. So it is a full package. It is three courses in one for one low fee. So join at landpmjob.com. Agents and assistants are browsing, shopping, and transacting alongside humans, while fraudsters use AI to build faster, cheaper, and more sophisticated attacks. Technical teams need device intelligence that evolves as quickly as these attacks do.

22:36Fingerprint is exactly that. Accuracy is the number one reason customers choose fingerprint, because their device intelligence goes deeper than identification alone. Resolving visitors with precision that holds up through spoofing and evasion attempts for months at a time. Fingerprint collects more than 100 signals across devices, network, and behavior to build a single identifier. Then provide signals like bot, agency, proxy, and VPN detection to give technical teams a complete picture of risk. They ship hundreds of product updates a year, so they adapt first as browsers update, privacy requirements change, and fraud tactics change.

23:11With one API call, teams can get this running in minutes. Over 6 ,000 companies rely on fingerprint every day. Join them now at fingerprint.com. Imagine learning AI product management, AI product strategy, AI product leadership, advanced PM with Claude Code, all from frontier leaders at OpenAI, Anthropic, and Google for less than$10 a day. Actually, you don't have to imagine it. That's exactly what product faculty's AI Builder Fellowship gets you. One yearly membership, six live cohorts. Not recorded videos or another content library you never finish. Real live sessions with Frontier AI leaders, hands-on AI build labs, executive insight sessions, capstone projects, unlimited retakes, and continuous access to new programs as the AI landscape evolves.

24:06You'll also get every new certification they add while you're a fellow at no extra cost. The goal is simple. help you master frontier AI skills to become an AI native product builder and operator, not just someone who talks about AI. Purchased separately, these programs are worth more than $19 ,245. The lowest price certification alone starts at$2 ,700. But founding members can join the entire fellowship for less than$3 ,600 per year. That's less than$10 a day. Soon, the price increases to$5 ,000 a year. So if you're serious about becoming the person your company turns to for AI, this is one of the highest ROI decisions you can make this year.

24:46Join the AI Builder Fellowship at productfaculty.com. You can ask it questions. It can orchestrate other agents for you. It can bring you the right people. It can send emails. It can send other Slack messages. So you're basically building your agentic environment, work environment within Slack. and you can build it to being in third-party agents as well. So that's sort of the tool set. From a developer perspective, we use Slack code quite a bit. For simple apps, you can write the spec there and you can actually build a simple UI in there. How cool is it for a PM to be able to go create a small mock in a Slack thread with their customer and just ask him, is this what you had in mind?

25:35Versus writing a PRD, getting an engineer to go build it and then create a link for it and send it back to the customer. The customer kind of looks at it and goes, eh, not really what I want. So basically this whole cycle is sped up exponentially and it's just a joy to work. So I'm trying to build this recipe card. I feel like at the base we've covered that tool layer. Right. The next layer is roles and you just talked about it, PMs and you also referred to it with evals. So a lot of people are wondering, what is the future of the product trio? the PM designer and engineer roles they all seem to be converging on top of each other so we humans are notoriously bad at predictions but what are you seeing on the ground right now are you seeing when you're staffing up a product team that you need as many PMs as before as many designers as before how are you thinking about that well that's a that's a big question and I think it's a really important one I having been an entrepreneur for I think the one thing that becomes very clear to you when you are an entrepreneur is that these lines that we draw are, you know, don't really exist.

26:41I think we create them in our own heads and then we specialize around them so then they do become real after a point. I think what the tool sets that we have available to us, what the large language models and the agents can do effectively now means that those lines are being blurred. They're being blurred in ways that are incredible to see. As an example, if you're a PM, if you're an engineer that has a customer first mindset, and many do, many don't want to build systems that are never going to see light of day. They want to build systems. They want to put their heart and soul into things that has an impact that customers will use every day.

27:22How can they test that? Well, if they have customers that they can work with directly, they can start putting products out in front of them, get direct feedback. Do you need a middleman? I don't think so. Conversely, if you're PM, your bottleneck may be waiting for that engineer. You've logged in the request for that engineer written a PRD and the engineer is working on something that was written before and you're waiting. You don't have to wait anymore. The product development lifecycle can be sped up significantly more. And now the bottleneck is, can you find customers fast enough and can you get to customer success, concept market fit, product market fit quickly.

28:05And one of the great things about working at Salesforce, I truly believe this, and you've seen here the energy here at Dreamforce, we just have incredibly loyal customers and we couldn't be more thankful for that. You've seen the trailblazers that have come here. And who are these people? These are people that started out in roles that were sort of fairly junior and implemented Salesforce and then grew with the company in implementing this. And now our customer service officers of the company, tremendous loyalties. And it's really easy for us to go talk to them because of the trust that Salesforce has built over decades and say, hey, will you try this?

28:47We've built something. We want some feedback. Will you try it? And it's super easy for us to get 10 pilot customers for anything we do. and we have that trust that once we build it, we will build it right and we will make them successful with it no matter how long it takes. So I think that sort of bottleneck, you know, where you need to go find customers is completely non-existent in Salesforce. So then what's the new bottleneck? The new bottleneck is, are you able to cook with them fast enough? Are you able to get feedback fast enough? Are you able to test the concept fast enough? And yes, you have Slack.

29:23They can join your Slack. We can start to build simple demos in Slack. And if they love it, all you have to see the hard signs, count them, and go, great, let's go get this built out. And then we have 10 pilot customers, and then we go from there. So just five years ago, that process took weeks, if not months, right? Executives and product managers would go off, and they'd figure out, this is the direction. Once they got buy-in from everybody, designers would create the prototype. Then it would finally get put in front of customers. Usually a user research team would be responsible for putting that in front of customers.

30:04So we had like three specialists touching a prototype and putting it in front of customers. Now engineers can do that. Should engineers be doing that? When should you have a PM or a designer or an engineer be doing these different tasks? I mean, ideally, we have a group that is doing it with customers. It's faster. It is more durable. And they each bring their perspective to design. The point is that the tools make each of them operate in each other's domains up to a degree. There is, of course, specialization. This ability to abstract out requirements by talking to four customers, having insight, having looked at the market, where the industry is going.

30:52That's something that the PMs, especially an experienced PM, can really bring to the table. Conversely, an engineer can just build it right the first time and now super fast, given the tools that are available to them. Same with designers. So we each come from a place where there is a strong skill set and an experience that we bring to the table. What the tool set's doing is that it's saying that you're not in that silo anymore. You shouldn't box yourself in that silo. You need that you can go take the first step with the customer and then pull in the expertise as needed. A good example here is that often you build something quickly, you get feedback on it quickly.

31:36The journey from zero to a thousand customers is a very different journey from a thousand to ten thousand customers where failure modes have a significantly bigger impact when you have ten thousand customers. So that journey is very different and you need different skill sets to be able to bring that into the fold. Would you over-engineer something for 10 ,000 customers day one? You wouldn't. So what this does is it basically increases the pace at which we can innovate with our customers. Innovation comes from two places. It can come from, you know, groundswell, bottoms-up innovation based on new technologies that are coming.

32:13God knows we've seen enough of that. But when that's married with the customer need that exists, which you can quickly test with customers, and that's where the magic is. And those two things kind of come together. So an engineer that has a very deep understanding of tech and has an opinion on where this is supposed to go, doesn't quite know whether the market is there or not, can test it out. And can they bring a PM along? We're sort of designed that way. We have a scrum team. The smallest unit of work in our organization, my team, is a scrum team. You have a PM, you have an engineering leader, a few engineers, you have a fractional designer, and they can go together and go get this sort of cycle sorted out pretty quickly.

32:54Okay. So we still have that traditional unit of a scrum team. As you're making the shift to AI native, do the ratios change or do the job descriptions change? It sounds like mainly the job descriptions change in that you're more in each other's lane, but the ratios are the same? I think it's early days. I don't think ratios have changed a whole lot. I don't think I've seen that happen. I've seen work bleeding across a little bit, which is great. And I expect to see more of that. What I really look for is the pace of zero to one. How quickly can we go from an idea to an implementation of that idea?

33:39and is it done in vacuum? I can go from an idea to implementation really, really quickly just by myself on my laptop with no feedback from anybody. Not a great idea. Don't recommend me do that, right? But if you're going to work with a team, if you have a set of customers, how quickly can we iterate with them? And I think our velocity has gone up significantly. We are producing more software than we ever were. And this is kind of a direct result of, teens using the tools, having a democratic approach to the tools, and then bringing customers into the fold so you can integrate fast? So what I've been talking a lot about in my newsletter and podcast is how a lot of PMs need to become AI PMs.

34:23AI PMs, compared to a regular PM, have two really important skills. Number one, AI prototyping, which you just talked about. Instead of waiting on designers and engineers to create prototypes, they prompt the AI themselves. In a Slack chat with the customer, they directly get feedback even before they're sharing it with a designer or engineer or UX researcher. And the second skill is AI evals, which is that they really become the subject matter expert to define, hey, this is our golden data set. These are the 100 input prompts. This is what I expect the output prompt to look like. It should be this quality.

35:05This output prompt would not be a quality. This would be an 8 out of 10. This would be what a 10 out of 10 output prompt looks like. Would you say that's accurate? To become an AI native org, do you need to hire these PMs that have these new skills? And are those the right skills? You are speaking my language. This is exactly the two things that are absolutely key. doing that up front you know really is the magic bullet I think those I mean I couldn't agree with you more is the short of it I think these two skills are invaluable in PMs so yeah short answer is yes we need to do more of that so we've built the tool layer people now understand the roles to close off the role layer how do you hire this new AI PM what are you looking for one is nothing to do with technology it's agency which I think is going to be the new short supply thing and beyond that I would say just a deep familiarity with agentic tools and just be very agent native what I mean by that is a mindset where you're not off doing the work but you're sort of teaching an agent to do the work you maybe do it once and then teach an agent to do it so that from then on forward, you are super fast in how you're responding to things and iterating on things.

36:32So that's one sort of skill set to look for. Second one is some degree of skepticism with the tool set, where you love the power it brings, but you don't always believe it, right? The best ones still have that craftsmanship mindset. They're not producing AI's law. They're putting the name to it, and they do not like to put a name to something that has AI slop. AI slop comes in many different forms. You know, a prototype that looks like any other prototype, AI slop. People just checking the box by sending a really long document, AI slop. So these are sort of clear do's and don'ts. I think there's no substitute for craftsmanship.

37:19There's no substitute for agency. I think those are two really important things to look for. Like, do they have attention to detail is a good example of craftsmanship. And then it's tooling. It is, how do you translate that skepticism into test cases and evals that are failing when you build the system that you're building or testing the system that you're testing? Can you set a bar on what good looks like? Because that's what translates into evals. And then, like you said, I think access to all the prototyping tools, which I think is the easy bit now. I think that pretty much everyone gets. Yeah, I think the tools are pretty easy.

38:02The roles we've given you some more clarity on. I think that third layer now is the life cycle that all of this is under. And you created this new term, ADLC. Yeah. Not the SDLC that we all know. So can you define those two terms for us and explain to us what makes the ADLC different? Sure. Sure. So SDLC is the software development lifecycle, which we all know and love that have been building software for years. ADLC is an equivalent for the agent, the agent development lifecycle, if you will. What is the life? If the software you're producing is not just a deterministic piece of software, but is an agent.

38:42What does how do you build it? How do you test it? How do you evaluate it? How do you deploy it? And then having deployed it, how do you observe it so that it continually improves? So these five elements are taken together is what I call the ADLC lifecycle. The holy grail, of course, is an agent that you've built and it's in production and can self-heal to some extent. And I'm not talking some fancy rocket science sci-fi stuff on self-healing. I mean simple things like you have an agent that's working fine in production, starts to drift a little bit. Why does it start to drift? Because it's grounded on some data on refund policies that just got changed.

39:25But somebody wrote it in a way that it's not very clear. Somebody updated the document, the agent's not working, the refund, or you're sending refund checks that are bigger than the amount that was needed. Can it self-heal? It can. It needs to detect what context was changed, whether that context was changed the right way or wrong way. that's the knob that you want to get to you want to get to a point where if you've designed this right there are two loops there's the inner loop which is you conjuring up an agent from your imagination working with the customer just to make to make sure that that's what they want and then going and building it off and then deploying it in that environment and the outer loop that journey starts once the customer starts to use the agent what is the life cycle once an agent is live?

40:14How do you evaluate in production? How do you update the agent in production? What does that loop look like? That requires a completely new set of tools that haven't been built or imagined. We're starting to see some of that come in, but in our world, that looks like a testing center for agent. That looks like an observability center for agent. That looks like we launched this at Dreamforce yesterday called Optimization Agent. It's the one that looks at all the agents in your agent force agents that are running in the org and look for opportunities to optimize. Maybe by adding new context or refining a prompt that is not working great.

40:55So think of this agent as like this agent that's observing all the agents and like looking to optimize. So that's kind of what I mean by the ADLC lifecycle, kind of playing on words with the software development lifecycle. Does the ADLC only apply then if you're building agents? I think it's broader than that. I think it has to do with an AI development lifecycle, which has an eval mindset baked in, which understands that anything built on data can drift because data itself drifts. And then having the right tool sets to observe that drift in production and see if that drift can be corrected in production.

41:37Large-scale AI systems predating agentic systems have this in rows. Like if you look at the search algorithm that Google runs or any other company runs, a recommender system that runs, they have all of these knobs built in and pre-built and tuned. We are pulling all of that out and making it available at scale in this new agentic era because everyone's building agents for doing things. and now they need this skill set to be able to evaluate these agents, to be able to make sure that they work, to be able to observe them, to be able to understand what is drifting, to be able to be told what's drifting and then bring the human in to correct it, to steer it, if you will.

42:18So it sounds like part of what is involved in the ADLC is moving a lot faster. If you were on a six-month ship cycle, it sounds like you need to almost go to a continuous deployment it, it can self-heal whenever it wants cycle. Yeah. And I think there's a very good reason why the pace is this way. It's because often the substrate layer, things that you build on, don't change as fast as they are today. I think about it this way. We've built software for decades and the substrate layer for most software is a database. How often has the database changed? How often has the programming language to program the database changed?

43:03Hardly ever. SQL still exists. We are in a world where the substrate layer is a model itself, and it's not staying still. It is evolving at a very rapid rate. The changes between what we saw in the current version of Claude and the previous version of Claude is significant. So what happens to all the agents that you built that worked great on the previous one. You need to evolve them. You need to simplify them. And there are certain dimensions on which you need to simplify. Imagine being able to do that with no ADLC. You have no test cases, no evals. How are you going to go flip the... You're just going to...

43:47If you didn't have all of that infrastructure built, the rate at which you move, the hugging of the tangent, if you will, it's not happening because you're mortally afraid of breaking something. But if you do, on the other hand, you can really go fast. And that's the mindset change. Everything downstream has to change though then, right? Your product marketing or customer support, everybody has to shift to this much more agile way of working. Some companies are still stuck in the past, right? They're still shipping every couple months. How does a company go from zero to 100? it feels like? How did they just completely turn around and how did you guys do it?

44:29Yeah, I think the first thing is, I think you nailed it. You have to be building and releasing continuously. You have to be doing that in discrete chunks that matter to customers. At the end of the day, we're shipping something that we want to have impact with customers that adds value to customers. Release is just a packaging mechanism, is all it is. from every Scrum team needs to be pushing code and making changes to code as rapidly as they can, as safely as they can, right? This is the trade-off. The trade-off is you don't want to break systems, right? If all you're going to do is iterate rapidly, once you have the kind of customer base that Salesforce has, things start to break, and then that slows you down even further.

45:16You want to be able to balance and do both. Yeah. People used to create lots of tech debt when they would switch to this way. Yeah, and often I tell our PMs, if you have tech debt on your teams, that's still your problem because you're going to eventually slow down and you won't know why because your engineers are fixing problems. And that means every engineer that's fixing a problem is not adding a new feature that the customer loves. So taking that into account, you have to account for what that means from a tech debt perspective. But for the first time, we can have agents help with tech debt too.

45:52right like how do you build this like this work we do with with our amazing security team there's work we do with our release management teams for building agents that are specialized that do code reviews what's how do you pull a pr a high quality pr really fast um what are the bottlenecks the bottlenecks bottleneck is no longer writing code the bottleneck is writing quality code, writing code that works all the time, having the test cases that fail before it hits production, right? These are now the bottlenecks, each of which, if you go really deep, you can start to build agents for and speed up by itself.

46:34So, yeah. Okay. So, I think we covered the lifecycle. I think the very last component at the very top of the pyramid is probably where you spend a lot of your time. how engineers, product managers, and executives who are really setting the strategy of what we're working on collaborate to build a roadmap. So what does that process look like? What does the planning process look like in this AI native organization? What do the different roles own? Great question. So let me take a step back first. I think at a high level, I'd say taste, judgment, play a much bigger part than they did before. That's what forms the intuition on the broad vector along which you want to move the organization, the team, the product.

47:22But I said broad vector. It's not a straight line, right? It's a broad direction. And then once you know what the broad direction looks like, you get a lot more refined using two mechanics. Mechanic number one is just getting customer feedback early in whatever it is that you're building. And that feedback, as we determined earlier, isn't that hard to get anymore in a very visceral way. You can just put a prototype in front of them now. You don't have to have seven meetings where you're describing the idea. That is one dimension of getting that vector right. The second dimension is existing products that are in the market that are being used by customers and they're loving it, but they have some gaps.

48:08this is where sort of our forward deployed teams are incredibly useful in how they work with customers, not just to make them successful, but they are the first line of defense and see what our customers see. And because they're closer to product and they're part of the product, and I have customer facing engineers that work directly with customers, they bring back all kinds of feedback from paper cut type things that shouldn't exist in the product, we should just fix it to, oh, the way we thought about this particular feature doesn't seem right because I work with three customers and they seem to be struggling.

48:45So yes, we did assign that vector the right way by talking to customers. But when it meets the real world, there are some gaps that need to be tuned or maybe some things need to be thought through. Now, imagine doing this at scale across many, many, many, many different teams. And you get a picture of what, you know, how that sort of looks like. But I feel like those are key things that we need to organize as a team. And from our perspective, as I said, we are in Slack all the time. We have great tools that summarize all of this, roll this up. We have fantastic reports that show up. We have a channel where all FDEs basically post exactly what's happening to the customer, what their pain points are, and is it a paper cut or is it a feature request.

49:32and we have PMs and engineering leaders looking at it all the time and matching it against that roadmap and seeing if something needs to change or we need to jump onto the customer side. We are working more with customers than we ever did before. I want to return to that role there because you brought up the forward deployed engineer. It's got to be the hottest, sexiest new role in tech. And for my audience, it's the forward deployed product manager that's super interesting to them. What do you see? Are these roles replacing the traditional? Are they a supplement? How do they fit in to the broader picture we just described of engineer, designer, PM?

50:14Yeah, I think the reason the role exists is because we are deploying sophisticated technology and evolving that technology at a very fast pace. The pace at which we're building is not the pace at which the technology is being diffused. So we're solving it with expert humans in the middle. We are trying to catch up our customers and we're trying to catch up the diffusion rate of technology with the customer base in a way that adds value to them so it's righteous in that way, to the pace at which we're innovating within the company. That's the gap. And that's what we're trying to fill with technology.

50:56with forward deployed engineers, PMs, designers, whatever. Broadly, I think what that means is the main skill is twofold. One, really being able to immerse oneself into the customer's pain point. They bought us to solve a problem and then falling in love with that problem and then helping the customer solve it using the tool sets that are available to them. And often, maybe the tool sets are not sufficient. That's okay. We should have the forward deployment team has the freedom to go find some other bit and be able to plug it in to make the whole work. And we've done that many times. Sometimes we build something custom with them and then big that into product, right?

51:36So do we need PMs there? Yes, because we just designed something custom with them. And it'd be nice to have PM input on that, which has the added benefit at roadmap planning to see whether that thing was really custom or do we think that all customers would need it? And if all customers would need it, how about we take a prototype? We already have one and take it out to a bunch of customers, get feedback on it. Right. So this sort of cycle of getting feedback in, bringing in PNs, designing in the right way with a designer or an engineer, just like working with a customer to go get it built out, making them successful is really the lens that is different.

52:16So what I'm hearing is that the forward deployed group is almost like doing a lot of this rapid prototyping on a custom scale and then it brings it back to the core group to decide if we want to build it in the product for everyone. Is that right? That's right. Okay. So you get to see a lot of customers who are going through this transformation from not AI native to AI native. They're basically hiring Agent Force to help them along that journey. What are the biggest mistakes that somebody who's watching this should keep in mind so that they can avoid those in their implementation? I'm a big believer in start simple.

52:55I'm a big believer in define value, then define metrics to capture that value and call it your ROI metric and get it built out end to end, scale it out, scale one, and then scale all. Often implementations don't take this flavor. It starts off as a whole full company transformation story and then very quickly, you know, has, you know, breaks out into this massive plan across the board and creates a lot of pressure on the teams that are implementing, even customers that are trying to derive value. So I think, you know, one piece of advice I have always is start with your hardest problem. That's okay.

53:36But be very clear about what value is going to look like and then break it down into, we've talked about this before, clear ROI metrics, clear scorers on the agent, clear evals on the agent that tie to value and ROI. And once a team is successful at building it, keep in mind, this is technology that we have kind of built ourselves, so it's easier for us. But when I look at others building agents, I get the sense that technology isn't fully diffused, the teams that are implementing still need a lot of enablement. And what that means is starting with one success under the belt is super important.

54:18I'd say that is one key thing. The other one I'd say is there are a lot of variables when it comes to building the system. What models do we use underlying? Should one train their own models? Should one start with the harness, what is the role of implementing the right context? There are a lot of variables. And I'm also a big believer in saying that the very first implementation, minimize the variables, get the work done in the simplest way possible, or camps razor, and then start to ask about what are the ways to make it more efficient. And there are many, right? The best ones that we work with start with two or three really critical implementations, get them right, and then they just go wall-to-wall with Agent Force.

55:08Deco, they were in Mark's keynote, and they're one of the customers I work with very, very closely. They're wall-to-wall Agent Force agents, all the way from a recruiting company based in Europe, one of the largest recruiting companies in Europe. They have pre-screening automated with an agent that calls you, and you can take an interview on a phone call. keep state, can ask questions, escalate to human, all of that baked in. They started with text, wall-to-wall, 17 different languages. They're deployed across Europe, and now they're going voice. So that's a good example of what setting a high bar, but starting with one or two use cases and really going all the way and then expanding and going wall-to-wall looks like.

55:53This has been a masterclass. Thank you. if people want to learn more about building an AI native product team and organization, where can they find you online? Online, I am on LinkedIn. You can find me on LinkedIn. I'm on Twitter, Jay. And of course, you can find all about Agent Force on Salesforce.com. It's where I live pretty much all the time. All right. Jesh, thank you so much for being on the podcast. Absolutely. My pleasure. All right. Okay. See you guys next time. I hope you learned as much from today's episode as I did. If you can do one thing that's totally free that would help the show, it would be to check that you're following on Apple and Spotify podcasts.

56:31Check that you've left ratings and reviews on those platforms. Check that you're subscribed on YouTube. Leave a like and a comment on this video and then share it with your friends. We're trying to make better and better podcasts. After two years, we think we've gotten something pretty good going. So let us know what we can do to make it even better, who else we should interview, and we will put on the best shows we possibly can. Finally, don't forget my offer for the bundle, you get an entire year of my paid newsletter. Plus my favorite AI tools, Bolt.New, Airtable, Speechify, Descript, Magic Patterns, Linear, Dovetail, Arise, and Mobbin.

57:08That's$27 ,000 worth of value for just$150. So check that out at bundle.akashi.com if it interests you. And I can't wait to share our next episodes.

57:27Thank you.